Uploaded image for project: 'mod-source-record-manager'
  1. mod-source-record-manager
  2. MODSOURMAN-338

Change chunk processing to use Kafka

    XMLWordPrintable

    Details

    • Template:
    • Sprint:
      Folijet Sprint 102, Folijet Sprint 103
    • Story Points:
      5
    • Development Team:
      Folijet
    • Release:
      R1 2021

      Description

      Change chunk processing to Kafka approach(can be moved from PoC) https://github.com/folio-org/mod-source-record-manager/pull/315
      Use pubsub lib to create Kafka consumer that would receive records sent for processing from mod-data-import.
      Add support for exactly one delivery for each chunk. JobExecutionSourceChunk can be reused. UUID will be received from mod-data-import. On constraining violations - skip chunk processing and add logs.
      Recieve answers from SRS and start processing in StoredMarcChunkConsumersVerticle and add exactly one delivery for each chunk.

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                OleksiiKuzminov Oleksii Kuzminov
                Reporter:
                Kateryna Senchenko Kateryna Senchenko
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases