Uploaded image for project: 'mod-data-import'
  1. mod-data-import
  2. MODDATAIMP-315

Use Kafka for data-import file processing

    XMLWordPrintable

Details

    • Folijet Sprint 102, Folijet Sprint 103
    • 5
    • Folijet
    • R1 2021

    Description

      Refactor sending records for processing to publish events to Kafka instead of sending HTTP calls:

      Change mod-data-import file processing to Kafka approach(can be moved from PoC) https://github.com/folio-org/mod-data-import/pull/130
      Create ProducerManager
      Add support for exactly one delivery for each chunk(can be added a unique UUID or hash for each chunk)

      TestRail: Results

        Attachments

          Issue Links

            defines

            New Feature - New feature requests UXPROD-2659 NFR: Refactor data-import flow to increase reliability

            • P2 - normal priority level, must be included in the current development cycle
            • Closed
            is blocked by

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MODPUBSUB-118 Create sub-project in mod-pubsub for utility transport layer classes

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Activity

              People

                Kateryna Senchenko Kateryna Senchenko
                Kateryna Senchenko Kateryna Senchenko
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases