Uploaded image for project: 'mod-data-import'
  1. mod-data-import
  2. MODDATAIMP-588

Spike: Investigate possibility of removing Kafka cache

    XMLWordPrintable

Details

    • Folijet Sprint 129
    • 5
    • Folijet
    • Lotus R1 2022

    Description

      There are CPU spikes related to events_cache topic and Kafka cache memory is not released for a long time. Investigate possibility of removing Kafka cache considering results of MODDATAIMP-500 spike. See described problems with Kafka cache MODINV-444 and MODINV-401. 

      Create necessary stories.

       

      TestRail: Results

        Attachments

          Issue Links

            defines

            New Feature - New feature requests UXPROD-3261 NFR: R1 2022 Lotus Data import performance work

            • P2 - normal priority level, must be included in the current development cycle
            • Closed
            is duplicated by

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MODINV-401 Spike: KafkaCache Memory not released (after a long time)

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Bug - A problem which impairs or prevents the functions of the product. MODINV-444 Spike: CPU spikes related to events_cache topic

            • P2 - normal priority level, must be included in the current development cycle
            • Closed
            relates to

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MODINV-588 Implement deduplication for Instances

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MODINV-589 Implement deduplication for Holdings

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MODINV-590 Implement deduplication for Items

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MODINV-591 Implement deduplication for Authorities

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Task - A task that needs to be done. MODINV-598 Remove Kafka cache from update handlers

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Task - A task that needs to be done. MODINV-603 Remove Kafka cache by handling Constraint Violation Exceptions for DataImportKafkaHandler

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Task - A task that needs to be done. MODSOURMAN-638 Remove Kafka cache by handling Constraint Violation Exceptions for StoreRecordsChunksKafkaHandler

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Task - A task that needs to be done. MODSOURMAN-639 Improve performance of saving journal records during import

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Task - A task that needs to be done. MODSOURMAN-640 Remove Kafka cache by handling Constraint Violation Exceptions for DataImportJournalKafkaHandler

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Task - A task that needs to be done. MODSOURMAN-641 Remove Kafka cache by handling Constraint Violation Exceptions

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Task - A task that needs to be done. MODSOURMAN-642 SPIKE: Investigate possibilities to combine job_execution, job_execution_progress and job_monitoring tables

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Draft

            Task - A task that needs to be done. MODSOURMAN-668 Restructure job_execution_progress table for DataImportKafkaHandler

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Activity

              People

                Nosko Serhii Nosko
                Kateryna Senchenko Kateryna Senchenko
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases