Uploaded image for project: 'mod-data-import'
  1. mod-data-import
  2. MODDATAIMP-625

Importing a file of 25,000 authority records does not complete

    XMLWordPrintable

Details

    • 3
    • Spitfire
    • Lotus R1 2022
    • TBD

    Description

      URL: https://metadata.ci.folio.org or Lotus Bugfest
      Metadata Login: diku_admin / admin or Lotus Bugfest

      1. Go to data import
      2. Import a file of 25,000 authority records msplit00000003.mrc
      3. Use the job profile: Default - Create SRS MARC Authority
      4. Hit Run

      Expected outcome: 25,000 authority records are created and job completes

      Actual outcome: 25,000 authority record job never completes. It hangs on 99%

      25,000 record is attached.

      Another issue identified by Roman Fedynyshyn

      During Create MARC authority import we have got error on mod SRS 

      [vert.x-worker-thread-10] ERROR rdChunksKafkaHandler [72160511eqId] RecordCollection processing has failed with errors with event: 'null', chunkId: '6c96f806-e9f5-4720-b680-b44361a3a0e7', chunkNumber '2119'-'19' with recordId: 'null'

       

      Steps to reproduce:

      using msplit00000001.mrc start import MARC authority

      TestRail: Results

        Attachments

          1. 25k_completed_with_errors.png
            25k_completed_with_errors.png
            62 kB
          2. 25k_still_running.png
            25k_still_running.png
            72 kB
          3. db_errors.txt
            13 kB
          4. errors.txt
            5 kB
          5. image-2022-01-27-16-48-01-913.png
            image-2022-01-27-16-48-01-913.png
            340 kB
          6. msplit00000001.mrc
            12.89 MB
          7. msplit00000003.mrc
            12.45 MB
          8. records_count_difference.png
            records_count_difference.png
            21 kB
          9. testing after fixes.png
            testing after fixes.png
            15 kB

          Issue Links

            has to be done before

            Task - A task that needs to be done. MODDATAIMP-642 Update wiki config and performance documentation with Lotus info

            • P2 - normal priority level, must be included in the current development cycle
            • Closed
            is blocked by

            Bug - A problem which impairs or prevents the functions of the product. MODSOURMAN-706 Error loading MappingParametersSnapshot (Kiwi HF2)

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Task - A task that needs to be done. MODSOURMAN-712 Timeout when importing MARC files - Lotus BF

            • P2 - normal priority level, must be included in the current development cycle
            • Closed
            is duplicated by

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. UIDATIMP-1087 Investigate rdChunksKafkaHandler ERROR

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed
            relates to

            New Feature - New feature requests UXPROD-3333 Lotus release -Spitfire - Technical debt

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed
            requires

            Task - A task that needs to be done. MODDICORE-250 Release v3.3.1 (R1 Lotus Bugfix)

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Task - A task that needs to be done. MODINV-653 Release v18.1.1 (R1 Lotus Bugfix)

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Activity

              People

                viacheslav_kolesnyk Viacheslav Kolesnyk
                kgambrell Khalilah Gambrell
                Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases