Uploaded image for project: 'mod-data-export'
  1. mod-data-export
  2. MDEXP-422

Gracefully handle MARC records that exceed the size limit

    XMLWordPrintable

Details

    • Story
    • Status: Closed (View Workflow)
    • P2
    • Resolution: Done
    • None
    • 4.1.1, 4.2.0
    • None
    • Firebird Sprint 119, Firebird Sprint 120
    • 5
    • Firebird
    • R2 2021 Hot Fix #3
    • Yes
    • via slack on 9/17/21
    • Institution specific (Data)

    Description

      Purpose/Overview:
      In the current data export implementation when the record exceeds max allowed MARC record limit (99,999 characters) the export job crashes with the following error:

      ExportManagerImpl Export is failed, cause: Record is too long to be a valid MARC binary record, it's length would be <projected size> which is more than 99999 bytes
      

      and .mrc file is not generated at all.

      The record might exceed the limit when the exported instance has thousands associated items and the export includes  SRS Bib and inventory holdings and items data.

      Requirements/Scope:
      If the record exceeds the MARC record length, then

      • Record should not be exported
      • The information about the record should be added to the user error log using the standard entry :
        <dateandtime> ERROR Record is too long to be a valid MARC binary record, it's length would be <projected record length> which is more than 99999 bytes
        {
        "Instance UUID": "<affected record uuid>"
        "Instance HRID": "<affected record hrid>"
        "Instance Title": "<title>"
        "Inventory record link": <link to the record>
        }
        
      • Remaining records are exported

      Approach:

      Acceptance criteria:

      • The export doesn't crash and error is handled gracefully
      • Information about the affected record is recorded in the export job log
      • Remaining records are
      • Unit tests are written
      • API tests are updated

      Additional information:

      TestRail: Results

        Attachments

          1. export result.PNG
            export result.PNG
            68 kB
          2. export result - error logs.PNG
            export result - error logs.PNG
            125 kB
          3. mapping profile json
            10 kB
          4. screenshot-1.png
            screenshot-1.png
            71 kB
          5. uuids_test_MDEXP_422.csv
            0.3 kB

          Issue Links

            Activity

              People

                TotoroDev Illia Daliek
                magdaz Magda Zacharska
                Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases