Uploaded image for project: 'mod-source-record-manager'
  1. mod-source-record-manager

Legacy 999 (non-ff) fields cause data import failure (Morning Glory)



    • 0
    • Folijet Support
    • Morning Glory (R2 2022)
    • remove or renumber all other 999 fields
    • Lehigh
    • Legitimate regression


      Using data import to overlay a record with legacy 999 other than the one FOLIO creates with the inventory and srs UUIDs fields fails silently.

      Steps to Reproduce:

      1. Log into FOLIO as diku_admin (affects kiwi-bugfest, and probably snapshot-load)
      2. Import the attached file single_local999_test3.mrc of MARC records, using the job profile: Default - Create instance and SRS MARC Bib
      3. Once imported, check to be sure that the local 999 field (with blank indicators) was retained, and a new 999 field (with ff indicators and the Instance and SRS MARC Bib UUIDs) was created.
      4. Go to Inventory and search for that Instance by HRID (do NOT use the hotlink in the Data Import log)
      5. Go to Actions/View source and confirm
        • The existing 999 with blank indicators was retained
        • A new 999 with ff indicators was created and has the Instance UUID in $i and SRS MARC Bib UUID in $s
        • NOTE:
          • This single_local999_test3.mrc file is not currently importing to Kiwi Bugfest (gets stuck in "Running")
          • On Kiwi Bugfest, if you search Instance by HRID insta7155403, you will see the record when it was previously imported, but the View source option is greyed out.
          • abreaux will check on the hosted ref envs once they come back to life).
          • On Kiwi demo env (https://folio-kiwi.dev.folio.org/), import for this single_local999_test3.mrc file also gets stuck in "Running" - so it appears that even importing to create when there is a 999 field is not working. It previously worked; see MODSOURMAN-204. When I import the file attached to MODSOURMAN-204 using the Default Create job profile (61PRINT160129 also attached here), it imports cleanly in Kiwi BF and in Kiwi demo env.
      6. Close the source record and the instance detail view. That should leave the 1 search result in your Inventory search results screen
      7. Go to Actions/Save instance UUIDs. That will trigger download of a csv file with that one UUID in it
      8. Go to the Data Export app and upload the csv file
      9. Select the Default job profile to export the associated MARC record.
      10. Doublecheck the exported file to make sure that the non-fff 99 field and the 999 ff field are both in the exported MARC record
      11. Go to Settings/Data Import and create the following profiles
        • Match profile: MARC 001 matching to Instance HRID
        • Field mapping profile for Instance: suppress from Inventory and add a statistical code
        • Action profile: Update Instance (and link above field mapping profile)
        • Job profile: Update instance
          • Match profile
            • If yes, then Update instance
      12. Go to the Data Import app
      13. Upload the exported MARC file
      14. Select the newly-created job profile and import the file
      15. Check Inventory to see if the Instance was updated properly (Suppressed from Inventory, and statistical code added)
      16. Check View source to ensure the SRS MARC record has the non-ff 999 field and the 999 ff UUID field
      17. NOTE: When I did this in Kiwi Bugfest, it worked properly. See https://bugfest-kiwi.folio.ebsco.com/inventory/view/d0cf4f85-f451-44b9-bcd1-891b69b27f3f and job 7806, with log https://bugfest-kiwi.folio.ebsco.com/data-import/job-summary/633ce3f2-e89e-465a-b80c-e3d83c2e5b8f?sort=recordNumber

      Expected Behavior
      Record is imported cleanly

      Additional Information
      The following error will come up in mod-source-record-storage:

      21:03:50.486 [vert.x-worker-thread-6] ERROR rdChunksKafkaHandler [873231eqId] Can't process the kafka record: 
      java.lang.IllegalArgumentException: Invalid UUID string: Y 

      Removing all Local 999 fields other than the one with $i and $s for UUIDs will work. Tested on mod-source-record-storage 5.1.9 and 5.0.5

      Attached a file to represent the record with one 999 (works) and legacy 999s (does not work)
      Notes from abreaux

      • This is a regression. this previously worked.
      • I wonder if it's related to the data in the Lehigh MARC record. It would be helpful to understand why the Lehigh file is bombing (event for create at this point), but my file is working (for create and update)


      TestRail: Results


          1. 61PRINT160129.mrc
            436 kB
          2. AMB_999_with_subs_is.mrc
            1 kB
          3. MODSOURCE-435 Part 1.mp4
            13.33 MB
          4. MODSOURCE-435 Part 2.mp4
            11.46 MB
          5. record-multiple-999s.mrc
            1 kB
          6. record-one-999.mrc
            0.8 kB
          7. single_local999_test3.mrc
            0.9 kB

          Issue Links



                Aivar Iusupov Aivar Iusupov
                ian Ian Hardy
                0 Vote for this issue
                6 Start watching this issue



                  TestRail: Runs

                    TestRail: Cases