Details
-
Bug
-
Status: Closed (View Workflow)
-
P1
-
Resolution: Done
-
None
-
Definitely affecting Cornell, MTSU, Skidmore
-
-
3
-
Folijet Support
-
R2 2021 Hot Fix #4
-
Yes
-
Juniper hotfix approved 25 Oct 2021 in the Slack release_bug_triage channel
-
!!!ALL!!!
-
Implementation coding issue
Description
Documents Case 3 in the comments on MODINVSTOR-815
This issue is observed in a Honeysuckle HF3 environment and in Iris, but may have been fixed by Hotfix 1
Additional info from one of cappadona's comments below:
- Data Import Run A: Create 10,300 SRS MARC Bib and Instances using new Data Import default job profile for Iris (cu_initial-create_10300recs.mrc
)
- Retrieve Instance UUIDs via SRS MARC Query API:
{ "fieldsSearchExpression": "948.d ^= 'cu-batch'" } - Export the full MARC for the 10,300 records using Data Export default job profile
- Create & associate DI profiles
- job profile – moddataimp419_job.json
- match profile (001 -> instance hrid) – moddataimp419_match.json
- action profile (UPDATE Instance on match) – moddataimp419_action.json
- mapping (overlay) – moddataimp419_mapping.json
- job profile – moddataimp419_job.json
- Process the exported MARC (cleanup OCLC identifiers in 035$a)
Note: When we originally ran this and encountered the error, we DID NOT strip the 999 ff - Data Import Run B: Update the SRS MARC Bib records using job profile from #4 (1 file: 10,300 records)
I decided to try running through the test on the Iris reference environment and was unsuccessful in making it through all steps. Here are the results:
- DI Run A: Initial create of 10,300 SRS MARC Bib and Instances (hrid: 17): 12 m
- Retrieve Instance UUIDs via SRS MARC Query API: 629 ms { "fieldsSearchExpression": "948.d ^= 'cu-batch'" }
- Export the full MARC for the 10,300 records (hrid: 8): 3 m
- DI profiles ported via API and manually linked/related
- Process the exported MARC (cleanup OCLC identifiers in 035$a AND strip 999 ff) with external script: 4 s
- DI Run B: Update the 10,300 SRS MARC Bib records: Stuck at 37% after ~10 m
A follow up to SRS MARC Query API reveals 6,471 of the records remain in their original state, so we can conclude that 3,829 were updated (37%)
{ "fieldsSearchExpression": "(035.a ^= '(OCoLC)oc' or 035.a ^= '(OCoLC)0') and 948.d ^= 'cu-batch'" }======================
When attempting to import a file - the import fails and the following message is observed in mod-source-record-storage logs
"idx_records_matched_id_gen", duplicate key value violates unique constraint
New Import's which fail are attempts to update records from a previous Data Import for a batch of appox 14K that also had issues (specifically the earlier batch failed with – Completed with errors status.)
Need analysis to understand state of related db table entries for Data Import attempts which are failing with this constraint error
TestRail: Results
Attachments
Issue Links
- blocks
-
MODSOURCE-398 Release v5.1.7 (R2 Juniper HF#4)
-
- Closed
-
- defines
-
UXPROD-3041 NFR: Data Import (Batch Importer for Bib Acq) R3 2021 Kiwi Technical, NFR, & Misc bug work
-
- Closed
-
- is blocked by
-
MODDATAIMP-488 SPIKE: Data import hangs on Iris via large files
-
- Closed
-
- is cloned by
-
MODSOURCE-399 Import fails with "'idx_records_matched_id_gen', duplicate key value violates unique constraint" SRS logs KIWI BF
-
- Closed
-
- is duplicated by
-
MODINVSTOR-815 Record was updated successfully but on DI logs status: "Completed with errors" and the instance did not get updated.
-
- Closed
-
- relates to
-
MODSOURCE-329 Create script to clean up Snapshot statuses in mod-source-record-storage (Iris Hotfix)
-
- Closed
-
-
MODDATAIMP-424 Import of 5 concurrent jobs become stuck
-
- Closed
-