Details
-
Story
-
Status: Closed (View Workflow)
-
P2
-
Resolution: Done
-
None
-
-
Folijet Sprint 101, Folijet Sprint 102
-
2
-
Folijet
-
Q3 2020 Bug Fix
-
Simmons
-
Data related (ex. Can be detected with large dataset only)
Description
We created a job profile for matching an incoming MARC 856 $u against the "Electronic access: URI" property of holdings records, overlaying if matched, or creating a new record if there is no match. The profile worked fine for a small record batch (1-5 records). When we tried it with a large batch (5000 records), database CPU utilization pegged out, and inventory-storage became unresponsive. Log investigation showed many warnings like:
WARNING: Doing FT search without index for holdings_record.jsonb->>'electronicAccess', CQL >>> SQL: electronicAccess = "\"uri\":\"https://ebookcentral.proquest.com/lib/simmons-ebooks/detail.action\?docID=147309\"" >>> get_tsvector(f_unaccent(holdings_record.jsonb->>'electronicAccess')) @@ tsquery_phrase(f_unaccent('"uri":"https://ebookcentral.proquest.com/lib/simmons-ebooks/detail.action?docID=147309"'))
Inspection of database logs showed long wait times for the query:
SELECT jsonb,id FROM sim_mod_inventory_storage.holdings_record WHERE get_tsvector(f_unaccent(holdings_record.jsonb->>?)) @@ tsquery_phrase(f_unaccent(?)) LIMIT ? OFFSET ?
TestRail: Results
Attachments
Issue Links
- blocks
-
MODINVSTOR-621 Release v19.4.4
-
- Closed
-
- defines
-
UXPROD-2551 NFR: Data Import (Batch Importer for Bib Acq) & PubSub Q3 2020 Technical, NFR, & Misc bug work
-
- Closed
-
- relates to
-
MODDICORE-80 Cannot match on eAccess URI field
-
- Closed
-