Overview: In folio-snapshot-load, importing a large MARC file using a job profile causes the import to stop.
Steps to Reproduce:
- Log into folio-snapshot-load as diku_admin
- Go to Settings/Data import/Field mapping profiles
- Create a field mapping profile with Incoming record MARC Bib and FOLIO record Instance. No additional mappings
- Create an action profile with action Create and FOLIO record = Instance, and link the above field mapping profile to it
- Create a job profile and link the above action profile to it
- Upload a 100MB+ MARC file (you can get some large files from here: https://drive.google.com/drive/folders/1ydR_KIdas9MI0iIhy7CLqwf_Tx8JFMTV?usp=sharing)
- On the choose jobs page, select the newly created job profile and run the import
Expected Results: The import should complete
Actual Results: It gets stuck
See attached video, plus screenshot from yesterday. I was able to reproduce the problem on multiple days. The import completes when using the (non-pubsub) secret button (took 24 minutes for 100,000 records), but not when using a pubsub job profile
Ideally we get back to 100,000 records. 10,000 records is acceptable for Q2, so long as we work toward 100,000 by the end of Q3. 1,000 records - not sure if this would be acceptable for the Q2 release, then aim to get back to 10K with a Q2 hotfix release. 100 records is not OK.