SPIKE: Data Import job is creating duplicate records with or without any background activity

Changed this ticket into a spike; depending on results, may create a new ticket for the actual hot fix

Overview:

Using "Create" job profile, when we run 2 data import jobs in parallel of size 25k records and 5k records, it creates duplicates records for 25k records job. In some cases, we have seen 5k job also create duplicate records. And as noted in a comment below. Cornell is able to reproduce on folio-snapshot with a file of ca. 9K records.

We have seen duplicate records been created consistently when some background activity is running.

After investigating further, we are seeing duplicate records also being created with a standalone data-import create job.

And in comments below, Jenn Colt confirms she was able to reproduce it on folio-snapshot

Steps to Reproduce:

With background activity

1. Log into PTF env tenant 1 as folio user
2. Select Create Instances, Holdings, and Items job profile
3. Run data import for 25k records
4. Log into PTF env tenant 2 as folio user
5. Select Create Instances, Holdings, and Items job profile
6. Run data import for 5k records

Without background activity

• Log into PTF env tenant 1 as folio user
• Select Create Instances, Holdings, and Items job profile
• Run data import for 25k records

Expected Results:

Data import job should create 25k instances, holdings, and items for Iris tenant 1

Data import job should create 5k instances, holdings, and items for Iris tenant 2
Actual Results:

Data import job is creating 25100 instances, 25100 holdings, and 25100 items for Iris tenant 1

Data import job is creating 5000 instances, 5000 holdings, and 5000 items for Iris tenant 2

Additional Information:

Iris release

Module versions in envs:

• mod-data-import-2.0.2
• mod-data-import-converter-storage-1.10.2
• mod-source-record-manager-3.0.5
• mod-source-record-storage-5.1.0-SNAPSHOT.88

Interested parties:

All users of data import