The latest versions of mod-source-record-storage v5.0.4 and mod-source-record-manager v3.0.6 have made UPDATE imports increasingly slow. The first updates go reasonably fast, but the second updates on the same record set never completes and take 6-8 hours to reach 82%, only to not complete and the job is stuck at this percentage of completion. Additionally, the following were observed:
1. Database CPU is pegged at 97% on the second update for the entire duration
2. mod-source-record-storage has been behaving inconsistently, sometimes persistently spiking at 600%, sometimes at 60% on updates. See this graph:
3. Even when the job was aborted, mod-srs still was consuming the same amount of CPU for a long time until it was aborted.
Note that this happened for 1K or 25K records UPDATEs
Steps to Reproduce:
On PTF environment (Cap2) because the profiles are there (or create similar profiles on another environment)
1. Perform a CREATE Import of 1K records using the PTF - Create 2" profile
2. Perform an UPDATE Import of 1K records with the profile "PTF Update Success 1"
3. Perform an UPDATE Import of 1K records using the "PTF Update Success 2" profile. The job will drag out for hours.
UPDATE Imports of 1K records (repeatedly or otherwise), should take less than 30 minutes
Repeated UPDATE imports of 1K records took hours