[PERF-144] Harvesting 10M using marc21_withholdings metadata prefix - FOLIO Issue Tracker

XML

Word

Printable

App name/area of issue:
OAI-PMH harvesting
Brief statement of the issue:
Determine the performance of the harvest of 10M inventory records with metadataPrefix set to marc21_withholdings
Steps to recreate the issue:

initiate the harvest with request: https://<edge_url>/oai?verb=ListRecords&metadataPrefix=marc21_withholdings&apikey=<apiKey>
in consecutive requests use the resumptionToken returned in the prior requests until all data is harvested. The request containing resumptionToken has following structure:
https://<edge_url>/oai?verb=ListRecords&apikey=<apiKey>&resumptionToken<resumptionToken>

How many users at any given time:
1
Volume of data:
10M
Expected response time:
Less than 24h
Main modules that are involved in the process (if obvious or if known):
edge-oai-pmh, mod-oai-pmh, srs modules, mod-inventory-storage
Any specific settings or items or scenarios:
The default settings for number of records in response is set to 100:

Please conduct the tests with post-honeysuckle edge- and mod-oai-pmh release (2.3.0 and 3.3.0 respectively)

defines

UXPROD-3027 OAI-PMH maintenance - Kiwi

is cloned by

PERF-145 Harvesting 10M using marc21 metadata prefix

relates to

MODOAIPMH-339 Address memory leaks

PERF-198 OAI-PMH Kiwi release -performance testing