[MODSOURCE-183] Add bulk query/streaming support to /records web API - FOLIO Issue Tracker

XML

Word

Printable

Details

Type: Task
Status: Closed (View Workflow)
Priority: P3
Resolution: Duplicate
Affects Version/s: None
Fix Version/s: None
Labels:
- migration-load

Template:
customfield_11100 39228
Story Points:
8
Development Team:
Folijet
Epic Link:
Batch Importer (Bib/Acq)

Description

Some web APIs stream database results through the web API without first buffering them in an array. Examples, are the web APIs for querying instances, holdings, and items in the Inventory module and the users web API in the users module. It appears that /records doesn't do this. If you don't limit the results to something like 1,000 and page through the data, the server process will crash with an out of memory error, or, Docker will kill off the process. One should be able to query and get back all records without having to page through the data.

TestRail: Results

Attachments

Issue Links

defines

UXPROD-559 Migrate Bibliographic & Holdings data

Closed

UXPROD-2790 NFR: Make some revisions to Source Record Storage to improve performance for data migration

Closed

is duplicated by

MODSOURCE-202 SRS revision: Stream get records; full record with marc record, raw record, and error record if any

Closed

Activity

People

Assignee:: William Welling

Reporter:: Jon Miller

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 02/Sep/20 9:37 PM

Updated:: 09/Feb/21 10:16 AM

Resolved:: 20/Jan/21 5:06 AM

Add bulk query/streaming support to /records web API