Details
-
Story
-
Status: Closed (View Workflow)
-
P3
-
Resolution: Done
-
None
-
-
EPAM BatchLoader Sprint 4, EPAM BatchLoader Sprint 5
-
8
-
Folijet
Description
ChangeEngine should be a new component that would provide functionality for parsing raw records.
Records should be processed sequentially, after parsing each record should be validated and saved as Valid or Error record accordingly. JobExecution and Snapshot statuses should be updated along the way.
For more details refer to the technical design.
In scope of this story an API for the Parser should be defined. Implementation should be provided only parsing MARC records using marc4j library.
Nice to have ChangeEngine should provide parallel execution option.
ChangeEngine should be configurable regarding to:
- the size of a working chunk to update the progress;
- the amount of simultaneously working threads;
Acceptance Criteria
- ChangeEngine component exists
- ChangeEngine interacts with MetadataService for getting and updating JobExecution statuses.
- ChangeEngine interacts with mod-source-record-storage via REST for updating Snapshot statuses.
- Functionality is covered with tests
TestRail: Results
Attachments
Issue Links
- is blocked by
-
MODSOURCE-6
Add CRUD API for Snapshot at mod-source-record-storage
-
- Closed
-
-
MODSOURCE-7
Add CRUD API for Record entity at mod-source-record-storage
-
- Closed
-
-
MODSOURMAN-3
Create technical design for Change Manager component from mod-source-record-manager
-
- Closed
-
-
MODSOURMAN-7
MetadataRepository: Implement API for JobExecution entities
-
- Closed
-
- relates to
-
MODDATAIMP-39
Create service for file chunking
-
- Closed
-
-
MODSOURMAN-9
Create ChangeManager component, expose API for JobExecutions creation
-
- Closed
-
-
MODSOURMAN-10 ChangeManager: Add functionality for saving chunks of raw data
-
- Closed
-
-
UXPROD-656 Data Import (Batch Importer for Bib Acq) Infrastructure and Planning
-
- Closed
-