Details
-
Bug
-
Status: Closed (View Workflow)
-
P2
-
Resolution: Done
-
None
-
-
EPAM BatchLoader Sprint 34, EPAM BatchLoader Sprint 35
-
5
-
Folijet
Description
Data Import still has the problem of not creating some instance records from a batch of MARC records. We used data import to migrate Simmons college, but only 110,000 out of 180,000 records loaded.
On 2020-01-18, I loaded a test batch of 1000 records to https://folio-snapshot-load.aws.indexdata.com/ and 50 records did not load. In the log, there is no instance ID in 999 $i or externalIdsHolder array for these 50 records.
Here is an example from the log:
{
"id": "c2bcaafc-ef79-4964-8732-feb6304add2e",
"snapshotId": "be1d4f89-23d4-4e6b-8095-bb0b5bd5f038",
"matchedId": "c2bcaafc-ef79-4964-8732-feb6304add2e",
"generation": 0,
"recordType": "MARC",
"rawRecord": {
"id": "93cb25e3-d4e1-4605-a6e2-974d66e44d6d",
"content": "02106cam a2200457 4500001000800000003000600008005001700014008004100031010001300072015001900085019001400104020001500118020001800133020001500151020001800166035003800184040021100222043001200433045000900445049000900454050001500463082001500478090001500493100001800508245003600526260004900562300002600611336002600637337002800663338002700691504005700718520025300775505036901028651004901397650003501446907003101481998003901512913000601551994001201557945007901569\u001e1031158\u001eOCoLC\u001e20181229023350.1\u001e740802s1975 nyum b a001 0 eng \u001e \u001fa74014976\u001e \u001faGB7631419\u001f2bnb\u001e \u001fa633883313\u001e \u001fa0231029632\u001e \u001fa9780231029636\u001e \u001fa0231083513\u001e \u001fa9780231083515\u001e \u001fa(OCoLC)01031158\u001fz(OCoLC)633883313\u001e \u001faDLC\u001fbeng\u001fcDLC\u001fdMUQ\u001fdBAKER\u001fdNLGGC\u001fdBTCTA\u001fdYDXCP\u001fdDEBBG\u001fdVOV\u001fdOCLCO\u001fdBDX\u001fdGBVCP\u001fdOCLCF\u001fdOCLCQ\u001fdOCLCO\u001fdCWCLL\u001fdOCL\u001fdUEJ\u001fdOCLCQ\u001fdCSJ\u001fdOCLCO\u001fdTYC\u001fdDHA\u001fdOCLCQ\u001fdALMSI\u001fdOCLCQ\u001fdCCH\u001fdCPO\u001fdOCLCQ\u001fdBGU\u001fdOCLCQ\u001fdUWO\u001fdDCHUA\u001e \u001fan-us---\u001e \u001faw2w6\u001e \u001faSCLL\u001e00\u001faE165\u001fb.W46\u001e00\u001fa917.3/03/5\u001e \u001faE165\u001fb.W46\u001e1 \u001faWelter, Rush.\u001e14\u001faThe mind of America, 1820-1860.\u001e \u001faNew York,\u001fbColumbia University Press,\u001fc1975.\u001e \u001faxvi, 603 pages\u001fc24 cm\u001e \u001fatext\u001fbtxt\u001f2rdacontent\u001e \u001faunmediated\u001fbn\u001f2rdamedia\u001e \u001favolume\u001fbnc\u001f2rdacarrier\u001e \u001faIncludes bibliographical references (pages 521-576).\u001e \u001faExamines the major social and political issues of the mid-nineteenth century in terms of commonly accepted antebellum beliefs, attitudes, and standards and their formative influence on the American character and subsequent socio-political patterns.\u001e0 \u001faMacrohistory : the long view -- Microhistory : the view since 1776 -- The mission of America -- Equal rights -- The conservative ethic -- Enterprise -- The politics of democracy -- The politics of whiggery -- The constitution of democracy -- Religion as a free institution -- The uses of education -- The frontier West as image of American society -- Consequences.\u001e 0\u001faUnited States\u001fxIntellectual life\u001fy1783-1865.\u001e 0\u001faPublic opinion\u001fzUnited States.\u001e \u001fa.b10033397\u001fb900717\u001fc190220\u001e \u001fabeat\u001fb900717\u001fcm\u001fda\u001fe-\u001ffeng\u001fgnyu\u001fh4\u001e \u001fa-\u001e \u001fa92\u001fbSCL\u001e \u001fg1\u001fi0114400750944\u001fj0\u001flbeat \u001fo-\u001fp$0.00\u001fs-\u001ft0\u001fu0\u001fv0\u001fw0\u001fx0\u001fy.i10033452\u001fz900717\u001e\u001d"
},
"parsedRecord": {
"id": "9d72c8bc-f8ab-4eaa-978a-9ed6535235cd",
"content": {
"fields": [
{
"001": "1031158"
},
{
"003": "OCoLC"
},
{
"005": "20181229023350.1"
},
{
"008": "740802s1975 nyum b a001 0 eng "
},
{
"010": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "74014976"
}
]
}
},
{
"015": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "GB7631419"
},
{
"2": "bnb"
}
]
}
},
{
"019": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "633883313"
}
]
}
},
{
"020": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "0231029632"
}
]
}
},
{
"020": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "9780231029636"
}
]
}
},
{
"020": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "0231083513"
}
]
}
},
{
"020": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "9780231083515"
}
]
}
},
{
"035": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "(OCoLC)01031158"
},
{
"z": "(OCoLC)633883313"
}
]
}
},
{
"040": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "DLC"
},
{
"b": "eng"
},
{
"c": "DLC"
},
{
"d": "MUQ"
},
{
"d": "BAKER"
},
{
"d": "NLGGC"
},
{
"d": "BTCTA"
},
{
"d": "YDXCP"
},
{
"d": "DEBBG"
},
{
"d": "VOV"
},
{
"d": "OCLCO"
},
{
"d": "BDX"
},
{
"d": "GBVCP"
},
{
"d": "OCLCF"
},
{
"d": "OCLCQ"
},
{
"d": "OCLCO"
},
{
"d": "CWCLL"
},
{
"d": "OCL"
},
{
"d": "UEJ"
},
{
"d": "OCLCQ"
},
{
"d": "CSJ"
},
{
"d": "OCLCO"
},
{
"d": "TYC"
},
{
"d": "DHA"
},
{
"d": "OCLCQ"
},
{
"d": "ALMSI"
},
{
"d": "OCLCQ"
},
{
"d": "CCH"
},
{
"d": "CPO"
},
{
"d": "OCLCQ"
},
{
"d": "BGU"
},
{
"d": "OCLCQ"
},
{
"d": "UWO"
},
{
"d": "DCHUA"
}
]
}
},
{
"043": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "n-us---"
}
]
}
},
{
"045": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "w2w6"
}
]
}
},
{
"049": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "SCLL"
}
]
}
},
{
"050": {
"ind1": "0",
"ind2": "0",
"subfields": [
{
"a": "E165"
},
{
"b": ".W46"
}
]
}
},
{
"082": {
"ind1": "0",
"ind2": "0",
"subfields": [
{
"a": "917.3/03/5"
}
]
}
},
{
"090": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "E165"
},
{
"b": ".W46"
}
]
}
},
{
"100": {
"ind1": "1",
"ind2": " ",
"subfields": [
{
"a": "Welter, Rush."
}
]
}
},
{
"245": {
"ind1": "1",
"ind2": "4",
"subfields": [
{
"a": "The mind of America, 1820-1860."
}
]
}
},
{
"260": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "New York,"
},
{
"b": "Columbia University Press,"
},
{
"c": "1975."
}
]
}
},
{
"300": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "xvi, 603 pages"
},
{
"c": "24 cm"
}
]
}
},
{
"336": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "text"
},
{
"b": "txt"
},
{
"2": "rdacontent"
}
]
}
},
{
"337": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "unmediated"
},
{
"b": "n"
},
{
"2": "rdamedia"
}
]
}
},
{
"338": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "volume"
},
{
"b": "nc"
},
{
"2": "rdacarrier"
}
]
}
},
{
"504": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "Includes bibliographical references (pages 521-576)."
}
]
}
},
{
"520": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "Examines the major social and political issues of the mid-nineteenth century in terms of commonly accepted antebellum beliefs, attitudes, and standards and their formative influence on the American character and subsequent socio-political patterns."
}
]
}
},
{
"505": {
"ind1": "0",
"ind2": " ",
"subfields": [
{
"a": "Macrohistory : the long view -- Microhistory : the view since 1776 -- The mission of America -- Equal rights -- The conservative ethic -- Enterprise -- The politics of democracy -- The politics of whiggery -- The constitution of democracy -- Religion as a free institution -- The uses of education -- The frontier West as image of American society -- Consequences."
}
]
}
},
{
"651": {
"ind1": " ",
"ind2": "0",
"subfields": [
{
"a": "United States"
},
{
"x": "Intellectual life"
},
{
"y": "1783-1865."
}
]
}
},
{
"650": {
"ind1": " ",
"ind2": "0",
"subfields": [
{
"a": "Public opinion"
},
{
"z": "United States."
}
]
}
},
{
"907": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": ".b10033397"
},
{
"b": "900717"
},
{
"c": "190220"
}
]
}
},
{
"998": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "beat"
},
{
"b": "900717"
},
{
"c": "m"
},
{
"d": "a"
},
{
"e": "-"
},
{
"f": "eng"
},
{
"g": "nyu"
},
{
"h": "4"
}
]
}
},
{
"913": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "-"
}
]
}
},
{
"994": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"a": "92"
},
{
"b": "SCL"
}
]
}
},
{
"945": {
"ind1": " ",
"ind2": " ",
"subfields": [
{
"g": "1"
},
{
"i": "0114400750944"
},
{
"j": "0"
},
{
"l": "beat "
},
{
"o": "-"
},
{
"p": "$0.00"
},
{
"s": "-"
},
{
"t": "0"
},
{
"u": "0"
},
{
"v": "0"
},
{
"w": "0"
},
{
"x": "0"
},
{
"y": ".i10033452"
},
{
"z": "900717"
}
]
}
},
{
"999": {
"ind1": "f",
"ind2": "f",
"subfields": [
{
"s": "c2bcaafc-ef79-4964-8732-feb6304add2e"
}
]
}
}
],
"leader": "02159cam a2200469 4500"
}
},
"deleted": false,
"order": 634,
"additionalInfo": {
"suppressDiscovery": false
},
"metadata": {
"createdDate": "2020-01-18T21:42:01.030+0000",
"createdByUserId": "5296fb83-3c0f-55bd-9feb-ff6cce17545f",
"updatedDate": "2020-01-18T21:42:01.030+0000",
"updatedByUserId": "5296fb83-3c0f-55bd-9feb-ff6cce17545f"
}
}
I will attach the batch to this issue.
Steps to fix:
- fix the problem with skipping of the chunk
- add more information to the error message
- add "entityPerRepeatedSubfield" to 020 tag
TestRail: Results
Attachments
Issue Links
- defines
-
UXPROD-2207 SRS MARC-Inventory Instance relationship refinement, part 3
-
- Closed
-