Uploaded image for project: 'mod-data-import'
  1. mod-data-import
  2. MODDATAIMP-508

SPIKE: Possible errors with importing a file not reported

    XMLWordPrintable

    Details

    • Template:
    • Story Points:
      5
    • Development Team:
      Folijet
    • Release:
      R3 2021
    • Affected Institution:
      TAMU, University of Tennessee Martin

      Description

      University of Tennessee-Martin imported a file of 78 records. 50 loaded fine. 28 disappeared without any indication in the log. Ann-Marie Breaux tried to reproduce on https://iris-cap1.int.aws.folio.org/ and https://folio-snapshot.dev.folio.org/ but unable. However, there are issues with their overall job profile, plus a logic issue that needs review.

      Review analysis and possible scenarios here

      Steps to reproduce:

      1. Go to Data Import App
      2. Import a marc file with any number of records - lets say 78
      3. Job logs show that the process completed with errors
      4. On checking the job log, 50 were imported successfully but there is no clue or errors for the rest of the 28 records

      Interested Parties: 
      Patty Wanninger provided the actual file for troubleshooting(Attached)

      Expected Results: 
      Errors  are reported for the erroneous/failed 28 records

      Actual Results:
      No information in logs about the failed 28 records

      Notes:

      SELECT * FROM tenant_mod_source_record_manager.job_execution_progress where jobexecutionid='2235f958-6d01-4802-9ad3-19b50f91eebe'

      {     "id": "e637fe60-e269-465a-9263-be3346b8ca15",     "total": 78,     "jobExecutionId": "2235f958-6d01-4802-9ad3-19b50f91eebe",     "currentlyFailed": 28,     "currentlySucceeded": 50 }

      SELECT * FROM tenant_mod_source_record_manager.journal_records where job_execution_id='2235f958-6d01-4802-9ad3-19b50f91eebe'
      order by source_record_order (shows first 50 record only)
      Have not been able to locate details regarding the failed 28

      In UT-Martin's tenant, the job is # 263, as shown in the screenshot
      Notes about the profiles used:

      • Location: pmev - see attached screenshot
      • Job profile
        • Name: Electronic Video - Create Instance & Holdings or Overlay
        • Accepted Data Type: MARC
        • Match profile: Electronic video - Overlay
          • For matches, Action 1: Electronic video - Update instance
          • For matches, Action 2: Electronic video - Update holdings
          • For non-matches: Action 3: Electronic video - Create instance
          • For non-matches: Action 4: Electronic video - Create holdings
      • Match profile
        • Name: Electronic video - Overlay
        • Incoming MARC Bib matches to Instance
        • Incoming record field: 001 (no indicators or subfield)
        • No qualifier or only compare
        • Exactly matches
        • Instance field: System control number
        • No qualifier or only compare
      • Action profile 1
        • Name: Electronic video - Update instance
        • Action: Update
        • Record type: Instance
      • Field mapping profile 1
        • Name: Electronic video - Create instance (this is also field mapping profile 3)
        • Incoming record type: MARC Bibliographic
        • FOLIO record type: Instance
        • Description: It is the profile for new records from Films on Demand.
        • Cataloged Date: ###TODAY###
        • Instance status term: "Batch loaded"
      • Action profile 2
        • Name: Electronic video - Update holdings
        • Action: Update
        • Record type: Holdings
      • Field mapping profile 2
        • Name: Electronic video - Create holdings (this is also field mapping profile 4)
        • Incoming record type: MARC Bibliographic
        • FOLIO record type: Holdings
        • Description: It is a profile to create associated holdings records from Films on Demand.
        • Holdings type: "Electronic"
        • Permanent location: "Electronic Video Collection (pmev)"
        • Electronic access
          • Add these to existing
          • Relationship: "Resource"
          • URI: 856$u
          • URL public note: "Electronic video; click here for direct access"
      • Action profile 3
        • Name: Electronic video - Create instance
        • Action: Create
        • Record type: Instance
      • Field mapping profile 3
        • Name:: Electronic video - Create instance (this is also field mapping profile 1)
      • Action profile 4
        • Name: Electronic video - Create holdings
        • Action: Create
        • Record type: Holdings
      • Field mapping profile 4
        • Name: Electronic video - Create holdings (this is also field mapping profile 2)
        • Incoming record type: MARC Bibliographic
        • FOLIO record type: Holdings
        • Description: It is a profile to create associated holdings records from Films on Demand.
        • Holdings type: "Electronic"
        • Permanent location: "Electronic Video Collection (pmev)"
        • Electronic access
          • Add these to existing
          • Relationship: "Resource"
          • URI: 856$u
          • URL public note: "Electronic video; click here for direct access"

        TestRail: Results

          Attachments

          1. Job 263.PNG
            Job 263.PNG
            161 kB
          2. Load_FOD_071521.mrc
            170 kB
          3. Load_FOD_071521.mrk
            156 kB
          4. pmev location.PNG
            pmev location.PNG
            150 kB
          5. utm-log-insights-0715-srm (1).log
            105 kB
          6. utm-log-insights-0715-srs (1).txt
            9 kB

            Issue Links

              Activity

                People

                Assignee:
                afedasiuk Aliaksandr Fedasiuk
                Reporter:
                sduvvuri Sobha Duvvuri
                Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases