Uploaded image for project: 'mod-data-import'
  1. mod-data-import
  2. MODDATAIMP-296

Issue migrating from Edelweiss to Fameflower

    XMLWordPrintable

Details

    • EPAM BatchLoader Sprint 39
    • 0.5
    • Folijet

    Description

      mod-data-import has been upgraded from v1.8.0 to v1.9.0 on https://bugfest.folio.ebsco.com and the following issues were encountered:

      1. Upgrade failed with the following error:

      INSERT INTO   tenant_mod_data_import.default_file_extensions (id, jsonb) VALUES   (     'f375ffe9-b00b-4786-b0a8-cd99f93e5aab',     '{       \"id\":\"f375ffe9-b00b-4786-b0a8-cd99f93e5aab\",       \"extension\":\".csv\",       \"description\": \"\",       \"dataTypes\":[         \"Delimited\"       ],       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"importBlocked\":false,       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '5cf7a0e5-f359-43f5-8c96-e5df74131183',     '{       \"id\":\"5cf7a0e5-f359-43f5-8c96-e5df74131183\",       \"extension\":\".dat\",       \"description\": \"\",       \"dataTypes\":[         \"MARC\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     'ebb05fe7-45d5-4eab-80cd-587bda42c7ca',     '{       \"id\":\"ebb05fe7-45d5-4eab-80cd-587bda42c7ca\",       \"extension\":\".edi\",       \"description\": \"\",       \"dataTypes\":[         \"EDIFACT\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '8d3a66b7-3a88-4f59-9d83-a50b020ac60a',     '{       \"id\":\"8d3a66b7-3a88-4f59-9d83-a50b020ac60a\",       \"extension\":\".gif\",       \"description\": \"\",       \"dataTypes\":[       ],       \"importBlocked\":true,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     'f340c632-a8b0-4d07-9c2f-b913e82d23d5',     '{       \"id\":\"f340c632-a8b0-4d07-9c2f-b913e82d23d5\",       \"extension\":\".inv\",       \"description\": \"\",       \"dataTypes\":[         \"EDIFACT\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     'e6747033-7a8f-45a8-beba-21468dd554c7',     '{       \"id\":\"e6747033-7a8f-45a8-beba-21468dd554c7\",       \"extension\":\".jpg\",       \"description\": \"\",       \"dataTypes\":[       ],       \"importBlocked\":true,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '1df36889-4d3a-482e-a8c8-9c67b8245feb',     '{       \"id\":\"1df36889-4d3a-482e-a8c8-9c67b8245feb\",       \"extension\":\".marc\",       \"description\": \"\",       \"dataTypes\":[         \"MARC\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     'f445092c-94b8-408a-a9f1-5edd8b5919c9',     '{       \"id\":\"f445092c-94b8-408a-a9f1-5edd8b5919c9\",       \"extension\":\".mrc\",       \"description\": \"\",       \"dataTypes\":[         \"MARC\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '0001886d-679b-4167-bcf6-9ba640929a8f',     '{       \"id\":\"0001886d-679b-4167-bcf6-9ba640929a8f\",       \"extension\":\".mrk\",       \"description\": \"\",       \"dataTypes\":[       ],       \"importBlocked\":true,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '8ffe5abc-a532-48cb-ab0e-827ea980154c',     '{       \"id\":\"8ffe5abc-a532-48cb-ab0e-827ea980154c\",       \"extension\":\".ord\",       \"description\": \"\",       \"dataTypes\":[         \"MARC\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '9dff36d2-cb05-4483-80bc-40dbb57888e6',     '{       \"id\":\"9dff36d2-cb05-4483-80bc-40dbb57888e6\",       \"extension\":\".pdf\",       \"description\": \"\",       \"dataTypes\":[       ],       \"importBlocked\":true,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '756800be-ec03-47b6-ace3-e62d1bd6c5dd',     '{       \"id\":\"756800be-ec03-47b6-ace3-e62d1bd6c5dd\",       \"extension\":\".png\",       \"description\": \"\",       \"dataTypes\":[       ],       \"importBlocked\":true,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '4a052746-5a21-44d1-a2ed-472c5747a488',     '{       \"id\":\"4a052746-5a21-44d1-a2ed-472c5747a488\",       \"extension\":\".tsv\",       \"description\": \"\",       \"dataTypes\":[         \"Delimited\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     'bd7ccd60-0528-4abf-88cc-a89b0b127be1',     '{       \"id\":\"bd7ccd60-0528-4abf-88cc-a89b0b127be1\",       \"extension\":\".txt\",       \"description\": \"\",       \"dataTypes\":[         \"Delimited\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '6cf5568b-4d37-4d35-9116-e8a52120779d',     '{       \"id\":\"6cf5568b-4d37-4d35-9116-e8a52120779d\",       \"extension\":\".xml\",       \"description\": \"\",       \"dataTypes\":[         \"MARC\"       ],       \"importBlocked\":false,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ),   (     '8d15a305-5bf8-497f-b2ff-6cffda5f13cf',     '{       \"id\":\"8d15a305-5bf8-497f-b2ff-6cffda5f13cf\",       \"extension\":\".zip\",       \"description\": \"\",       \"dataTypes\":[       ],       \"importBlocked\":true,       \"userInfo\":{         \"firstName\":\"\",         \"lastName\":\"\",         \"userName\":\"System\"       },       \"metadata\":{         \"createdDate\":\"2019-01-01T11:22:07Z\",         \"createdByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"createdByUsername\":\"System\",         \"updatedDate\":\"2019-01-01T11:22:07Z\",         \"updatedByUserId\":\"00000000-0000-0000-0000-000000000000\",         \"updatedByUsername\":\"System\"       }     }'   ) ON CONFLICT DO NOTHING

      This is because tables are created with "_id" instead of "id" - this issue has been addressed in RMB 25.0.0.
      Since mod-data-import upgraded its RMB version from 23.12.0 to 29.1.5 - upgrade scripts need to be written for the migration to be successful.

      Attached is the file containing a list of indices that need to be dropped, functions that need to be created/deprecated and update statements needs to alter schema.

      TestRail: Results

        Attachments

          Issue Links

            blocks

            Task - A task that needs to be done. MODDATAIMP-298 Release v1.9.1

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed
            defines

            New Feature - New feature requests UXPROD-2220 NFR: Data Import (Batch Importer for Bib Acq) & PubSub Q2 2020 Technical, NFR, & Misc bug work

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Activity

              People

                ruslan_lavrov Ruslan Lavrov
                sduvvuri Sobha Duvvuri
                Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases