Uploaded image for project: 'mod-user-import'
  1. mod-user-import
  2. MODUIMP-4

Bulk import of users needs performance improvements




      We need performant bulk import of users for both migration and ongoing operations.

      The current user import module submits users one-by-one, which is rather slow at a scale of, say, 90K users, especially when trying to test data loading at scale. SysOps foresees similar issues with loading data into other modules.

      There are two general scenarios:

      1. At migration time, sites will need to do an initial bulk import from their existing systems, and new UUIDs will somehow need to be minted as part of the migration process.
      2. In ongoing operations, sites will do regular bulk updates from their campus identity management (IdM) infrastructure to refresh their campus users. In this case, existing UUIDs and some internal FOLIO fields will need to be preserved.

      The migration scenario here seems like a special case of the bulk import to support operations.

      There will be a need for bulk imports across FOLIO modules for both migration and operations. Users seems like a good starting point to consider some common strategies or techniques for bulk import that could inform best practices for project as a whole.

      This is related to MODUSERS-3.

      TestRail: Results


          1. users.zip
            4.42 MB
          2. vagrant_top.png
            95 kB

          Issue Links



                Unassigned Unassigned
                tod Tod Olson
                0 Vote for this issue
                10 Start watching this issue



                  TestRail: Runs

                    TestRail: Cases