Uploaded image for project: 'mod-pubsub'
  1. mod-pubsub
  2. MODPUBSUB-181

mod-source-record-manager unable to create Kafka topics if tenant ID doesn't match \w{4,}

    XMLWordPrintable

    Details

    • Template:
    • Sprint:
      Folijet Sprint 116
    • Story Points:
      1
    • Development Team:
      Folijet
    • Release:
      R1 2021 Hot FIx #2
    • Hot Fix Approved by Cap Planning?:
      Yes
    • Affected Institution:
      Lehigh, Simmons
    • RCA Group:
      Incomplete requirements

      Description

      NOTE: Extent of affected libraries is not clear. Currently 2 Index Data clients, no EBSCO or ByWater clients; unclear if any other hosted or on prem libraries

      This seems to mean that tenant IDs with less than 4 consecutive word characters will not work with data import.

      To replicate using a recent folio/snapshot Vagrant box:

      1. Create a tenant with an ID of 4 or more characters (e.g. test)
      2. Enable the same set of modules as used by the diku tenant in the Vagrant VM
      3. Create an admin user (e.g. using the bootstrap-superuser.pl script)
      4. Create a UI instance for interacting with the backend (e.g. by cloning platform-complete and running yarn start --tenant test --port 3001)
      5. Import a simple MARC file using the default profile. It should work.

      Then:

      1. Create a tenant with an ID of less than 4 characters (e.g. tst)
      2. Enable modules and create admin user and UI instance as above
      3. Import a simple MARC file using the default profile. It will fail silently.

      kafka-topics.sh --list shows:

      folio.Default.diku.DI_COMPLETED
      folio.Default.diku.DI_INVENTORY_INSTANCE_CREATED_READY_FOR_POST_PROCESSING
      folio.Default.diku.DI_PARSED_RECORDS_CHUNK_SAVED
      folio.Default.diku.DI_RAW_RECORDS_CHUNK_PARSED
      folio.Default.diku.DI_RAW_RECORDS_CHUNK_READ
      folio.Default.diku.DI_SRS_MARC_BIB_INSTANCE_HRID_SET
      folio.Default.diku.DI_SRS_MARC_BIB_RECORD_CREATED
      folio.Default.test.DI_COMPLETED
      folio.Default.test.DI_INVENTORY_INSTANCE_CREATED_READY_FOR_POST_PROCESSING
      folio.Default.test.DI_PARSED_RECORDS_CHUNK_SAVED
      folio.Default.test.DI_RAW_RECORDS_CHUNK_PARSED
      folio.Default.test.DI_RAW_RECORDS_CHUNK_READ
      folio.Default.test.DI_SRS_MARC_BIB_INSTANCE_HRID_SET
      folio.Default.test.DI_SRS_MARC_BIB_RECORD_CREATED
      folio.Default.tst.DI_RAW_RECORDS_CHUNK_READ
      

      Note that only one of the topics required for data import is created for the tst tenant. Log from mod-source-record-manager is attached.

      I suspect the actual problem is in a shared library created from mod-pubsub in the KafkaTopicNameHelper class, but it manifests in this case in mod-source-record-manager. I suspect that this will need to be fixed in other modules, as well.

      This bug is present in the Iris release and prevents tenants that don't match the pattern from being able to upgrade from Honeysuckle with a fully working system.

      Additional info from Jakub Skoczen: Regarding the limitation introduced by this regular expression check – it should be lifted completely and allow any valid FOLIO tenant ID e.g there should be no length limits (1 char names should be accepted) or limits wrt to specific characters used. If there are limitations wrt Kafka topic naming, the tenant ID should be encoded rather than carry these limitations to the tenant ID.

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                abreaux Ann-Marie Breaux
                Reporter:
                wayne Wayne Schneider
                Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases