Uploaded image for project: 'mod-data-import'
  1. mod-data-import
  2. MODDATAIMP-210

When scaling container horizontally - local cache for the request isn't accessible across replicas

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • P2
    • Resolution: Won't Do
    • None
    • None
    • Running in a Rancher provisioned Kubernetes cluster - spread across multiple hosts and datacenters.

    • 0
    • Folijet
    • Q3 2020

    Description

      When running mod-data-import in production, it is desirable to scale the module horizontally for availability and performance. The request gets locally cached in the module - and that cached space isn't shared between the replicas. Thus, if one module replica takes the request, and another replica processes it, and lastly another replica actually does the load - the cached data file is unavailable and the error is thus:

      19:00:09.749 [processing-files-thread-pool-3] ERROR ileChunkingProcessor [56312257eqId] Can not process file: ./storage/upload/629c1ad7-4134-4466-9df1-ddef0cc93464/0e3155c0-d7a0-4e6f-bad7-5697bde9a3df/amdb_sample.mrc. Cause: Can not initialize reader. Cause: File './storage/upload/629c1ad7-4134-4466-9df1-ddef0cc93464/0e3155c0-d7a0-4e6f-bad7-5697bde9a3df/amdb_sample.mrc' does not exist

      java.lang.IllegalArgumentException: Can not initialize reader. Cause: File './storage/upload/629c1ad7-4134-4466-9df1-ddef0cc93464/0e3155c0-d7a0-4e6f-bad7-5697bde9a3df/amdb_sample.mrc' does not exist

      It may be desirable to present this cached space to the module as shared storage - such as NFS.

      TestRail: Results

        Attachments

          Issue Links

            Activity

              People

                Unassigned Unassigned
                jroot Jason Root
                Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases