Uploaded image for project: 'UX Product'
  1. UX Product
  2. UXPROD-3261

NFR: R1 2022 Lotus Data import performance work

    XMLWordPrintable

    Details

    • Template:
      UXPROD features
    • Front End Estimate:
      Out of scope
    • Back End Estimate:
      Jumbo: > 45 days
    • Confidence factor:
      Medium
    • Development Team:
      Folijet
    • Calculated Total Rank:
      0
    • PO Rank:
      124

      Description

      Team estimation - 45 days

      UXPROD-3135 was split into UXPROD-3193 for stability and reliability and UXPROD-3191 for performance; Ann-Marie Breaux to close UXPROD-3135 once all issues moved from it to the new features

      Current situation or problem:

      1.Kafka producer closed after sending

      2.WARN message when no handler found

      3. Kafka cache resource consumption
      4. Data import impacts other processes

      5. High resource consumption to get job(s) status/progress

       

      In scope

      Out of scope

      Use case(s)

      Proposed solution/stories
      *1.*Create pool of active producers. Start pool on module launch, close on shutdown. Reuse connections. Add max/min pool sizes.
      2. Do not subscribe to messages you're not going to process OR Lower log lever for this type of messages.

      *3.*Remove Kafka cache. Modules that do not do persistent changes will sometimes (on duplicates read) do unnecessary calls. Can be optimized further upon adding distributed in-memory cache (ex hazelcast) (blocked by 6 PUT LINK TO FEATURE in p.6)
      4. SPIKE REQ.: Need investigation (possible solution - configure rate limiter). Relates to High CPU/Memory consumption on modules
      5. Add some kind of caching for progress tracking (database or in-memory)

      Links to additional information:

      Data Import Stabilization plan - Vladimir Shalaev - FOLIO Wiki

      Questions

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                abreaux Ann-Marie Breaux
                Reporter:
                Taisiya Trunova Taisiya Trunova
                Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                  Dates

                  Created:
                  Updated:

                    TestRail: Runs

                      TestRail: Cases