Uploaded image for project: 'RAML Module Builder'
  1. RAML Module Builder
  2. RMB-863

Free streamGet PreparedStatement fixing Aggressive DB Memory Consumption



    • Bug
    • Status: Closed (View Workflow)
    • P2
    • Resolution: Done
    • 33.0.0
    • 33.1.0
    • None
    • CP: sprint 119, CP: sprint 123
    • 5
    • Core: Platform


      When running a check-in/out test with the Juniper release for 30 minutes, the database consumes memory aggressively. Depending on the load, the DB could use up anywhere from 800MB (1 user) to 2GB of memory (20 users). Running the test for more than 3 hours would exhaust all the available memory that the DB started out with, 7GB. Investigate the cause of this aggressive memory consumption.

      Steps to Reproduce A:
      Run the check-in/out test for 30 minutes with at least 5 concurrent users and observe the DB's memory usage pattern in Cloudwatch

      Expected Results A:
      Memory consumption would be steady and does not drop much more as long as it has claimed enough memory for the current load.

      Actual Results A:
      Memory keeps on being claimed but not returned during the test duration.

      Steps to reproduce B:
      Build and run mod-configuration 5.7.0 jar:

      java -jar mod-configuration-server/target/mod-configuration-server-fat.jar

      POST /_/tenant to create schema for diku and run GET requests against the empty configuration table:

      curl -w"\n" -s -H "Content-type: application/json" -H "X-Okapi-Tenant: diku" -d '{}' http://localhost:8081/_/tenant
      for i in {1..99999} then; do curl -s -H "X-Okapi-Tenant: diku" http://localhost:8081/configurations/entries > /dev/null; done

      There is no record and each GET returns an empty set.
      Run pg_top (or ps -F ax) to watch the memory consumption of the database process of the connections of mod-configuration.

      Expected result B:
      The database memory usage stays below 20 MB per connection.

      Actual result B:
      The database memory usage for one of the four mod-configuration connections increases by 90 MB per minute.

      Additional Information:
      See https://wiki.folio.org/pages/viewpage.action?pageId=65116448 for more details.

      The memory issue is caused by RMB's streamGet that doesn't close PreparedStatement after use. The bug has been introduced by https://github.com/folio-org/raml-module-builder/commit/d38030b9d8929b0d497bd48167fffd05c560bc10 and is in RMB >= 33.0.0.

      streamGet is used by only 4 modules (GitHub search):

       After RMB has been released with this fix these 4 modules should upgrade RMB and release a new version.

        Juniper w/o fix (HotFix#2) Juniper with fix (HotFix#3) Kiwi w/o fix Kiwi with fix
      mod-configuration 5.7.0 5.7.1 MODCONF-94 5.7.1
      mod-inventory-storage 21.0.6 21.0.8 MODINVSTOR-795 22.0.0
      mod-permissions 5.14.0 5.14.2 MODPERMS-155 5.14.2
      mod-users 18.0.0 18.0.1 MODUSERS-273 18.1.0 18.1.1

      Interested parties:
      julianladisch jakub

      TestRail: Results


          Issue Links



                julianladisch Julian Ladisch
                mtraneis Martin Tran
                0 Vote for this issue
                5 Start watching this issue



                  TestRail: Runs

                    TestRail: Cases