Uploaded image for project: 'RAML Module Builder'
  1. RAML Module Builder
  2. RMB-581

Extend PgUtil.streamGet() to support manual selection of a set of columns

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • TBD
    • Resolution: Unresolved
    • None
    • None
    • None
    • DevOps: Sprint 95
    • Core: Platform

    Description

      The PgUtil.streamGet() may be used for some exceptionally large data sets.
      The current design does not seem performant resource-wise (memory consumption, network bandwidth, and to some extend time).
      This happens because the entire jsonb appears to be return from the database to the backend module before it casts/converts/maps that response to some smaller object.

      Ideally, provide a way to:

      • Get only a selection of some fields, such as hrid.
      • Get that response without object mapping.
      • Maintain stream support.
      • May involve improving *.raml functionality to designate non-object results.
      • Consider additional actions on fields, such as select encode((jsonb->>'id')::bytea, 'base64') from instance;.
      • Potentially get the table columns and not just jsonb keys/properties (get 'id' instead of 'jsonb->id') for performance reasons.

      This potentially blocks MODINVSTORE-439, but can alternatively be considered tech-debt to instead not block and utilize the currently functional but less performant implementation of PgUtil.streamGet().

      TestRail: Results

        Attachments

          Issue Links

            Activity

              People

                Unassigned Unassigned
                kday Kevin Day
                Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                  Created:
                  Updated:

                  TestRail: Runs

                    TestRail: Cases