Uploaded image for project: 'UX Product'
  1. UX Product
  2. UXPROD-1928

APIs for batch downloads (exports)

    XMLWordPrintable

    Details

    • Template:
    • Back End Estimate:
      XXL < 30 days
    • Estimation Notes and Assumptions:
      Includes API design, RMB support and rollout to a selected module (PoC). Excludes rollout across all FOLIO modules.
    • Development Team:
      Core: Platform
    • Calculated Total Rank:
      77
    • PO Rank:
      9
    • Rank: Chalmers (Impl Aut 2019):
      R4
    • Rank: Chicago (MVP Sum 2020):
      R1
    • Rank: Cornell (Full Sum 2021):
      R1
    • Rank: Duke (Full Sum 2021):
      R4
    • Rank: 5Colleges (Full Jul 2021):
      R1
    • Rank: GBV (MVP Sum 2020):
      R2
    • Rank: Lehigh (MVP Summer 2020):
      R1
    • Rank: MO State (MVP June 2020):
      R1
    • Rank: TAMU (MVP Jan 2021):
      R1
    • Rank: U of AL (MVP Oct 2020):
      R1

      Description

      Problem statement

      FOLIO's API are focused on fetching (GET) single record at a time and while this process can be tuned through various HTTP optimisation techniques (keepalive, pipelining) it's generally tricky to ensure that all clients are configured properly.

      To ensure good performance, it is much easier to provide dedicated batch APIs for batch fetching (downloads).

      Implementation wise, batch downloads should support streaming of data directly from the HTTP connection into a DB connection.

      Streaming downloads

      There is an example on how to implement streaming downloads with RMB in mod-users. Requires code review and maybe polishing.
      https://github.com/folio-org/mod-users/compare/user-stream-example

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                jakub Jakub Skoczen
                Reporter:
                jakub Jakub Skoczen
                Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases