Uploaded image for project: 'FOLIO'
  1. FOLIO
  2. FOLIO-2563

SPIKE: propose prevention of DoS via CQL query

    XMLWordPrintable

    Details

    • Template:
    • Sprint:
      CP: sprint 87, CP: sprint 88, CP: sprint 89, CP: sprint 90
    • Story Points:
      5
    • Development Team:
      Core: Platform

      Description

      Description

      Several application endpoints accessible by normal users perform CQL (Common Query Language) queries leveraging CQL-Java. CQL is a formal language for representing queries to information retrieval systems such as web indexes, bibliographic catalogs and museum collection information.1 The application can be forced to perform an OR expression over objects that match a certain patterns. As a result, the CQL engine is forced to do very expensive queries to search and parse the result of the operations. Although it appears that the full result of the operation is filtered downstream when returned to the user, It was noticed that each request to the target endpoint would take around 29 seconds to respond back. An attacker can exploit this by performing a denial of service attack by requesting several GET requests simultaneously or in short intervals, resulting in the exhaustion of the server’s memory resources.

      Steps to reproduce

      Perform the following request with the thread count set to 20 and throttle time set to 1 milliseconds. Substitute the value of X-Okapi-Token header with a token belonging to a user that has inventory.instances.collection.get permission.

      GET /inventory/instances?limit=1000&query=%28keyword%20all%20%22ncc%22%29%20or%201=1
      urldecoded query: (keyword all "ncc") or 1=1
      Users can use the Inventory front-end "Query search" slot to enter any CQL: https://folio-snapshot.aws.indexdata.com/inventory?qindex=querySearch&sort=Title

      Observe that all inventory related endpoints will become unresponsive

      Acceptance criteria

      Propose an approach that will prevent heavy queries to cause a module to become unresponsive. Provide a PoC implementation for review.

      Potential solutions:

      Query validation based:

      • RMB: validate query length
      • RMB: validate query structure (max number of OR expression)
      • RMB: validate if query field and relation is backed up by an index (in schema.json, reject if not)
      • RMB: validate known problematic searches (e.g cql.allRecords)
      • RMB: wide truncation queries

      Query execution based:

      • Can Postgres analyze query runtime?
      • Can Postgres limit the runtime for a SELECT
      • If not, can we timeout a connection for long running queries

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                mikhail.fokanov Mikhail Fokanov
                Reporter:
                jakub Jakub Skoczen
                Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases