Uploaded image for project: 'UX Product'
  1. UX Product
  2. UXPROD-1941

Wait for POC of Elastic Search. Front-end query pre-processor: support words, phrases and booleans

    XMLWordPrintable

Details

    • XL < 15 days
    • Medium < 5 days
    • Core: Platform
    • 1
    • R4
    • R1
    • R2
    • R2
    • R2
    • R2
    • R2
    • R3
    • R4

    Description

      Problem statement

      Core search apps in FOLIO (ui-users, ui-inventory) accept only a simple search input string from users. This input cannot include any special characters (* wildcard being an exception), quotes (to represent phrases) or booleans.

      From this input the UI generates CQL (boolean) queries according to hard-coded recipes. Either simple ones like index="{userInput}" or more complicated
      search expression across several search indexes. E.g, given user input:

      john smith

      ui-users will generate:

      firstName="john smith" OR lastName="john smith" OR username="john smith"

      This approach is generally problematic and results in inadequate search behavior (see e.g UIU-939, UIIN-435, UIIN-564). With the workaround provided through UIU-1068, some problems with this approach are addressed but others remain (like UIIN-602) and the workaround creates an impediment for providing more sophisticated search functions in the future (boolean search, ranking, etc) . We would like to redesign how the UI (apps) handle user search input and include support for both simple term searches and boolean expressions (with quotes, boolean operators and parenthesis).

      Proposed solution

      The proposed solutions consists of two parts: changes in the UI (SearchAndSort component) and in the backend query converter (which resides in RMB).

      UI changes: Design and implement (in JavaScript) a front-end query pre-processor that will provide a simple to use front-end query syntax. The pre-processor must support handling simple tokens (words), quoted tokens (phrases) and boolean operators (AND OR NOT) and convert the user query into CQL that can than be handled by the back-end. See UXPROD-1015 for the eHoldings boolean search screencast.

      Example (inventory):

      Given, user input for the all search drop-down: "the c programming language" kernighan OR knuth the UI generates the following CQL:

      a) assuming we have a back-end search index for all: all="the c programming language" AND all="kernighan" OR all="knuth"

      b) assuming the UI needs to expand all into two indexes: title and author: (title="the c programming language" OR author="the c programming language" AND (title="kernighan" OR author="kernighan") OR (title="knuth" OR author="knuth")

      Obviously, option b) leads to a longer and more complex CQL query. Hence a backend extension is proposed to address this.

      Backend: changes. Add a new feature which will allow creating "compound indexes" for a list of JSON properties (or for a specific subset of JSON), see RMB-385. This will allow module authors to create "virtual" search fields, e.g fullName that includes both userName and firstName.

      Alternative solution considered but rejected

      Alternative is to add extensions to the back-end CQL java parser that would not be conformant to the spec but would allow us to use the language verbatim in the UI and support quoting phrases and using booleans (see RMB-428)

      TestRail: Results

        Attachments

          Issue Links

            blocks

            New Feature - New feature requests RMB-428 cql-java: AND multiple tokens

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. UXPROD-869 Advanced Search (within apps)

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • In progress
            is blocked by

            New Feature - New feature requests RMB-385 add 'queryIndexName' to schema.json and allow compound indexes

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed
            relates to

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. STCOM-1180 CLONE - Implement Advanced search modal

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. STSMACOM-767 CLONE - Implement Advanced search modal

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. UIIN-868 Inventory search. Holdings segment. Search by using query language

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. UIIN-869 Inventory search. Item segment. Search by using query language

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. UIIN-1920 Implement Advanced search modal

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Task - A task that needs to be done. UIU-1068 expand the list of query terms into AND and OR expression

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            New Feature - New feature requests UXPROD-140 Q4 2019 Timebox for Priority Inventory Search and Filter Enhancements

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-1015 Boolean/Query Search for Users

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Draft

            New Feature - New feature requests UXPROD-1820 RMB 27 release features and core module rollout (performance, virtual indexes, search enhancements)

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-2119 RMB 28 release features and core module rollout (performance, virtual indexes, search enhancements)

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-2180 Q1 2020 Timebox for Priority Inventory Search and Filter Enhancements

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-2298 Q2 2020 Timebox for Priority Inventory Search and Filter Enhancements

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-2443 Q3 2020 Timebox for Priority Inventory Search and Filter Enhancements

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-2712 Inventory Elastic Search (Lotus): Timebox for Priority Inventory Search and Filter Enhancements

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            New Feature - New feature requests UXPROD-3513 Inventory Elastic Search (Morning Glory): Timebox for Priority Inventory Search and Filter Enhancements

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MSEARCH-305 BE- Inventory. Keyword search should use normalization of Resource titles and Contributor names

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. MSEARCH-362 BE: Inventory. Keyword search should use normalization of Resource titles and Contributor names

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Draft

            Bug - A problem which impairs or prevents the functions of the product. UIIN-602 no records found when searching titles for words out of order.

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. UIIN-724 Inventory search. Instance segment. Search by using query language

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Activity

              People

                Unassigned Unassigned
                jakub Jakub Skoczen
                Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                  Created:
                  Updated:

                  TestRail: Runs

                    TestRail: Cases