[UXPROD-2806] Create Elasticsearch indexes for Inventory search - FOLIO Issue Tracker

XML

Word

Printable

Details

Type: New Feature
Status: Closed (View Workflow)
Priority: P2
Resolution: Done
Affects Version/s: None
Fix Version/s: R1 2021
Component/s: None
Labels:
- r1-2021-at-risk

Template:

UXPROD features
Epic Link:
Elasticsearch
Front End Estimate:
XXL < 30 days
Front-End Confidence factor:
Low
Back End Estimate:
XXL < 30 days
Development Team:
Falcon
Calculated Total Rank:
Rank: Chicago (MVP Sum 2020):
R1
Rank: Cornell (Full Sum 2021):
R1
Rank: Duke (Full Sum 2021):
R1
Rank: 5Colleges (Full Jul 2021):
R1
Rank: GBV (MVP Sum 2020):
R2
Rank: Lehigh (MVP Summer 2020):
R2
Rank: TAMU (MVP Jan 2021):
R2
Rank: U of AL (MVP Oct 2020):
R2
Score:
16
Showstopper for Summer 2021 Implementers?:
No
Showstopper Comments from Summer 2021 Implementers:

Hide
[Steven from Five Colleges: Five College Showstopper based the complexity of our locations] [Magda responded to Steven: Hi Steven Bischof could you provide some use cases where the current implementation of Inventory search is insufficient for Five Colleges' needs?] [Tod from Chicago: The issue for us is around needed improvements to search and search results in Inventory, many of which have become tied to Elasticsearch implementation. The problem with search in Inventory is that there are many cases where the available identifying text is not enough distinguish between the results. Without those needed improvements, we will be having some difficult conversations about whether we can support our workflows.][Shawn & Kay from MI St U. Need efficient search to enable pre-order work, adding cataloging to complicated sets, such as government documents.]

Show
[Steven from Five Colleges: Five College Showstopper based the complexity of our locations] [Magda responded to Steven: Hi Steven Bischof could you provide some use cases where the current implementation of Inventory search is insufficient for Five Colleges' needs?] [Tod from Chicago: The issue for us is around needed improvements to search and search results in Inventory, many of which have become tied to Elasticsearch implementation. The problem with search in Inventory is that there are many cases where the available identifying text is not enough distinguish between the results. Without those needed improvements, we will be having some difficult conversations about whether we can support our workflows.][Shawn & Kay from MI St U. Need efficient search to enable pre-order work, adding cataloging to complicated sets, such as government documents.]
Showstopper December 11 Meeting Summary:

Hide
UPDATE: This feature was changed to NOT be a showstopper given a thin-thread version of ~~UXPROD-2712~~ is the actual showstopper.
The understanding by the implementers is that this feature needs to be completed before they can the improved searching that they need (as described in ~~UXPROD-2712~~). Right now users cannot tell the difference between an ebook and a regular book. Magda, the PO, pointed out that this feature is to implement Elasticsearch in addition to the existing search-it will not be integrated as part of R1 2021. So, we need to figure out what the Iris summer 2021 implementers REALLY need. Charlotte recommends that we look at the user stories that define ~~UXPROD-2712~~ and decide which ones are 'showstoppers'. We will move the 'showstoppers' to a separate UXPROD feature and estimate that work. This feature would be re-planned for R2 2021.

Show
UPDATE: This feature was changed to NOT be a showstopper given a thin-thread version of UXPROD-2712 is the actual showstopper. The understanding by the implementers is that this feature needs to be completed before they can the improved searching that they need (as described in UXPROD-2712 ). Right now users cannot tell the difference between an ebook and a regular book. Magda, the PO, pointed out that this feature is to implement Elasticsearch in addition to the existing search-it will not be integrated as part of R1 2021. So, we need to figure out what the Iris summer 2021 implementers REALLY need. Charlotte recommends that we look at the user stories that define UXPROD-2712 and decide which ones are 'showstoppers'. We will move the 'showstoppers' to a separate UXPROD feature and estimate that work. This feature would be re-planned for R2 2021.
Showstopper Capacity Planning Team Recommendation:

Description

Current situation or problem:

The existing inventory search does not provide enough flexibility in building the search criteria, does not perform as expected when searches in large collection takes long to return the data. Also, in searches that have more than 1000 records fulfilling the search, the count of the matching records is incorrect.

In scope
Searches against inventory instances, holdings and items:

Instance	Holdings	Items
Keyword search (title, contributor, identifier)	Effective location	Item Status
Effective location	Holdings permanent location	Effective location
Language	Suppress from discovery	Holdings permanent location
Resource type		Material type
Format		Suppress from discovery
Mode of issuance
Nature of content
Staff suppress
Suppress from discovery
Date created (from, to)
Date updated (from, to)
Source

Out of scope

Other than inventory records searches
Updates to existing inventory search.

Proposed solution/stories

1. Define mappings that will determine

which fields should be treated as full text search fields
which fields contain numbers and dates and their format
custom rules for dynamically added fields

2. Define approach to text analysis (tokenization and normalization)

Links to additional info

See tech council discussion on Oct 14: https://wiki.folio.org/display/TC/2020-10-14+Meeting+notes
Search design discussion: https://wiki.folio.org/display/DD/Search
Draft search planning: https://docs.google.com/document/d/1enxt2ZRwwVXzo9m1Ghl2tcBW6QxFJzlr6wIGAfTO7x4/edit#heading=h.gtly99rhjpm9

Questions

TestRail: Results

Attachments

Issue Links

blocks

FOLIO-2573 Paused due to POC of Elastic Search. All Search Options Should be Supported by a Database Index

Closed

MODINVSTOR-683 Wait for POC of Elastic Search: Slow search in Instances when filtered on createdDate

Closed

MODINVSTOR-692 Unresponsive Instance Query Search

Closed

RMB-685 totalRecordsRounded

Blocked

UIIN-1049 Paused due to POC of Elastic Search. Bugfest: Inventory. Effective Location Result Counts Don't Add Up (Off by Unacceptable Amount) And Seem Suspicious

Closed

UIIN-1070 Paused due to POC of Elastic Search. BugFest: Inventory. Search for Term and Filter by Status = Available Always Results in 1,000 Results

Closed

UIIN-1258 Paused due to POC of Elastic Search. Keyword Search in Inventory app does not work for Japanese title

Closed

UIIN-1365 Paused due to POC of Elastic Search. Honeysuckle bugfest: filter-only search for date created and source takes minutes to complete

Closed

UXPROD-2276 Extend the keyword index. Add barcode to Keyword search in Inventory. Display holdings and item record directly with exact match

In progress

UXPROD-2369 Wait for POC of Elastic Search - Implement exact hit count (when using PostgreSql)

Closed

UXPROD-2623 Wait for POC of Elastic Search - Round estimated search result hit count (totalRecords)

Closed

UXPROD-2687 Wait for POC of Elastic Search. Inventory. When searching by keyword, search options and/or filter, then highlight found holdings or item record, in top of the instance record

Blocked

UXPROD-2712 Inventory Elastic Search (Lotus): Timebox for Priority Inventory Search and Filter Enhancements

Closed

is defined by

MODINVSTOR-647 Refactor ItemStorageAPI and HoldingsStorageAPI before implementing domain events

Closed

MODINVSTOR-674 Implement bulk instance endpoint

Closed

MSEARCH-1 Instance - support keyword search fields

Closed

MSEARCH-2 Instance - HRID searches

Closed

MSEARCH-3 Instance - hierarchical filter by item effective locations - inventory

Open

MSEARCH-4 Instance - filters/facets

Closed

MSEARCH-9 Tenant level language support

Closed

MSEARCH-10 Indexing of instances

Closed

MSEARCH-12 Items - keyword search by effective call number

Closed

MSEARCH-13 [Spike]: Instance - investigate performance impact of autocomplete

Closed

MSEARCH-15 Instance - support keyword search fields - (title, contributors, identifiers)

Closed

MSEARCH-16 Instance - support title keyword search

Closed

MSEARCH-17 Instance - support contributors keyword search

Closed

MSEARCH-18 Instance - support publisher keyword search

Closed

MSEARCH-19 Instance - support keyword search subjects field

Closed

MSEARCH-20 Instance - support keyword search by classifications field

Closed

MSEARCH-21 Instance - support keyword search by notes field

Closed

MSEARCH-22 Instance - support keyword search within electronic access fields

Closed

MSEARCH-23 Instance, holdings and items - support keyword search tags

Closed

MSEARCH-24 Instance - fields supporting phrase match -ISxN Identifiers

Closed

MSEARCH-25 Instance - UUID search

Closed

MSEARCH-26 Holdings - HRID searches

Closed

MSEARCH-27 Items - HRID searches

Closed

MSEARCH-31 Item - keyword search by barcode

Closed

MSEARCH-32 Holdings - hierarchical facet by permanent locations

Open

MSEARCH-33 Item - suppress from discovery filters/facets

Closed

MSEARCH-34 Holdings- suppress from discovery filters/facets

Closed

MSEARCH-35 Holdings - additional fields supporting autocomplete

Closed

MSEARCH-36 Item - additional fields supporting autocomplete

Closed

MSEARCH-43 Instance - reference data - filters/facets

Closed

MSEARCH-44 Item - material types - filters/facets

Closed

MSEARCH-47 Item - combine Item's effective location and item's status filters

Closed

MSEARCH-48 Instance - search by item's effective locations

Closed

MSEARCH-55 Instance - filter/facet by item effective locations

Closed

MSEARCH-56 Holdings - filter/facet by permanent locations

Closed

MSEARCH-58 Support advanced search

Closed

MSEARCH-61 Item - effective location autocomplete

Open

MSEARCH-62 Holdings - permanent location autocomplete

Open

MSEARCH-63 Item - status filters/facets

Closed

MSEARCH-81 Holdings - keyword search by call number

Closed

UISEES-1 Elasticsearch for Inventory search - brainstorming possible approach

Closed

UISEES-2 Autocomplete for reference data - Instance - Mode of issuance

Closed

UISEES-3 Reference data - filters/facets - Instance - Mode of issuance

Closed

UISEES-4 Prepare environment with forked inventory

Closed

UISEES-5 Add link to the existing inventory search

Closed

UISEES-6 Spike - UI for Facet component

Closed

UISEES-7 Inventory clone powered by Elasticsearch

Closed

UISEES-8 Effective locations - hierarchical facet

Open

UISEES-9 Keyword search by title, contributor, identifier textbox

Closed

UISEES-10 Onboarding

Closed

UISEES-11 Spike - Investigate approach for the search update

Closed

UISEES-12 Update icon on Inventory-es

Closed

UISEES-13 Add "Proof of concept" label

Closed

UISEES-14 Instance - language filter/facet

Closed

UISEES-15 Instance - Tags filter/facet

Closed

UISEES-16 Instance - Source filter/facet

Closed

UISEES-17 Instance - Suppress from discovery filter/facet

Closed

UISEES-18 Instance - Staff suppress filter/facet

Closed

UISEES-19 Autocomplete for reference data - Item - material type

Closed

UISEES-20 Reference data - filters/facets - Item - Material type

Closed

UISEES-21 Item - Tags filter/facet

Closed

UISEES-22 Holdings - Tags filter/facet

Closed

UISEES-23 Item - Suppress from discovery filter/facet

Closed

UISEES-24 Holdings - Suppress from discovery filter/facet

Closed

UISEES-26 Replace Query Search with Advance Search option

Closed

UISEES-28 Create CheckboxFilter component with the filtering/show more

Closed

UISEES-29 Update filters options for Instances with the facet data.

Closed

UISEES-30 Update filters options for Items with the facet data.

Closed

UISEES-31 Update filters options for Holdings with the facet data.

Closed

UISEES-32 Replace input-inventory-search control with one textbox

Closed

UISEES-34 Add the supported search options

Closed

UISEES-35 Add suggestions to the “Advanced search” textbox

Closed

UISEES-36 Creating a parser and rendering the results

Closed

UISEES-37 Replacing existing query search

Closed

UISEES-39 Missing dependency in package.json

Closed

UISEES-40 Remove Inventory- es settings

Closed

relates to

MSEARCH-57 Instance, holdings and items - support keyword search by effective location

Closed

(8 blocks, 76 is defined by , 1 relates to)

Activity

People

Assignee:: Magda Zacharska

Reporter:: Magda Zacharska

Front End Estimator:: Magda Zacharska

Back End Estimator:: Magda Zacharska

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 10/Nov/20 4:24 PM

Updated:: 17/Jan/22 5:11 PM

Resolved:: 19/Apr/21 9:46 PM

Create Elasticsearch indexes for Inventory search