We have a process in place to load holdings into our DB table - make requests to RM API holdings endpoints and populate holdings table in our DB. This process uses the https://github.com/folio-org/okapi/blob/master/doc/guide.md#timer interface and populates data every 2 days. This process is fragile in the sense that anything could go wrong in hitting the RM API endpoints or populating our database and we need to figure out how we can harden the process.
Few approaches that we discussed:
1. Create a separate endpoint that gives us the status of the progress of the process.
2. Add retry mechanism in case something fails - at least 3 retries
3. What happens if someone tries to filter resources by tags while the table is being populated/has no entries - we end up making multiple requests to RM API - which can be avoided if we add an updated_at column to our holdings table. Based on when the entry was last updated, we decide whether we want to insert/update the entry or leave it as is - this eliminates the need to truncate the entire holdings table and re-populate increasing performance.
4. What happens if RM API is down? Approach outlined in 3. helps us still retain holdings data albeit stale.