Very slow check out behavior: Orchid CSP 3

1. Stanford University will be blocked from going live on Nolana HF1 if this isn't addressed. 2. Stanford University 3. There is no workaround available. 4. Checkin will be impacted 5. To address this, the team made the decision to decouple circulation rules building and parsing from circulation operations as much as possible. Instead of having the code check the age of the cache every time a circ operation occurs, we established a timer endpoint, similar to the ones for operations like aged-to-lost. The timer will rebuild the rules every three minutes by default, and can be customized by users to run as seldom or often as they like. Because the rules are now rebuilt automatically by a separate process, the rules cache code no longer needs to care about how old the cache is or when it was last rebuilt-it just takes whatever is there if the rules have been built, and builds them if they are not present. In addition, we now purge and rebuild the rules cache every time the circ rules are modified. This is necessary for the automated tests in the module to work correctly, and also makes sense for deployed code, as you would want the rules used to be refreshed if you’ve modified them. This should make circ operations speedier, as they now only rebuild the rules if they are not present in the cache, which should only happen if the system has been re-initialized and the timer has not had a chance to run yet. At all other times, it should simply use whatever is in the cache. 6. Because this issue is observable in our current production code, we are planning on testing it by loading a lengthy circulation rules file into one of the development environments (vega scratch), then run circulation operations that require the circ rules (checkin, checkout, requests, etc). We can pretty easily force the circ rules to re-load on every transaction by making slight modifications to the file between transactions. We can observe the delay this causes the operation and also check the logs to see if the thread blocking errors appear. If circulation operations are noticeably faster with the code in place that without it, and the thread blocking errors are not present, we’ve succeeded. 7. Rollback is tricky, because defining “failure” in this area is somewhat subjective. Whether circulation operations experience a delay because of parsing the rules file is dependent on a number of factors outside of our control, the most salient of which are the length of the circ rules file and the computing resources available to the parser on a given system. Also, circ operations consist of a lot more operations than just parsing the rules file. This fix should produce some noticeable improvement for implementers who are currently experiencing delays due to this specific issue, and will definitely remove thread blocking problems. The only reason we could see for rolling this back is if it causes some kind of serious issue, which seems very unlikely.