Uploaded image for project: 'mod-circulation'
  1. mod-circulation
  2. CIRC-1178

Not all loans are anonymized if there are > 5000 closed loans (with userid)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: P1
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 22.0.2, 22.1.0
    • Labels:
      None
    • Template:
      Standard Bug Write-Up Format
    • Sprint:
      Prokopovych - Sprint 118, Prokopovych - Sprint 119, Prokopovych - Sprint 120
    • Story Points:
      2
    • Development Team:
      Prokopovych
    • Release:
      R2 2021 Bugfix
    • Affected Institution:
      Cornell

      Description

      Overview:
      In an Iris Hotfix-2 environment (with loan history per attached image) we
      observe that the process for anonymizing loans is running (image below) and that SOME loans are getting anonymized but not ALL of the loans which should be anonymized.

      There are approximately 8K loans that are closed and have a userId

      SELECT * FROM <tenant>_mod_circulation_storage.loan where jsonb->'status'->>'name' = 'Closed' and jsonb->>'userId' is not null
      

      Of the 8K loans, approximately 6K have associated fee fines and should NOT be anonymized

      with closed_loans_with_users as
      (
      SELECT * FROM tenantid_mod_circulation_storage.loan where jsonb->'status'->>'name' = 'Closed' and jsonb->>'userId' is not null
      )
      select * from closed_loans_with_users cu
      where EXISTS (
      select ac.jsonb->>'loanId'
      FROM tenantid_mod_feesfines.accounts ac, closed_loans_with_users cl
      where ac.jsonb->>'loanId' =  cu.id::text)
      

      One possible reason for this is due to fetching of 5K loans by the anonymize process
      https://github.com/folio-org/mod-circulation/blob/c27229437f5029b8afe9ba8eecd2c58e67c3bf01/src/main/java/org/folio/circulation/domain/anonymization/service/LoansForBorrowerFinder.java#L27

      Possible reason for this is:
      When the system fetches the first 5000 loans to anonymize, those are made up of loans that have open fees and fines, which means they won’t be anonymized.
      This stops other loans which don’t have open fees / fines and are older than the threshold for anonymization from being picked up.

      Request is for additional logging to help confirm this is the reason for failure of loans being anonymized (and to resolve issue if confirmed)

      Steps to Reproduce:

      1. Set up loan history (so that loans are anonymized - per image)
      2. Environment contains >5K loans that are not anonymized (due to fee/fines associated with the loans)

      Expected Results:
      Closed loans without fees and fines are anonymized based on systems return date
      Closed loans with fees and fines (and all fees / fines are closed) are anonymized based on systems return date
      Actual Results:
      Only some of Closed loans that should be anonymized are

      Chosen (interim) solution

      • increase the number of loans checked to 50 000 (can be overridden by an environment variable)
      • increase the interval between runs to 1 hour (can be overridden in Okapi)
      • increase the guideline memory requirements for the module (by 256Mb)

      Additional Information:
      URL:
      Interested parties:
      Marc Johnson Anya Arnold Debra Howell Holly Mistlebauer

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                marcjohnson Marc Johnson
                Reporter:
                cgodfrey Carole Godfrey
                Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases