Uploaded image for project: 'mod-orders-storage'
  1. mod-orders-storage

Implement transactional outbox pattern to store audit log in the transactional manner



    • ACQ Sprint 155, ACQ Sprint 156
    • 5
    • Thunderjet
    • Orchid (R1 2023)
    • TBD


      An order/order line update process needs to atomically update the database and send messages/events to track audit log to avoid data inconsistencies and bugs.

      There’s no guarantee that the transaction will commit. Similarly, if a service sends a message after committing the transaction there’s no guarantee that it won’t crash before sending the message. This problem can be solved using transactional outbox pattern 


      Need to perform following steps:

      • create new table that will store events 
      • modify source code of order/order line create/edits to store events into new separate table
      • implement separate process that will fetch data from new table and send messages to kafka

      Flow described in this diagram:

      More details can be found here: https://microservices.io/patterns/data/transactional-outbox.html 

      There is potential issue with this apporach when there is more than 1 instance of mod-orders-srotage deployed. It can lead to situation that these instances simultaniously reading messages from this new 'Messages' table and we will have message's duplicated.

      Approach to implement this:

      • Add Timer Interface to mod-orders-storage https://github.com/folio-org/okapi/blob/master/doc/guide.md#timer-interface . Okapi will call necessary logic in mod-orders-storage to reade outbox table and send messages to kafka. Examples can be found in mod-users, mod-licenses, mod-data-export, mod-circulation-storage
      • Create new table outbox_table_bloker with one line
      • Implement listener in mod-orders-storage for this call from Okapi. This listener should call  SELECT * FOR UPDATE FROM OUTBOX_TABLE_BLOCKER. If without success - nothing need to do, otherwise - continue the main logic to read messages from outbox_table and send them to kafka. After finish need to do commit/rollback for outbox_table_blocker

      This approach well documented in this document: https://wiki.folio.org/display/DD/Implementing+the+Transactional+Outbox+pattern 

      TestRail: Results


          Issue Links



                Nosko Serhii Nosko
                Nosko Serhii Nosko
                Dennis Bridges Dennis Bridges
                0 Vote for this issue
                2 Start watching this issue



                  TestRail: Runs

                    TestRail: Cases