Details
-
Story
-
Status: Closed (View Workflow)
-
P3
-
Resolution: Done
-
None
-
-
ACQ Sprint 155, ACQ Sprint 156
-
5
-
Thunderjet
-
Orchid (R1 2023)
-
TBD
Description
Purpose/Overview:
An order/order line update process needs to atomically update the database and send messages/events to track audit log to avoid data inconsistencies and bugs.
There’s no guarantee that the transaction will commit. Similarly, if a service sends a message after committing the transaction there’s no guarantee that it won’t crash before sending the message. This problem can be solved using transactional outbox pattern
Requirements/Scope:
Need to perform following steps:
- create new table that will store events
- modify source code of order/order line create/edits to store events into new separate table
- implement separate process that will fetch data from new table and send messages to kafka
Flow described in this diagram:
More details can be found here: https://microservices.io/patterns/data/transactional-outbox.html
There is potential issue with this apporach when there is more than 1 instance of mod-orders-srotage deployed. It can lead to situation that these instances simultaniously reading messages from this new 'Messages' table and we will have message's duplicated.
Approach to implement this:
- Add Timer Interface to mod-orders-storage https://github.com/folio-org/okapi/blob/master/doc/guide.md#timer-interface . Okapi will call necessary logic in mod-orders-storage to reade outbox table and send messages to kafka. Examples can be found in mod-users, mod-licenses, mod-data-export, mod-circulation-storage
- Create new table outbox_table_bloker with one line
- Implement listener in mod-orders-storage for this call from Okapi. This listener should call SELECT * FOR UPDATE FROM OUTBOX_TABLE_BLOCKER. If without success - nothing need to do, otherwise - continue the main logic to read messages from outbox_table and send them to kafka. After finish need to do commit/rollback for outbox_table_blocker
This approach well documented in this document: https://wiki.folio.org/display/DD/Implementing+the+Transactional+Outbox+pattern
TestRail: Results
Attachments
Issue Links
- clones
-
MODAUD-141 Add Kafka infrastructure code to mod-audit
-
- Closed
-
- defines
-
UXPROD-3215 Display a history of edits for the Order and Order line records
-
- Closed
-
- has to be done before
-
MODORDSTOR-325 Replace PGUtil for Edit Order to use PostgresClient directly
-
- Closed
-
-
MODORDSTOR-326 Replace PGUtil for Edit Order Line to use PostgresClient directly
-
- Closed
-
- is cloned by
-
MODORDSTOR-324 Add audit log message schemas and producer for sending these Kafka messages
-
- Closed
-