Skip to content

Single-transaction commit for records and audit marker

The ingested records and the file-level audit marker are written in a single database transaction — which means both live in the same database. This eliminates the class of failures where records land but the marker doesn't (causing duplicate loads on retry) or the marker lands but records don't (causing silent data loss). The alternative — a separate audit store updated after the data write — requires a two-phase commit or saga pattern to be safe, which is complex and still leaves a failure window. Constraining the destination to a transactional database (PostgreSQL) is the price; it rules out data lakes and columnar stores as the primary destination, but that is an acceptable constraint for a system where reliability is the core value proposition.