Append-only: why a correction should never overwrite

There are two ways to correct a wrong number in a system.

You can change it. Or you can add a new entry that says the old one was wrong, by how much, and why.

The first is simpler, produces a tidier screen, and destroys the ability to answer almost every question anybody will later ask. The second costs one extra row.

What an edit actually removes#

When a stored value is overwritten, three things go with it.

What it used to be. Obvious, and the least important.

That it ever changed. This is the real loss. A value that was corrected looks identical to a value that was right the first time, so the fact of the correction — which is often the interesting event — leaves no trace.

Everything that was decided while the old value was in place. Somebody approved a payment against a total. Somebody ordered material against a quantity. Those decisions were reasonable given what was on the screen, and after an edit there is no way to show what was on the screen.

The third is why this is a governance matter and not a database preference. An edited record makes past decisions indefensible, including correct ones.

The shape of an append-only correction#

Do not change the entry. Write another one.

For a stock entry: the original marked cancelled, and a reversing entry of the opposite sign. For a payment: a reversal and a fresh entry. For a quantity: an adjustment with its own date and reason.

There is a detail in the stock version worth borrowing generally. Our reversal writes two halves in one movement: the original marked cancelled, and a reversing correction of the opposite sign which is itself born already cancelled.

That shape is deliberate. A reader that filters out cancelled rows sees neither half. A reader that forgets to filter sees both, and both add to zero. The answer is correct under either reading — which is what you want when you cannot personally audit every piece of code that will ever count your stock.

It is also keyed so that a second reversal of the same entry is impossible, which closes the obvious way to turn a correction into a hole.

The objection, and the answer#

The objection is always the same: the screen gets cluttered. Nobody wants to look at a list of transactions where half the rows are cancelled.

The answer is that presentation and storage are different problems. The default view can show the net position and nothing else. What matters is that the history exists underneath it and can be reached by anybody who needs it, without an engineer.

An append-only ledger with a clean default view is strictly better than a mutable one. An append-only ledger with no view at all is worse, which is why the presentation work is not optional and is usually where these projects fail.

Where the discipline has to hold#

Financial records, obviously.

Stock movements, because a physical balance is the sum of its history and an edited history produces a balance that cannot be explained.

Anything with a reason attached. A write-off, a deduction, a rejection. The reason is the record — the amount is arithmetic anybody could recompute. We refused, deliberately, to allow a payment schedule containing written-off stages to be deleted, because deleting it would take with it every recorded reason for every write-off on it, and those reasons are the only surviving answer to who decided that money would never come.

Approvals and signatures. These attach to a state of the world; if the state can be changed afterwards, the approval means nothing.

The audit trail itself, absolutely and without exception — which is really the same statement, since the trail is the append-only record of everything else.

The trail that cannot be written by hand#

Ours is a hash chain: each entry incorporates a fingerprint of the one before it. Alter, insert or remove an entry and every entry after it stops verifying.

There is a consequence of that which we did not anticipate and have decided to keep. No audit row can be written by hand. Not by a developer with database access, not by an administrator fixing something, not by a migration script. The chain either was produced by the application's own recording path or it does not verify.

That is inconvenient about twice a year. It is also the property that makes the trail worth anything, because a trail that a sufficiently senior person can adjust is a trail that says whatever that person wants it to say.

Derived values are the exception, and must be computed#

Append-only applies to facts. It does not apply to conclusions drawn from them.

A running balance, a total, a status — these should be computed from the entries, not stored and maintained. A stored derived value is a second copy of a fact, and second copies drift.

We learned this expensively. A payment total, a paid flag and a schedule of stages all described the same money, and the flag was never written by the staged path. Nine separate places in the application asked the flag rather than the schedule, each written by somebody who reasonably assumed it meant what it looked like it meant. The account is a fact that had no owner.

So: entries are appended and never edited; conclusions are computed and never stored. Where a stored conclusion is unavoidable for performance, exactly one piece of code should own it, and everything else should ask that code.

What it costs#

Storage, which is free. A little more work at presentation time, which is real. And a genuine discipline problem: somebody will always want to make a bad row disappear, and the honest answer is that they cannot, only that they can correct it visibly.

One further cost is easy to defer and expensive to discover: a history is only yours if it survives the day you leave the system, and most exports carry the current values and none of the corrections — getting your data back out.

That last one is the whole point and the reason it needs to be a rule rather than a preference. A system where the right people can quietly erase things is a system whose records are worth what those people's memories are worth.

The short version#

Correct by adding, never by overwriting.

Write reversals in a shape that reads correctly whether or not the reader filters out cancelled rows. Keep reasons, and refuse to delete anything that carries them. Compute totals rather than storing them. And make the audit trail structurally impossible to write by hand.

Then show a clean view on top of all of it, because a history nobody can read is a history nobody will maintain. What that history has to contain is set out in what an audit trail is for.