The machine proposes, a person decides

The rule is short, and in our own work it came from the owner of the business rather than from anybody building the software: never auto-link anything.

A machine may look at a bank transaction and conclude it paid a particular invoice. It may look at a photograph of a delivery slip and conclude it belongs to a particular order. It may look at two tasks and conclude they are the same piece of work.

In every case the conclusion is recorded as a suggestion. A person taps to confirm or to reject. Only the tap makes it a link. Merging two records that appear to name the same buyer is the same act with more at stake, because it also settles whose enquiry it is — one buyer, three enquiries.

Why, given that the machine is usually right#

The objection is reasonable. If the match is correct nine times out of ten, insisting on a tap costs nine unnecessary actions to prevent one error.

The answer is that the cost of the tenth is not what it appears to be.

An automatic link produces a record that is indistinguishable from a human decision. Six months later somebody reads this transaction paid that invoice and has no way of knowing that nobody ever agreed. They act on it. They reconcile against it. They tell a vendor something based on it.

The error is not the wrong pairing. The error is the false confidence in the record afterwards, and that confidence is unbounded — it propagates into every decision downstream, and none of those decisions carry a marker saying they rested on a guess.

A wrong suggestion, by contrast, is refused in two seconds and leaves nothing behind.

Asymmetry, not caution#

This is not a general position that machines should be distrusted. It is a specific observation about asymmetric costs.

The cost of asking is small, bounded, and paid immediately by somebody who has the context. The cost of a wrong automatic action is unbounded, paid later, by somebody who does not.

Whenever costs are shaped like that, the answer is to ask. Where they are not — where a wrong action is cheap and easily reversed, and asking is expensive — automate freely. Sorting, ranking, filtering, drafting, extracting, summarising are all fine. It is asserting a fact into the record that requires a person.

What we had to undo#

When we adopted the rule, some links had already been created automatically by the machine's own authority.

We converted them back into suggestions, in the same migration that introduced the rule.

There was no honest alternative. A record created by a mechanism you have since decided is not trustworthy does not become trustworthy by being a few weeks old. Leaving them would have meant a permanent stratum of records that looked like decisions and were not, with no way to tell them apart afterwards.

That is an uncomfortable amount of work to take on for a principle, and it is the part that tells you whether the principle is real.

What a good suggestion looks like#

Six properties, learned mostly by getting them wrong.

It carries its reasoning. Not a score — a sentence. Exact amount, vendor name in the narration is something a person can agree or disagree with. A number out of a hundred is something they either believe or ignore. The general argument is the app should tell you why.

It meets an evidence floor. A weak guess is worse than no suggestion, because reviewing it costs the same as reviewing a good one. We found that a near-miss amount alone was being treated as evidence, and it is not — it is an absence of contradiction. It now needs corroboration beside it. The mechanics are in matching a bank line.

The same floor applies everywhere. We had a second surface offering the same kind of pairing, written separately, with its own lower threshold. So a match the main matcher would have refused was available as a one-tap action somewhere else. One rule, one implementation, asked by every screen.

Both answers are permanent. A rejection is remembered, so the same wrong pair is never proposed again. A suggestion system that forgets its rejections re-asks the same question weekly and is muted within a month.

It is reachable. Ours were not, for a while — the suggestions were being hidden by the very filter meant to rank them, and a related action existed as code with no button anywhere in the application. Built is not the same as reachable, which is its own piece: written, and never wired.

Undo is complete. Undoing an attachment must also close the open question it answered, or a stale card keeps rendering on a record the money never belonged to.

Where the line falls#

It is worth being concrete, because "propose, do not act" can be read as paralysis.

Machine acts freely: ranking a queue, extracting text from an image, transcribing speech, drafting a message, computing a total, flagging an anomaly, sorting by risk, filling a form field the person can see and change before submitting.

Machine proposes only: creating a link between two records, attaching money to an order, marking something paid, closing a task, sending a message to a person outside the company, changing a permission, writing to an audit trail on somebody's behalf.

The test is simple. If the action produces something a future reader would interpret as a decision, a person has to have made it.

It is also better for the machine#

An unexpected benefit: confirmations and rejections are training data.

A system that acts alone learns nothing, because it never finds out whether it was right. A system that proposes accumulates, with every tap, a labelled judgement from somebody who knew the answer. Ours uses confirmed matches to learn vendor labels, which improves later suggestions.

Asking is not the price of safety here. It is the mechanism by which the thing gets better.

The short version#

Never let a machine assert a fact into the record on its own authority.

Not because machines are unreliable, but because an automatic error is indistinguishable from a decision, and it will be trusted by people who cannot know it was never made.

Propose, with reasoning, above an evidence floor, from one implementation, remembering both answers, with a complete undo.

Then a wrong guess costs two seconds instead of a year.