BE Teck Notes Back to be-teck.com

5 min read

An alert that rings for everything is an alert nobody hears

Priority is the wrong axis, because everybody's work is important. Sort by what the message asks the reader to do, and there are only three answers.

Every organisation that installs a notification system goes through the same arc.

The alerts are useful. More things become alerts, because each addition is individually justified. The volume rises. People stop reading. Somebody proposes a priority flag. Everything becomes high priority, because everybody's own work is important. People stop reading the high priority ones.

At the end of it the system delivers nothing, and the organisation has spent a lot of engineering effort to arrive at silence.

Importance is the wrong axis#

The instinct, when a message is missed, is to reach for importance: mark it urgent, give it a red badge, put it higher in the list.

That fails for a structural reason. Importance is assessed by the sender, and every sender's own work is important to them. A scale on which everything can be placed at the top is not a scale. Within a few months the flag carries no information at all, and you have taught people to ignore the one visual signal you had.

The useful question#

The question that does work is not how important a message is. It is what it asks the reader to do, and there are only three answers.

  • Nothing. You should know this happened. Read it whenever.
  • Later. Look at this next time you are looking at things.
  • Now. Somebody is standing still until you act.

That is a property of the message, determined by its type, not a judgement made by a sender about their own priorities. Which is exactly why it stays meaningful: nobody can inflate it.

A signature request is always the third. It is not information about work; it is a request for a specific person's hand, and until that hand moves a document, a payment and usually another human being are all stopped.

A comment on a task is almost always the first. A task assigned with a date next week is the second, and so is a follow-up owed to a customer, provided somebody put a date on it rather than an intention — follow-up is a schedule, not a feeling.

What went wrong for us#

A document was sent for signature. It sat there. The person who had to sign did not know they had to sign it.

The request went out carrying a generic type — question — and types are mapped onto rows in a routing grid. Question had been mapped, sensibly enough, onto the row for comments, whose default delivery is a bulletin: held back until the next scheduled broadcast, ringing nothing, arriving as one line among a dozen others that also did not need doing right now.

A document holding up a payment was delivered as chatter.

Every component in the chain did its job correctly. The design had simply never been asked what a signature request is. It now has its own type, sits on the row for things awaiting approval, and says plainly that it asks for an act. It arrives on its own and it rings. It also says which page of the document the signature goes on, because sign this without here is still a small piece of homework. The full account is filed as chatter.

Quiet is a feature#

The fix, when this surfaces, is almost never to make things louder.

Loudness is a resource with a fixed supply. An application that rings for everything is an application people mute, and a muted application delivers nothing at all.

The quiet tier is doing real work for the messages that genuinely belong to it. The mistake was never that quiet delivery exists — it was that a request for action had been classified into it.

So the repair is to stop lying about what the message wants. Once a request says this asks somebody to act, the routing follows on its own and nobody has to argue about priority.

Accuracy is the whole asset#

A separate failure, and the more expensive one.

A chaser in our system rang the finance team every few days about money that had already left the account, because a payment made in stages never wrote the field that meant this is paid.

Nobody reported it. That is the part worth sitting with. It is not that somebody noticed and was ignored. It is that a recurring reminder which is occasionally wrong teaches you to skim the whole class of reminder, and once you skim it, the one that matters costs the same as the ones that do not.

The chaser was not merely useless. It was spending the credibility of every other chaser in the building. An alerting system's accuracy is not a quality attribute to be improved over time — it is the entire asset, and a system that is right most of the time is worth close to nothing.

The underlying data fault is described in a fact that had no owner.

One event, one message#

A smaller discipline that matters more than it sounds.

A delivery failure in our system used to open one ticket per recipient. So a single five-minute outage during a message to eight people produced eight tickets, each needing its own manual close.

The outage was one event. It is now one ticket, with the affected recipients carried inside it.

Any alerting design should be checked for this: what happens when one cause produces many symptoms. Systems that alert per symptom flood at exactly the moment attention is scarcest, which is the worst possible timing. There is more on that in one outage, one ticket.

Tell the person who can act#

The last rule, and it is the one that most often gets a system used rather than disabled.

Six mailboxes in our estate had lost their permission to send, one of them for over a week, and they stayed broken because of who was being told. The failure raised a ticket for the IT team — but IT cannot reconnect somebody else's mail account. Only its owner can, from their own browser, with their own password.

So the ticket sat open while the person whose mail had quietly stopped flowing never heard a word.

The routing question is not only how loud. It is to whom, and the answer is whoever can actually do something. A correctly urgent alert delivered to somebody powerless is the same as no alert, plus a false sense that it was handled.

The short version#

Route by what a message asks the reader to do, not by how important its sender thinks it is. There are three answers and only one of them rings.

Keep the quiet tier, because it is what makes the loud one mean something. Make sure the alerts are right, because a chaser that is sometimes wrong spends the credibility of every other chaser you have.

Collapse many symptoms into one event. And send it to the person who can act, which is frequently not the person your escalation table names.

Have a gap worth closing?

If something in your daily work is broken in a way everybody has stopped complaining about, that is exactly what we want to hear.

Write to hello@be-teck.com

More notes