,

Anatomy of a Production Incident: Our Postmortem Template

Anatomy of a Production Incident: Our Postmortem Template — T-Square engineering blog

TL;DR — A postmortem is not “what went wrong” — it is “what would have caught this earlier, and why did it not?”. Our template has seven sections and explicitly forbids individual blame. The goal is system improvement, not accountability theatre.

Blameless postmortem template — seven sections“What would have caught this earlier?” is the section most teams skip. Without it, the same incident recurs. — /postmortem sections 1 · Summaryone paragraph, plain language 2 · Timelinedetect → escalate → resolve 3 · Impactquantified, not vibes 4 · Root causereproducible technical 5 · What would havecaught this earlier?THE section most teams skip 6 · Follow-ups (owned) 7 · Lessons names go in the timeline · never in the root cause
“What would have caught this earlier?” is the section most teams skip. Without it, the same incident recurs.

Every team has incidents. Mature teams turn incidents into permanent improvements. Immature teams turn them into a finger-pointing meeting and the same incident happens again six months later. The difference is the postmortem process.

The seven sections

1. Summary

One paragraph: what happened, who was impacted, how long it lasted, how it was resolved. No jargon. A non-engineer should understand it.

2. Timeline

Timestamps, in order. First detection, escalation, initial mitigation attempt, full resolution. Plus the moment when someone first noticed the precursor (often earlier than detection).

3. Impact

Who was affected, how, and how much. Number of users, requests, dollars, hours. Quantified, not vibes.

4. Root cause

The actual technical cause, written so someone could reproduce the failure. Not “we missed something” — what specifically failed, in what code path, under what conditions.

5. What would have caught this earlier?

The most important section, and the one most teams skip. Test? Type? Monitor? Code review checklist? Specific, actionable, build-able.

6. Follow-ups

Concrete tickets, owners, dates. Each one tied back to “what would have caught this earlier?”. If the follow-up list is empty, the postmortem is incomplete.

7. Lessons

Generalizable principles. What does this incident tell us about our system as a whole? Often a one-line maxim that stays with the team for years.

The blameless rule

Names go in the timeline for context. They do not go in the root cause. “Engineer X pushed bad code” is not a root cause — “we had no canary deployment to catch X's bad code before it reached 100% of traffic” is. The system, not the human.

Frequently asked questions

Should every incident get a postmortem?

Every incident that hit users, or that could have if circumstances were slightly different. Internal-only “we caught it before it shipped” still warrants a postmortem because the system that almost shipped it is the same system that will ship the next one.

How public should postmortems be?

As public as the customer impact warrants. Customers who were impacted deserve more than a Slack one-liner. Internal-only postmortems for issues that did not reach users.

Working on something similar?

T-Square architects, builds and operates production systems for learning, AI and custom software products. Talk to a senior engineer for a second opinion.

— /more

Keep reading