# Blameless Postmortems: The Incident Review That Changes Something

URL: https://donedeploy.com/incident-postmortem/
Published: 2026-11-20

Most teams write postmortems. Far fewer get anything out of them. The document is written a week late by whoever was on call, it lists a timeline and a root cause, it ends with five action items, and three months later the same outage happens again while those items sit in a backlog nobody opens.

The problem is rarely the template. It is everything around it: which incidents get a review, who writes it, how the meeting runs, and whether the action items have an owner and a date. This post is the incident postmortem process we set up with clients, in the order it happens, with the template at the end.

## What a postmortem is for

One line: a postmortem turns an incident into a change that makes the next one smaller, rarer or shorter.

If a review does not lead to a change in code, configuration, alerting, runbooks or process, it was a story, not a postmortem. Stories are pleasant to read. They do not stop the pager.

## Why blameless is not optional

“Blameless” gets treated as a soft value. It is a practical one. In a review where people expect blame, they leave out the detail that matters most: what they saw, what they believed at the time, and why the action they took looked right. Without that, you cannot see the real cause, which is almost always a system that made the wrong action easy.

Blameless does not mean nobody is accountable. It means the question changes:

Instead of asking Ask Who ran the command? What made running that command look safe? Why did nobody notice? What did the alerts and dashboards show at that moment? Why was the runbook not followed? Was the runbook findable, correct and usable under pressure? Who approved this change? What did the review show, and what could it not show?

If the answer to an incident is “be more careful”, the review has stopped too early. Careful is not a control.

## When to write one

Decide the triggers in advance and write them down, so nobody has to argue on the day about whether an incident “deserves” a review. Typical triggers:

- Customers were affected beyond an agreed threshold, such as a period of errors or unavailability your team has defined.
- Data was lost, corrupted or exposed, at any scale.
- A rollback or a manual change to production was needed to recover.
- An SLO burned through a large share of its error budget. If you have not set those up yet, [SLOs You Can Defend](https://donedeploy.com/slos-you-can-defend/) is a good place to start.
- The on-call engineer asks for one. This trigger matters more than it looks: near misses are the cheapest incidents to learn from.

Small incidents can share a lighter format. A short review of a near miss is worth more than a long review nobody has time to write.

## Capture while it happens

The best postmortems are half written before the incident ends. During the response:

- **Keep one incident channel** and post every observation, decision and action there, with a timestamp. Chat history becomes the timeline.
- **Name an incident lead** who coordinates, and a separate person who takes notes if the incident is large. The person typing commands should not also be writing the record.
- **Save the evidence early.** Dashboards, graphs, log queries and the exact deploy or change involved. Retention windows close, and a graph that is gone cannot be added later.

This is also where tooling earns its keep. Reconstructing a timeline from four systems is the slowest part of writing a review, and it is the kind of reading work we hand to automation, as we described in [AI Agents in Your Infrastructure](https://donedeploy.com/ai-agents-in-production-infrastructure/). A drafted timeline is a starting point; the people who were there check it.

## Write the draft within days, not weeks

Memory fades fast. Aim to have a draft within a few working days, written by someone who was involved, ideally with help from the person who was on call rather than as an extra job for them alone.

The draft has a clear order:

1. **Summary.** Three or four sentences a manager or customer-facing colleague can read: what broke, who was affected, for how long, and whether it is fixed.
2. **Impact.** What users saw, which services and regions, and any data affected. Use the numbers you have from monitoring; do not estimate what you can measure.
3. **Timeline.** Timestamps from first signal to full recovery, including when the issue started, when it was detected, when someone responded and when it was mitigated. The gaps between those points are often more useful than the cause.
4. **Contributing factors.** Not a single root cause. Real incidents usually need several things to go wrong at once: a change, a missing check, an alert that did not fire, a dependency that behaved differently. List each.
5. **What went well.** The alert that did fire, the rollback that worked, the runbook that helped. These are the things to protect.
6. **Action items.** Covered below, because they decide whether any of this was worth it.

## Run the review meeting

The meeting is not where the document is written. It is where the people involved test it.

- **Share the draft beforehand** so the meeting is spent on questions, not reading.
- **Have a facilitator** who was not deeply involved. Their job is to keep the questions on systems, not people, and to stop the room from jumping to fixes before the facts are agreed.
- **Walk the timeline** and ask, at each decision point, what the person knew and what they could see.
- **Agree the contributing factors**, then the action items, in that order.

An hour is usually enough. If it is not, the incident may need two reviews, or the draft was not ready.

## Action items that get done

This is where most postmortem processes fail. A good action item is:

- **Specific.** “Add an alert on queue age for the billing worker”, not “improve monitoring”.
- **Owned.** One named person, not a team.
- **Dated.** A target date agreed in the meeting.
- **Tracked where work is tracked.** A ticket in the same backlog as feature work, linked from the postmortem, not a list at the bottom of a document.
- **Sized honestly.** Split items into quick fixes for this sprint and larger changes that need planning. A review with only large items will produce nothing for months.

Prefer changes that remove a whole class of failure over changes that ask people to remember something. A check in CI that blocks a bad config, as in [the CI process](https://donedeploy.com/ci-process/), beats a line in a checklist. A credential that cannot delete a production volume beats a note saying “do not delete production volumes”. If the incident involved a leaked credential, the response steps in [Best Practices for Secrets Management](https://donedeploy.com/secrets-management-best-practices/) belong in the action list too.

Then review the open items on a schedule, monthly works for most teams, and treat an overdue item from an incident like an overdue security fix.

## Share what you learned

A postmortem read by three people teaches three people. Publish reviews where the whole engineering team can find them, keep them in one place with consistent titles and tags, and read across them every few months. Patterns show up only in the collection: the same service, the same kind of change, the same gap between when an issue starts and when anyone notices.

## The template

Copy this into your wiki or repository and adjust it to your team:

```
# Incident: <short description>  (<date>)
Status: draft | reviewed | actions complete

Incident lead:          Author:          Review date:
## Summary

What broke, who was affected, for how long, current state.
## Impact

User-visible effect, services and regions, data affected, figures from monitoring.
## Timeline (UTC)

- hh:mm  Issue starts

- hh:mm  First alert or report

- hh:mm  Responder acknowledges

- hh:mm  Mitigation applied

- hh:mm  Full recovery confirmed
## Contributing factors

- ...
## What went well

- ...
## Where we got lucky

- ...
## Action items

| Action | Type (prevent / detect / respond) | Owner | Due | Ticket |

|---|---|---|---|---|

```

The “where we got lucky” section is worth keeping. It names the things that would have made the incident much worse, and those are often the best action items of all.

## Where to start this week

- Write down your postmortem triggers and share them with whoever is on call.
- Take your last incident and check its action items: how many have an owner, a date and a ticket?
- Put the template above where your team will find it during an incident, not after.

DoneDeploy sets up on-call, incident runbooks and a post-incident report template as part of our [AI observability](https://donedeploy.com/devops-services/monitoring/) work, with engineers checking every report. If the same incidents keep coming back for your team, [book a free call](https://donedeploy.com/lets-talk/) and we will go through your last few with you.
