Incident command
Give each incident phase an owner and an evidence output
The phases can overlap. Preserve facts, distinguish confirmed impact from hypothesis and keep one decision log while responders work.
Detect and declare
Recognize a user or security promise is at risk.
- Owner
- Incident commander
- Evidence
- Trigger, scope, time and initial severity
- Decision
- Declare, monitor or escalate
Stabilize
Stop additional harm while preserving necessary evidence.
- Owner
- Technical lead
- Evidence
- Actions, access, snapshots and side effects
- Decision
- Contain, isolate or disable
Understand
Build a timeline and test competing explanations.
- Owner
- Investigation lead
- Evidence
- Correlated logs, changes, provider events and gaps
- Decision
- Expand or narrow affected scope
Communicate
Tell affected people what is known and what to do.
- Owner
- Communications lead
- Evidence
- Approved facts, audience and update cadence
- Decision
- Internal, customer, provider or authority notice
Recover
Restore service and verify critical user journeys.
- Owner
- Recovery lead
- Evidence
- Clean state, acceptance checks and monitoring
- Decision
- Reopen, stage or roll back
Learn
Remove causes and improve detection without blame theater.
- Owner
- Product and engineering owner
- Evidence
- Timeline, contributing conditions and assigned actions
- Decision
- Fix, accept, redesign or retire
Operating principle
Keep one factual timeline and one decision authority
Multiple responders can investigate in parallel, but conflicting changes and private side conversations make the incident harder to understand. Record timestamps, observations, commands, decisions and owners in one controlled log.
The incident commander coordinates consequence and priorities; the deepest technical expert does not automatically own every business or communication decision.
- Preserve original evidence
- Rotate exposed credentials
- Separate facts from hypotheses
- Verify recovery with user journeys
Applied example
Incident example: leaked provider key
A production API key appears in a public repository commit. The team deletes the file but does not know whether the key was used or copied into build logs.
- Deleting the file does not revoke the credential
- Repository history and downstream copies remain
- Provider audit events can help bound exposure
- Rotation may affect active workloads
Plain answers
Questions to resolve before shipping
When should a small team declare an incident?+
Define triggers from consequence, uncertainty and urgency. Formal declaration can improve coordination even before the root cause is known.
Should we immediately take the site offline?+
Choose containment from likely harm and reversibility. Some incidents require shutdown; others are safer with targeted isolation.
What belongs in a post-incident review?+
A factual timeline, impact, detection, contributing conditions, effective responses and owned corrective actions with verification.