Recovery Planning
Sequencing the return to production — prioritizing services by criticality and dependency, pre-return security checks, phased re-enablement, and the rollback plan if re-compromise occurs during recovery.
Eradication is declared complete. The business wants everything back online immediately. But "everything" consists of 230 services with complex dependencies. Bringing up the file server before Active Directory is fully restored causes authentication failures that look like re-compromise. Bringing up a web application before its database dependency is ready causes data corruption. And restoring from a backup that includes the pre-compromise state of a misconfigured service re-opens the initial access vector. Recovery planning is not "turn everything back on" — it's a sequenced, verified re-enablement with a monitoring gate between each phase.
Service Recovery Priority Model
Prioritize by two axes: business criticality (how badly is the business hurt without this?) and recovery risk (how likely is this to re-introduce a problem if we bring it back incorrectly?).
Recovery Priority Matrix
═══════════════════════════════════════════════════════════════════
Business Criticality
HIGH │ [2] Restore quickly │ [1] Restore first
│ with monitoring │ with maximum care
│ e.g. Email, ERP │ e.g. AD, DNS, core auth
─────┼────────────────────────────────────────────────
LOW │ [4] Last │ [3] Restore soon
│ e.g. dev environments │ e.g. internal tools
│ │
└──────────────────────────────────────────────
LOW risk of re-introduction HIGH risk of re-introduction
(from clean backup or reimage) (from in-place, or legacy system)
Recovery phases:
Phase 1 (hours 0-4): Core infrastructure — AD, DNS, DHCP, PKI, core auth
Phase 2 (hours 4-12): Critical business apps — email, ERP, finance systems
Phase 3 (hours 12-48): Business-important apps — collaboration, CRM, reporting
Phase 4 (days 2-7): Non-critical systems — dev environments, legacy systems
Phase 4: Monitor-gate between each phase before proceeding
Service Dependency Mapping
Before building the recovery sequence, map dependencies. Bringing up a dependent service before its upstream dependencies causes outages that look like re-compromise to a stressed IR team.
# Map Windows service dependencies on a single host
# Useful for understanding what must start before what
Get-Service | Where-Object { $_.DependentServices -or $_.ServicesDependedOn } |
Select-Object Name,
@{N="DependsOn"; E={ ($_.ServicesDependedOn | Select-Object -Expand Name) -join ", " }},
@{N="RequiredBy"; E={ ($_.DependentServices | Select-Object -Expand Name) -join ", " }} |
Where-Object { $_.DependsOn -or $_.RequiredBy } |
Format-Table -AutoSize
# For application-layer dependencies, you need documentation or
# infrastructure discovery tools (e.g., Azure Service Map, Dynatrace, etc.)
# A pragmatic approach: interview the application owners for each P1/P2 service
# and record: "This service requires [list] to be available before it starts"Pre-Return Security Checks
Each system must clear security checks before being returned to production. These are in addition to the eradication checklist — these checks are specific to the return-to-production state.
| Check | What it confirms | Who performs it |
|---|---|---|
| Patch level verification | System is at current patch level, including the patch for the initial access vector | Patch management team or IR lead |
| Configuration baseline comparison | Current configuration matches the approved baseline — no settings changed by attacker remain | IR analyst or infrastructure team |
| Backup integrity verification | If restored from backup, confirm the backup predates the compromise and the restore was successful | Backup administrator |
| Credential re-issue confirmation | All service accounts used by this system have been reset and the new credentials are configured correctly | IR identity team |
| EDR agent active and reporting | EDR is healthy, policies applied, reporting to console | Security operations |
| Log forwarding verified | Windows events, application logs reaching SIEM | SIEM team |
| MFA enforced for privileged access | No privileged access to this system is possible without MFA | IAM team |
The Monitored Return Window
When a system returns to production after IR, treat it as an elevated-risk asset for a monitoring period. Any behavioral anomaly should be investigated immediately rather than triaged with standard priority.
Monitored Return Window — 7 Days Post-Recovery
═══════════════════════════════════════════════════════════════════
Day 0-2: Heightened alert monitoring
├── All EDR alerts on recovered systems → immediate escalation
├── New scheduled task or service creation → immediate investigation
├── Authentication anomalies → immediate investigation
└── Outbound connections to new external IPs → block and investigate
Day 2-7: Elevated monitoring
├── Daily review of EDR telemetry on recovered hosts
├── Daily SIEM hunt: IOC list from the incident applied to live traffic
└── Compare: current persistence state vs. post-eradication baseline
Day 7 sign-off: IR lead + SOC lead confirm monitoring window complete
→ Remove elevated alert priority
→ Return to standard SOC monitoring
→ Add permanent detections based on incident IOCs to detection stack
If attacker activity detected during monitoring window:
→ Return to containment phase immediately
→ Re-run full eradication checklist
→ Root cause: what eradication step was missed?
The Rollback Plan
Recovery from a rollback — re-compromise occurs while returning to production — requires having the rollback plan documented before recovery starts.
| Phase | Rollback trigger | Rollback action |
|---|---|---|
| Phase 1 — Core infra returned | Attacker activity on AD or core auth infrastructure | Re-isolate all returned infrastructure, return to eradication phase |
| Phase 2 — Business apps returned | Attacker activity on any Phase 2 system | Re-isolate the specific system, continue recovery for unaffected systems |
| Phase 3/4 — Non-critical systems | Attacker activity on returned system | Re-isolate that system, confirm other returned systems are clean before continuing |
| Any phase | New indicators suggesting broader re-compromise | Full halt on recovery, re-assess eradication completeness across all returned systems |
Recovery from the immediate incident is not the end of IR. Three things must happen after systems return to production before the incident is truly closed: (1) the lessons learned review (Ch15), (2) the detection gap analysis — what new detections would have caught this earlier, (3) the hardening verification — has the initial access vector been permanently closed and verified? Organizations that return to production and declare the incident closed without completing these steps are at high risk of re-attack by the same or related threat actors, who often re-attack within 30-90 days after observing the recovery.
Q & A
Q: Business leadership is demanding the ERP system come back online before Active Directory is fully restored. Is this possible?
In most environments, no — and if it were done, ERP authentication would fail (most ERPs authenticate via AD/LDAP). The IR lead should explain: "Bringing ERP up before AD is restored will result in authentication failures that look identical to re-compromise alerts. It will also require us to temporarily configure the ERP with local credentials that bypass your standard access controls. I recommend restoring AD first — estimated time [X hours] — and then ERP can come up with full authentication working in [Y hours] after that. Bringing them up simultaneously would actually take longer because we'd spend time debugging authentication failures." Frame it as a timeline discussion, not a security vs. business argument.
Q: A backup predates the compromise by 60 days. That means 60 days of data is lost if you restore from it. How do you handle this?
This is a data loss decision requiring authorization from the data owner and the appropriate business executives — not the IR team's call to make unilaterally. The IR team's role is to present the options: (1) Restore from the clean 60-day-old backup — 60 days of data loss but definitively clean. (2) Remediate in place — no data loss, but risk of residual attacker presence. (3) Restore from the 60-day backup and apply transaction logs/WAL to replay more recent transactions (if the database supports this and the logs are available and uncompromised). Present the options with their risks and let the business make the decision. Document the decision and who made it.