Chapter 14

Monitored Recovery

What to watch for as systems return to production — the specific SIEM queries, EDR hunts, and behavioral indicators that catch re-compromise during the return window.

Scenario

Day 1 of recovery. The first batch of systems is back online. Your SOC has 200 normal daily alerts. On top of that, you've added every IOC from the incident to the watchlist — 15 IPs, 8 domain names, 4 file hashes, and 3 usernames. The recovery monitoring window is supposed to be "heightened," but in practice, the SOC is drowning in alerts and the recovery-specific monitoring is getting the same triage priority as a routine phishing alert. This chapter is about making monitored recovery concrete — specific hunt queries, specific EDR watchlists, and a daily review structure that doesn't drown the team.

Daily SIEM Hunt Queries During Recovery

Run these queries daily against the previous 24 hours during the recovery monitoring window. Each is targeted at re-compromise indicators specific to the incident.

Bashrecovery-hunts.splunk
# Splunk hunt queries for recovery monitoring window
# Run daily against the last 24 hours

# 1. Connections to incident-specific IOC IP list
index=network_traffic earliest=-24h
| where dest_ip IN ("185.220.101.47", "45.142.212.100")  # IOCs from this incident
| stats count by src_ip, dest_ip, dest_port, app
| sort -count

# 2. Authentication by incident-specific accounts (should be disabled/reset)
index=windows_security earliest=-24h EventCode=4624
| where Account_Name IN ("jsmith","mwilliams","svc_deploy")
| stats count by Account_Name, Logon_Type, Workstation_Name, IpAddress

# 3. New scheduled tasks or services on recovered systems
index=windows_security earliest=-24h EventCode IN (4698, 7045)
| where host IN ("LAPTOP-01","SERVER-FILES01","DC-02")  # recovered systems list
| table _time, host, EventCode, TaskName, ServiceName, ImagePath, AccountName

# 4. PowerShell command execution with specific IOC strings
index=windows_powershell earliest=-24h EventCode=4104
| where ScriptBlockText LIKE "%185.220.101.47%" OR ScriptBlockText LIKE "%evil-c2%"
| table _time, host, ScriptBlockText

# 5. DNS queries for sinkholed domains (indicates infected host still beaconing)
index=dns earliest=-24h
| where query IN ("evil-c2-domain.com","another-c2.net")  # sinkholed domains
| stats count by src_ip, query
| sort -count

EDR Watchlist Configuration

During the recovery window, configure specific detection rules in your EDR that would alert immediately on re-compromise indicators — separate from the general detection stack.

Indicator typeEDR actionAlert priority
File hash from incident malwareCreate custom IOC with "detect and prevent" action; alert to IR lead direct, not standard SOC queueP1 — immediate
Known C2 IP/domain from incidentNetwork connection block + alert for any recovered host attempting connectionP1 — immediate
Malware family behavioral pattern (e.g., Cobalt Strike beacon behavior)Behavioral rule if EDR supports it; or enable enhanced Cobalt Strike detection policyP1 — immediate
New account creation on recovered systemsAlert on Event 4720 (local user created) and Event 4728/4732 (added to group) from recovered hostsP2 — within 30 min
Disabled account authentication attemptsAlert on Event 4625 (logon failure) for specifically disabled incident accountsP2 — within 1 hour

Behavioral Indicators of Re-Compromise

Beyond the specific IOCs from the incident, these behavioral patterns indicate re-compromise regardless of whether the same tools are being used.

  Recovery Monitoring: Behavioral Re-Compromise Indicators
  ═══════════════════════════════════════════════════════════════════

  HIGH CONFIDENCE indicators (investigate immediately):
    ├── New persistence mechanism on recovered host (after baseline capture)
    ├── Outbound connection to new external IP from recovered host
    ├── Authentication using a previously disabled account
    ├── New admin account created on recovered host
    └── Process spawning unusual child processes on recovered host
        (lsass.exe spawning a child, Word.exe launching PowerShell, etc.)

  MEDIUM CONFIDENCE indicators (investigate within 4 hours):
    ├── Unusual scheduled task timing on recovered host
    ├── Large outbound data transfer from recovered host to external IP
    ├── Service account used in a new location or new service
    └── LSASS memory access from non-EDR, non-AV process

  LOW CONFIDENCE (investigate within 24 hours — may be normal):
    ├── New software installed on recovered host
    ├── PowerShell script execution (if AMSI/SBL enabled, review script text)
    └── Remote desktop connection from unfamiliar source IP

  When any HIGH CONFIDENCE indicator triggers:
    → Immediately re-contain the affected host (EDR isolation)
    → Notify IR lead
    → Do NOT wait to confirm before isolating

Daily Review Structure

The recovery monitoring period needs a structured daily review to prevent alert fatigue from killing the heightened monitoring posture.

  Daily Recovery Monitoring Review (30 minutes)
  ═══════════════════════════════════════════════════════════════════

  Participants: IR lead, SOC lead, infrastructure lead

  1. SIEM hunt results review (10 min):
     Run the 5 daily hunt queries
     Any hits → immediate investigation
     Zero hits → document "clean" and proceed

  2. EDR watchlist review (5 min):
     Any EDR alerts on the recovered-host watchlist in past 24h?
     Review and disposition each

  3. New system return status (10 min):
     What systems are coming back online today?
     Pre-return checks complete for each?
     Any concerns before authorization?

  4. Rollback review (5 min):
     Any anomalies that warrant re-evaluation of the eradication
     declaration? Any patterns that suggest missed attacker persistence?

  Gate criteria to end recovery monitoring window:
    ✓ 7 days with zero re-compromise indicators
    ✓ All planned systems returned to production
    ✓ SOC lead and IR lead both sign off

Q & A

Q: Three days into recovery, you get an EDR alert on one of the recovered hosts that turns out to be a false positive from a new software deployment. How do you handle this without desensitizing the team?

Tune the specific false positive rule, document the tuning decision with the reason, and communicate to the team: "This alert on HOST-X was a false positive because [admin software deployment]. We've tuned the rule to exclude that specific behavior. All other instances of this alert remain high-priority." The discipline is to tune specific false positives rather than lowering the priority of the entire alert class. If multiple false positives are coming from the same root cause (e.g., all IT-deployed software triggers the same rule), that's a broader tuning problem to address during the post-incident review, not by deprioritizing the alert during recovery. Every real positive you miss during the recovery window risks a complete restart of the IR process.

Q: You're five days into the recovery monitoring window with no indicators. Business wants to end the monitoring window early and return to normal operations. Should you agree?

Seven days is a minimum, not an arbitrary number — it covers the maximum Kerberos ticket lifetime (10 hours) by a large margin, gives time for any delayed triggers (like a WMI subscription that triggers on a 7-day timer), and provides a defensible record. If business pressure is strong, compromise at 5 days only if: (1) the incident had short dwell time and limited scope, (2) all recovered systems have passed full eradication validation, and (3) the SOC commits to continuing the daily hunt queries and EDR watchlist monitoring even after the formal window closes. Don't let the end of the "monitoring window" be the end of incident-specific monitoring — maintain the IOC watchlist in the detection stack permanently (or until the IOCs age out).