Chapter 58

Registry Internals

Hive files, cell structure, value types, transaction logs, and offline parsing — understanding what the registry actually is on disk lets you recover data other tools miss.

Scenario

You're doing dead-box forensics on a seized laptop. You can't boot the machine (it might be encrypted after boot). You pull the drive, mount it read-only, and need to determine: what user accounts existed, what programs ran automatically at startup, and whether any malicious software modified the system. All of that data lives in the registry hive files — binary files on disk that Windows reads at boot and during operation. Knowing their location, structure, and how to parse them offline is what separates a forensicator who finds the answer from one who doesn't.

Registry Hive Files

The registry is stored in hive files — binary files that Windows maps into memory. Each hive corresponds to a registry subtree.


  Registry Hive Map
  ══════════════════════════════════════════════════════════════════

  Registry View                  On-Disk Location
  ─────────────                  ────────────────
  HKEY_LOCAL_MACHINE (HKLM)
    \SYSTEM           ──────────► C:\Windows\System32\config\SYSTEM
    \SAM              ──────────► C:\Windows\System32\config\SAM
    \SECURITY         ──────────► C:\Windows\System32\config\SECURITY
    \SOFTWARE         ──────────► C:\Windows\System32\config\SOFTWARE
    \HARDWARE         ──────────► (volatile — in-memory only, not on disk)
    \BCD00000000      ──────────► \Boot\BCD  (EFI boot config)

  HKEY_USERS (HKU)
    \.DEFAULT         ──────────► C:\Windows\System32\config\DEFAULT
    \S-1-5-18         ──────────► (SYSTEM user — from SYSTEM hive)
    \S-1-5-21-...-500 ──────────► C:\Users\[username]\NTUSER.DAT
    \S-1-5-21-...-500_Classes ─► C:\Users\[username]\AppData\Local\
                                 Microsoft\Windows\UsrClass.dat

  HKEY_CURRENT_USER (HKCU)  ──► (alias for current user's NTUSER.DAT)

  Additional amcache / shimcache:
    C:\Windows\AppCompat\Programs\Amcache.hve
    C:\Windows\System32\config\SYSTEM (shimcache in ControlSet\Control\SessionManager\AppCompatCache)

    
Common mistake

SAM and SECURITY cannot be copied while Windows is running via normal file copy — they are locked by the SYSTEM account. Use Volume Shadow Copies (VSS), registry export APIs, or forensic imaging tools. During live response: reg save HKLM\SAM sam.hiv requires SYSTEM privileges but works; alternatively use secretsdump.py via Impacket which reads via registry RPC.

Hive Binary Structure

Each hive file is a structured binary database. Understanding the structure matters when you need to use low-level tools or recover deleted keys.


  Registry Hive File Structure
  ══════════════════════════════════════════════════════════════════

  ┌───────────────────────────────────────────────────────────────┐
  │  Base Block (4096 bytes, also called "regf" header)           │
  │  Magic: "regf"                                                │
  │  Primary sequence number (incremented on every write start)   │
  │  Secondary sequence number (incremented on write complete)    │
  │  Timestamp (FILETIME — last write time)                       │
  │  Major/minor version (1.3 for modern, 1.5 for W8+)            │
  │  Root cell offset (offset into hive bins area)                │
  │  Hive length, cluster factor                                  │
  └───────────────────────────────────────────────────────────────┘
  ┌───────────────────────────────────────────────────────────────┐
  │  Hive Bins (variable number)                                  │
  │  ┌─────────────────────────────────────────────────────────┐  │
  │  │  Bin Header (32 bytes)                                  │  │
  │  │  Magic: "hbin"                                          │  │
  │  │  Offset from start of hive bins                        │  │
  │  │  Size (always multiple of 4096)                        │  │
  │  └─────────────────────────────────────────────────────────┘  │
  │  ┌─────────────────────────────────────────────────────────┐  │
  │  │  Cells (variable size, aligned to 8 bytes)              │  │
  │  │  Size (positive = free, negative = allocated)           │  │
  │  │  Signature: "nk"=key node, "vk"=value, "sk"=security   │  │
  │  │             "lf"/"lh"=index, "li"=index, "ri"=root idx │  │
  │  └─────────────────────────────────────────────────────────┘  │
  └───────────────────────────────────────────────────────────────┘
  ┌───────────────────────────────────────────────────────────────┐
  │  Transaction Log Files (SYSTEM.LOG1, SYSTEM.LOG2)             │
  │  Written before the hive — enables crash recovery             │
  │  If primary sequence ≠ secondary: crash during write,         │
  │  transaction log used to recover consistent state             │
  └───────────────────────────────────────────────────────────────┘

    

Key Cell (nk) Fields

Each registry key is stored as an nk (named key) cell:

Registry Value Types

Type IDNameDescriptionCommon Use
0x0001REG_SZNull-terminated UTF-16LE stringPaths, names, most common
0x0002REG_EXPAND_SZUTF-16LE string with %VARIABLE% expansionImagePath (%SystemRoot%\system32\svchost.exe)
0x0003REG_BINARYRaw binary dataEncoded config, certificates, credentials
0x0004REG_DWORD32-bit little-endian integerFlags, counts, boolean-like settings
0x0005REG_DWORD_BIG_ENDIAN32-bit big-endian integerRare; some network config
0x0006REG_LINKSymbolic link to another keyHKCU → HKU\SID mapping
0x0007REG_MULTI_SZMultiple null-terminated strings, double-null terminatedDriver groups, LSA packages list
0x000BREG_QWORD64-bit little-endian integerLarge counters, timestamps
0x0000REG_NONENo defined value typeDefault value placeholders

Transaction Logs and Dirty Hives

Windows uses a write-ahead logging scheme to prevent corruption. Each hive has up to two transaction log files (SYSTEM.LOG1, SYSTEM.LOG2). Before writing to the hive, changes are written to the log. After the write completes, the sequence numbers are updated.

During forensics, you may encounter a "dirty" hive — the primary sequence number differs from the secondary, meaning the system crashed mid-write. Forensic tools like Registry Explorer and python-registry automatically replay the transaction log to reconstruct a consistent state.

Why this matters

Malware that writes to the registry and immediately crashes the system (or is killed mid-write) may leave a transaction log entry that isn't committed to the main hive yet. Forensic tools that don't process transaction logs will miss this evidence. Always collect .LOG1 and .LOG2 files alongside hive files.

Offline Parsing with python-registry

Pythonparse_registry.py
from Registry import Registry

# Parse NTUSER.DAT offline (mounted drive or forensic image)
reg = Registry.Registry("C:/mount/Users/victim/NTUSER.DAT")

# Walk the Run key for persistence
def get_run_keys(reg):
    try:
        run_key = reg.open("Software\\Microsoft\\Windows\\CurrentVersion\\Run")
        print(f"[HKCU Run] Last written: {run_key.timestamp()}")
        for value in run_key.values():
            print(f"  {value.name()} = {value.value()}")
    except Registry.RegistryKeyNotFoundException:
        print("No HKCU Run key found")

get_run_keys(reg)

# Parse SOFTWARE hive for installed software
reg_soft = Registry.Registry("C:/mount/Windows/System32/config/SOFTWARE")
uninst = reg_soft.open(
    "Microsoft\\Windows\\CurrentVersion\\Uninstall"
)
for subkey in uninst.subkeys():
    try:
        name = subkey.value("DisplayName").value()
        install_date = subkey.value("InstallDate").value()
        print(f"{name} installed {install_date}")
    except:
        pass

# Find recently modified keys (useful for forensic timeline)
def find_modified_after(key, cutoff_dt, path=""):
    if key.timestamp() > cutoff_dt:
        print(f"Modified: {key.timestamp()} | {path}\\{key.name()}")
    for subkey in key.subkeys():
        find_modified_after(subkey, cutoff_dt, f"{path}\\{key.name()}")

Live Registry Monitoring APIs

For detection during live operation, several mechanisms can watch for registry changes:

MechanismGranularityDetection Use
RegNotifyChangeKeyValueKey-level; signals when anything under a key changesWatch specific persistence keys (Run, Services)
ETW Microsoft-Windows-Registry providerPer-value operation with process name and PIDHigh-fidelity; used by some EDRs and Sysmon Events 12–14
Sysmon Events 12/13/14Key create/delete (12), value set (13), key/value rename (14)Most practical for SOC; filter to high-value keys
Object Access Audit (4657)Requires SACL on specific keys; per-value access loggingHigh value but noisy; apply to critical keys only
WMI RegistryKeyChangeEventKey-level change notification via WMI subscriptionCan be used by monitoring tools; not real-time

VSS for Offline Registry Access

Volume Shadow Copies contain point-in-time snapshots of the registry hives, even the locked SAM and SECURITY hives. This is how tools like secretsdump work locally:

PowerShellaccess_vss_registry.ps1
# List available VSS snapshots
vssadmin list shadows

# Access SAM via shadow copy (copy without lock)
$shadow = "\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1"
Copy-Item "$shadow\Windows\System32\config\SAM" C:\temp\SAM
Copy-Item "$shadow\Windows\System32\config\SYSTEM" C:\temp\SYSTEM
Copy-Item "$shadow\Windows\System32\config\SECURITY" C:\temp\SECURITY

# Or via symlink trick (older technique):
# cmd /c mklink /d C:\shadow \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\
Key insight

The VSS technique for reading SAM works because shadow copies bypass the file locking that prevents direct copy of live hives. Impacket's secretsdump.py --local uses this exact method on a live system (with admin privileges). Detection: Sysmon Event 9 (RawAccessRead) targeting PhysicalDrive0 or the volume, or Event 11 showing SYSTEM file copies from shadow copy paths.

Key Forensic Artifacts by Hive

HiveKey Forensic Artifacts
SYSTEMServices (malicious drivers/services), timezone, computer name, USB device history (USBSTOR), network interfaces, shimcache (program execution history)
SAMLocal user accounts, password hashes (LM/NT), login counts, last login time, account lockout status
SECURITYLSA secrets (cached domain creds, service account passwords, autologon), audit policy
SOFTWAREInstalled programs (Uninstall key), run keys (HKLM\Run), startup folder, Windows version, installed patches, browser history (IE), network provider order
NTUSER.DATUser run keys (persistence), RecentDocs, UserAssist (program execution), Shellbags (folder access), typed URLs, search history, MRU lists, mapped drives
Amcache.hveProgram execution history — first execution time, file path, SHA1 hash

Q & A

Q: What is the difference between NTUSER.DAT and UsrClass.dat?

Both are per-user registry hives. NTUSER.DAT contains the main user settings (Run keys, UserAssist, RecentDocs, Shellbags for folder access, etc.) and is loaded as HKCU. UsrClass.dat contains COM/OLE class registrations and per-user shell extension settings for Windows Explorer shell items — it's loaded as the HKCU\Software\Classes subkey. For malware analysis, NTUSER.DAT is usually more relevant (persistence, user-specific config). UsrClass.dat is important for Shellbags analysis (folder access history) and COM hijacking (HKCU\Software\Classes\CLSID).

Q: How does Windows handle a crash during a registry write?

Windows uses a two-phase commit with the transaction log files (.LOG1/.LOG2). Before writing to the hive, Windows writes the dirty sectors to the LOG file and increments the primary sequence number. After writing to the hive completes, Windows increments the secondary sequence number to match. On boot, if primary ≠ secondary, Windows replays the LOG to restore the hive to a consistent state. From a forensic perspective, you should always collect the LOG files alongside the hive and use a parser that replays them (Registry Explorer does this automatically).

Q: Why is the Shimcache (AppCompatCache) stored in the SYSTEM hive, not per-user?

Shimcache is maintained by the Application Compatibility subsystem (shim infrastructure) which operates at the system level, not per-user. It records metadata about executables that have run (or in some Windows versions, just been accessed) system-wide — path, size, last modification time. Because it's system-wide and written by the kernel's compatibility layer, it lives in HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\AppCompatCache. This makes it extremely valuable forensically: even if a malware binary is deleted, its entry in Shimcache may persist, providing evidence of execution.