Chapter 25

NT API vs Win32

The layered Windows API architecture — Win32 (kernel32/kernelbase), Native API (ntdll), and the kernel boundary — and why the layer a function is called at determines whether EDR hooks fire

Scenario

An analyst hunting in an EDR dashboard finds that a suspicious process performed memory allocation (VirtualAlloc telemetry), but the subsequent WriteProcessMemory hook never fired. The process called NtWriteVirtualMemory directly via ntdll — the layer below where the EDR hooked WriteProcessMemory in kernel32. Every layer skipped is a telemetry gap. Mapping the layers tells you where the gap is.

The Windows API Layers

 ┌─────────────────────────────────────────────────────┐
 │            User Application                          │
 │  WriteFile(), CreateProcess(), VirtualAlloc()        │
 └──────────┬──────────────────────────────────────────┘
            │ calls
 ┌──────────▼──────────────────────────────────────────┐
 │  Win32 Layer — kernel32.dll / kernelbase.dll         │
 │  CreateFileW()       → NtCreateFile()                │
 │  WriteProcessMemory() → NtWriteVirtualMemory()       │
 │  CreateProcessW()    → NtCreateUserProcess()         │
 │  (parameter validation, string conversion, etc.)     │
 └──────────┬──────────────────────────────────────────┘
            │ calls (thin wrappers)
 ┌──────────▼──────────────────────────────────────────┐
 │  Native API — ntdll.dll                              │
 │  NtCreateFile()       stub → syscall #55             │
 │  NtWriteVirtualMemory() stub → syscall #3A           │
 │  NtCreateUserProcess()  stub → syscall #C0           │
 │  (no logic, just: mov eax,; syscall; ret)    │
 └──────────┬──────────────────────────────────────────┘
            │ syscall instruction (CPL 3 → CPL 0)
 ┌──────────▼──────────────────────────────────────────┐
 │  Windows Kernel (ntoskrnl.exe) — Ring 0              │
 │  KiSystemCall64 → SSDT dispatch → actual kernel fn  │
 │  NtCreateFile: IRP → filesystem driver               │
 └─────────────────────────────────────────────────────┘

ntdll.dll — The Lowest User-Mode Layer

ntdll.dll is the only DLL that talks directly to the kernel. Its "Nt" and "Zw" function stubs are identical in user mode — each loads the syscall number into EAX and executes the syscall instruction (x64) or sysenter (x86):

; NtAllocateVirtualMemory stub in ntdll.dll (x64, simplified)
NtAllocateVirtualMemory:
    mov    r10, rcx          ; save RCX (syscall clobbers it)
    mov    eax, 0x18         ; syscall number (varies by Windows version!)
    test   byte ptr [SharedUserData+0x308], 1
    jne    .int2e_path        ; legacy int 2e path (very old Windows)
    syscall
    ret
.int2e_path:
    int    0x2e
    ret

The syscall number (SSN) is not stable — it changes between Windows versions and even between minor updates. Malware that hardcodes syscall numbers breaks silently on unintended OS versions. Tools like Halo's Gate and Tartarus' Gate (Chapter 26) address this by finding the correct SSN dynamically from ntdll.

KernelBase vs kernel32

In Windows Vista and earlier, kernel32.dll contained all Win32 API implementations. Starting with Windows 7, a large portion of the kernel32 implementation was moved to kernelbase.dll. Most kernel32 functions now simply forward to kernelbase.dll. This was done to allow the "MinWin" core of Windows to ship without the higher-level kernel32 semantics.

DLL	Role	Example functions
kernel32.dll	Win32 API facade; most functions forward to kernelbase.dll; historical entry point	CreateFileW → kernelbase!CreateFileW; VirtualAlloc → kernelbase!VirtualAlloc
kernelbase.dll	Actual Win32 implementation; calls ntdll stubs	CreateFileW, VirtualAlloc, ReadProcessMemory, OpenProcess
ntdll.dll	Thin syscall stubs; also: loader, heap, exception, LDR	NtCreateFile, NtAllocateVirtualMemory, LdrLoadDll, RtlAllocateHeap
advapi32.dll	Security APIs; forwards to sechost.dll and kernelbase.dll	OpenProcessToken, RegOpenKeyExW
sechost.dll	Security host services (tokens, registry, services)	OpenSCManagerW, OpenProcessToken

Call Chain: WriteProcessMemory

Tracing a single API call from Win32 to the kernel illustrates the layering:

; WriteProcessMemory call chain (simplified)

; Your code:
WriteProcessMemory(hProcess, lpBase, lpBuffer, nSize, &written);

; Lands in kernel32!WriteProcessMemory:
;   → just a JMP to kernelbase!WriteProcessMemory

; kernelbase!WriteProcessMemory:
;   Validates parameters
;   Calls NtWriteVirtualMemory(hProcess, lpBase, lpBuffer, nSize, &written)

; ntdll!NtWriteVirtualMemory stub:
mov r10, rcx
mov eax, 0x3A      ; SSN for NtWriteVirtualMemory on Win10 21H2
syscall
ret

; Kernel: KiSystemCall64 → NtWriteVirtualMemory kernel implementation
;   MmCopyVirtualMemory + access checks

Evasion via Layer Bypass

Bypass level	Technique	What's bypassed
Skip kernel32 → kernelbase	Call kernelbase!VirtualAlloc directly, not kernel32!VirtualAlloc	EDR hooks placed in kernel32
Skip Win32 entirely	Resolve NtAllocateVirtualMemory from ntdll; call it directly	EDR hooks on kernel32 and kernelbase functions
Skip ntdll stub (direct syscall)	Extract SSN from ntdll, execute syscall instruction in own code	EDR hooks on ntdll stubs (inline patches at ntdll stub prologue)
Skip ntdll stub (indirect syscall)	Use syscall; ret gadget from within ntdll to avoid hook detection	EDR stacks-walk analysis that checks syscall origin

Detection Impact

Understanding the API layers is essential for detection engineering because the layer where a hook is placed determines what it can observe:

ETW kernel providers — fire in the kernel regardless of which user-mode layer was skipped. ETW can observe system call execution even if all ntdll stubs are bypassed by direct syscalls, because ETW instruments the kernel implementation, not the user-mode stub.
ntdll inline hooks — if an EDR patches the ntdll stub prologue, direct syscalls from shellcode bypass these hooks. But the kernel still executes — ETW and kernel callbacks still fire.
Stack unwinding analysis — an ETW event fired from NtAllocateVirtualMemory has a call stack. If the stack walk finds the syscall instruction executed within ntdll, that's normal. If it finds syscall executed from a MEM_PRIVATE region or from shellcode, that's anomalous. This is how modern EDRs detect direct syscall usage.

Q & A

If most kernel32.dll functions just forward to kernelbase.dll, why does malware still call kernel32 instead of kernelbase directly?

Portability and compatibility. Malware targeting a range of Windows versions from XP through Windows 11 uses kernel32.dll because: (1) kernel32.dll has been present and stable across all Windows NT versions since NT 3.1. kernelbase.dll was introduced in Windows 7 — code calling kernelbase directly breaks on XP/Vista. (2) The documented Win32 API surface is kernel32, not kernelbase. Malware written with standard Windows headers links against kernel32. (3) Many injection techniques (CreateRemoteThread, VirtualAllocEx, WriteProcessMemory) are documented in kernel32 and that's what tutorials, POC code, and toolkits use. (4) Even kernel32→kernelbase forwarding means the function is still accessible via kernel32 with no overhead. There's no advantage to calling kernelbase directly unless specifically trying to bypass a kernel32-layer hook. So the typical progression is: (a) commodity malware calls kernel32 (documented, stable); (b) advanced malware resolves ntdll exports directly (bypasses kernel32/kernelbase hooks); (c) sophisticated malware uses direct syscalls (bypasses ntdll hooks).

How does Windows itself know which syscall numbers to use, since they change between OS versions?

Windows itself doesn't need to know — syscall numbers are baked into ntdll.dll at build time. The ntdll.dll that ships with each Windows version contains stubs that have the correct syscall numbers for that exact kernel version. The pairing is enforced: you can't run Windows 11's ntdll.dll on a Windows 10 kernel because the kernel version check (KUSER_SHARED_DATA.NtSystemRoot and compatible-version fields in the PE) prevents it. The loader also verifies that ntdll.dll's version is compatible. The kernel's system call dispatcher (KiServiceTable in the SSDT) is a fixed-size array indexed by syscall number. The ntdll stub for each function has the matching array index for that specific kernel build hardcoded into the mov eax, N instruction. When Microsoft ships a Windows update that changes the SSDT layout (adds or removes syscall slots), they also ship a new ntdll.dll with updated SSNs. Security tools that hard-code SSNs (for direct syscalls) break when the OS updates. The techniques covered in Chapter 26 (Hell's Gate, Halo's Gate, Tartarus' Gate) solve this by dynamically finding the current SSN from the live ntdll on disk or in memory.