Chapter 1

Windows Architecture

User mode vs kernel mode, CPU protection rings, the Hardware Abstraction Layer, the Windows Executive, and why this boundary is the most important line in Windows security

Scenario

A piece of malware runs on a user's machine and attempts to hide its process from Task Manager. To do this, it patches a kernel data structure. But it's running in user mode. How does it cross the boundary? What stops it? What happens when the protection fails? Understanding Windows architecture answers all of these questions — and tells you where every attack and every defense lives in the stack.

CPU Protection Rings

The x86 and x64 CPUs enforce privilege levels through a hardware mechanism called protection rings. The CPU always knows what ring the currently executing code is in, and enforces restrictions accordingly.


  CPU Protection Rings (x86/x64)
  ─────────────────────────────────────────────────────────────────
                         ┌───────────────────────────────┐
                         │          Ring 0               │  ← Kernel mode
                         │   OS kernel, drivers,         │    Full hardware access
                         │   Windows Executive           │    Can execute privileged instructions
                         │                               │    Unrestricted memory access
                         └───────────────────────────────┘
                    ┌────┴───────────────────────────┴────┐
                    │             Ring 1                   │  ← Not used by Windows
                    │         (Reserved/Unused)            │    Historically for OS services
                    └────┬───────────────────────────┬────┘
                    ┌────┴───────────────────────────┴────┐
                    │             Ring 2                   │  ← Not used by Windows
                    │         (Reserved/Unused)            │    Historically for device drivers
                    └────┬───────────────────────────┬────┘
                         ┌───────────────────────────────┐
                         │          Ring 3               │  ← User mode
                         │   Applications, DLLs,         │    Restricted hardware access
                         │   Win32 API calls             │    Cannot execute privileged instructions
                         │                               │    Cannot directly access kernel memory
                         └───────────────────────────────┘

  Windows uses only Ring 0 (kernel) and Ring 3 (user).
  Rings 1 and 2 exist in the CPU spec but Windows ignores them.

The current privilege level (CPL) is stored in bits 0-1 of the CS (code segment) register. CPL=0 means Ring 0; CPL=3 means Ring 3. The CPU checks this before every privileged operation and raises a General Protection Fault (#GP exception) if Ring 3 code tries to execute a privileged instruction like IN, OUT, LGDT, or LIDT.

User Mode vs Kernel Mode

Windows collapses the four-ring model into two modes. The practical difference between them is enormous:

Property	User Mode (Ring 3)	Kernel Mode (Ring 0)
Who runs here	Applications (notepad.exe, chrome.exe, malware.exe)	Windows kernel, HAL, device drivers, antivirus kernelmode components
Memory access	Only own virtual address space. Cannot access kernel memory directly.	All physical memory. All virtual addresses. Every process's memory.
Crash impact	Process crashes. Other processes unaffected. No BSOD.	System crash (BSOD). All processes die. Machine reboots.
Privileged instructions	Cannot execute. CPU faults if attempted.	Unrestricted. Can reprogram hardware, halt the CPU, change page tables.
I/O access	No direct port I/O. Must go through kernel via system calls.	Direct port access via IN/OUT instructions.
System calls	Entry point into kernel mode for services	Processes system call requests from user mode

The Full Windows Architecture Stack


  ┌─────────────────────────────────────────────────────────────────┐
  │                    USER MODE (Ring 3)                           │
  │                                                                  │
  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │
  │  │ User Process │  │ User Process │  │  Windows Subsystem   │  │
  │  │ (notepad.exe)│  │ (malware.exe)│  │  Process (csrss.exe) │  │
  │  └──────┬───────┘  └──────┬───────┘  └──────────────────────┘  │
  │         │                 │                                      │
  │  ┌──────▼─────────────────▼─────────────────────────────────┐  │
  │  │        Subsystem DLLs (kernel32.dll, user32.dll, gdi32)  │  │
  │  └──────────────────────────┬───────────────────────────────┘  │
  │  ┌───────────────────────────▼──────────────────────────────┐  │
  │  │            ntdll.dll  (NT Layer DLL)                     │  │
  │  │  • Lowest user-mode library                              │  │
  │  │  • Contains stubs for every NT system call               │  │
  │  │  • SYSCALL / INT 2E instruction here                     │  │
  │  └──────────────────────────┬───────────────────────────────┘  │
  └─────────────────────────────┼────────────────────────────────── ┘
  ═════════════════════════ USER/KERNEL BOUNDARY ═══════════════════
  ┌─────────────────────────────┼────────────────────────────────── ┐
  │  ┌──────────────────────────▼──────────────────────────────┐   │
  │  │            System Service Dispatcher                     │   │
  │  │  (KiSystemCall64 — routes syscall to correct handler)   │   │
  │  └──────────────────────────┬────────────────────────────── ┘  │
  │  ┌───────────────────────────────────────────────────────────┐  │
  │  │                     Windows Executive                     │  │
  │  │  I/O Mgr | Memory Mgr | Process Mgr | Object Mgr |       │  │
  │  │  Security | Cache Mgr | Config Mgr  | PnP Mgr    |       │  │
  │  └──────────────────────────┬────────────────────────────── ┘  │
  │  ┌───────────────────────────▼──────────────────────────────┐  │
  │  │                  Windows Kernel                           │  │
  │  │  Thread scheduling | Interrupts | Synchronization        │  │
  │  │  Exception handling | DPCs | APCs                        │  │
  │  └──────────────────────────┬────────────────────────────── ┘  │
  │  ┌───────────────────────────▼──────────────────────────────┐  │
  │  │                    HAL (ntoskrnl.exe)                     │  │
  │  │  Hardware Abstraction Layer — talks to physical hardware  │  │
  │  └──────────────────────────┬────────────────────────────── ┘  │
  │                   KERNEL MODE (Ring 0)                         │
  └─────────────────────────────┼─────────────────────────────────┘
                                 │
                   ┌─────────────▼─────────────┐
                   │  Physical Hardware         │
                   │  CPU | RAM | Disk | NIC    │
                   └───────────────────────────┘

The Hardware Abstraction Layer (HAL)

The HAL sits at the bottom of the kernel-mode stack and provides a uniform interface to hardware that varies across PC configurations. Different motherboards handle interrupt controllers (APIC vs legacy PIC), timers, and bus types differently. The HAL abstracts this variation so that the rest of the kernel doesn't need to know what hardware it's running on.

On modern Windows x64, the HAL is compiled directly into ntoskrnl.exe (the NT kernel image). On older systems it was a separate hal.dll. From a security perspective, the HAL is mostly a clean abstraction layer — malware at Ring 0 can call HAL APIs directly to manipulate hardware in ways that bypass OS security, but rootkits more commonly target the layers above the HAL.

The Windows Executive

The Executive is the main body of the Windows kernel — the code that implements the OS services that user-mode processes call into. It's implemented in ntoskrnl.exe. The Executive is divided into named components, each responsible for a specific subsystem.

Component	Responsibility	Security Relevance
Object Manager	Manages all kernel objects (files, processes, threads, events, mutexes)	Every resource is an object with a security descriptor; access control happens here
Process Manager	Creates, terminates, and tracks processes and threads	Process creation callbacks; where EDR hooks to detect new processes
Memory Manager	Virtual memory, page file, VAD tree, memory protection	VirtualAllocEx, WriteProcessMemory, DEP enforcement
I/O Manager	Device I/O, driver loading, IRP routing	Minifilter drivers intercept file I/O here for AV scanning
Security Reference Monitor (SRM)	Access checks, audit logging, privilege enforcement	The enforcement engine for ACLs, tokens, and privileges
Configuration Manager	Windows Registry implementation	Registry callback notifications; persistence detection
Cache Manager	File system cache	Less directly relevant to security but involved in file-level attacks
PnP Manager	Plug-and-play device discovery and driver loading	Driver signing enforcement; BadUSB attack path

The Windows Kernel

Below the Executive sits the Kernel (also called the microkernel), which handles the lowest-level OS functions. The distinction between "Executive" and "Kernel" can be confusing because both live in ntoskrnl.exe. The practical split:

Kernel: Thread scheduling and context switching, interrupt and exception dispatching, DPCs (Deferred Procedure Calls), APCs (Asynchronous Procedure Calls), synchronization primitives (spinlocks, mutexes at kernel level)
Executive: All the named subsystems above — the higher-level OS services built on top of the Kernel's primitives

The Windows Kernel is not a microkernel in the strict OS-theory sense — device drivers run in Ring 0 alongside it, not in isolated user-mode processes. This architectural choice prioritizes performance but means a buggy or malicious driver can crash or compromise the entire OS.

Mode Transitions — Crossing the Boundary

User mode code cannot directly call kernel functions. It must cross the boundary through a controlled gate. On x64 Windows, this happens via the SYSCALL instruction.


  System Call Flow (x64 Windows)
  ──────────────────────────────────────────────────────────────────
  User Mode:

  1. Application calls CreateFile() in kernel32.dll
     ↓
  2. kernel32.dll calls NtCreateFile() in ntdll.dll
     ↓
  3. ntdll.dll NtCreateFile stub:
       mov r10, rcx           ; save first argument
       mov eax, 55h           ; syscall number (SSN) for NtCreateFile
       syscall                ; ← CROSSES THE BOUNDARY

  ══════════════════════════════════════════════════════════════════

  Kernel Mode:

  4. CPU saves user-mode RIP, RSP, RFLAGS to kernel stack
     Loads kernel-mode RSP from KPCR (IA32_LSTAR MSR)
     Sets CPL to 0
     ↓
  5. KiSystemCall64 (system call dispatcher):
       • Reads syscall number from EAX (55h = NtCreateFile)
       • Looks up handler in SSDT (System Service Descriptor Table)
       • Calls NtCreateFile() in the executive
     ↓
  6. NtCreateFile runs in kernel mode
     Returns NTSTATUS
     ↓
  7. SYSRET instruction: restores user-mode context, sets CPL=3
  8. ntdll stub returns to kernel32, which returns to application

The SSDT

The System Service Descriptor Table is an array of kernel function pointers indexed by syscall number. When KiSystemCall64 receives syscall number 0x55, it looks up index 0x55 in the SSDT and calls the function pointer there. Rootkits historically patched SSDT entries to redirect system calls to their own code — this was the dominant kernel hooking technique on 32-bit Windows. On 64-bit Windows, PatchGuard (Kernel Patch Protection) periodically checks the SSDT and BSODs the system if modifications are detected.

Architecture's Relevance to Malware

The user/kernel boundary shapes the entire attack and defense landscape:

Attack / Defense	Where It Lives	Why
Injecting DLL into another process	User mode (Ring 3)	CreateRemoteThread, WriteProcessMemory are user-mode Win32 API calls
LSASS credential dumping (Mimikatz)	User mode	OpenProcess + ReadProcessMemory are user-mode APIs; enough for lsass.exe access
Hiding processes (DKOM rootkit)	Kernel mode (Ring 0)	Process list is in EPROCESS structures — only kernel code can modify it
Bypassing EDR hooks	User mode	Direct syscalls skip hooked ntdll stubs; stays in Ring 3 the whole time
EDR process monitoring	Kernel mode	PsSetCreateProcessNotifyRoutine is a kernel callback API — must be a driver
BSOD-based defense evasion	Kernel mode	Triggering a kernel crash clears evidence from volatile memory

Key Insight

Most malware — including sophisticated APT tools — operates entirely in user mode. User mode is enough to steal credentials, inject into processes, persist across reboots, and exfiltrate data. You don't need Ring 0 for most attacks. Kernel-mode malware (rootkits) exists but is rarer because (a) driver signing requirements on 64-bit Windows make it harder and (b) kernel bugs cause BSODs that attract attention.

Q & A

If user mode can't access kernel memory, how does malware like Mimikatz read LSASS memory?

Mimikatz doesn't directly access kernel memory — it uses a perfectly legal user-mode API sequence: OpenProcess(PROCESS_VM_READ, lsass_pid) followed by ReadProcessMemory(). These are Ring 3 Win32 API calls that go through the kernel via system calls. The kernel validates that the calling process has the required access rights (specifically, PROCESS_VM_READ), and if the caller has SeDebugPrivilege or is running as SYSTEM/Administrator, the kernel grants the access. The kernel then does the actual memory copy for you and returns the data to user mode. So Mimikatz doesn't break the user/kernel boundary — it works entirely within it, using kernel services to read another process's memory through the sanctioned API. This is why EDRs focus on detecting the behavior (a non-system process opening LSASS with VM_READ access) rather than the mechanism, since the mechanism is a normal system call.

What is ntdll.dll and why does it matter so much in security research?

ntdll.dll is the lowest-level user-mode DLL on Windows. Every other DLL and executable ultimately depends on it. Its two main jobs: (1) it contains the user-mode stubs for every Windows system call — NtCreateFile, NtCreateProcess, NtAllocateVirtualMemory, etc. These stubs contain the SYSCALL instruction that crosses into kernel mode. (2) it contains the Windows loader (LdrLoadDll, LdrInitializeThunk) and the heap manager. It matters in security for two reasons: First, EDR products hook ntdll.dll functions in user mode — by replacing the first bytes of functions like NtCreateProcess with a jump to their monitoring code, they intercept every process creation. Second, malware evades these hooks by doing "direct syscalls" — instead of calling through ntdll's hooked stub, the malware contains its own copy of the mov eax, SSN / syscall sequence and calls the kernel directly, bypassing ntdll entirely. This is a fundamental cat-and-mouse: EDR hooks ntdll; sophisticated malware bypasses ntdll.

Does Windows really only use Ring 0 and Ring 3? What about hypervisors?

Modern Windows actually has a fourth privilege level below Ring 0: Ring -1, or the hypervisor level (VMX root mode). Hyper-V (Windows' built-in hypervisor) runs at this level, below the operating system itself. When Hyper-V is active (which it is whenever Credential Guard, Device Guard, or Windows Sandbox is enabled), the Windows kernel runs as a "guest" in a virtual machine managed by Hyper-V. From the kernel's perspective, it's still Ring 0, but Hyper-V is actually more privileged. This architecture is called Virtualization Based Security (VBS). It's used to create an isolated, highly-protected environment (the Secure World / Virtual Trust Level 1) where LSASS credentials can be stored beyond the reach of even a compromised kernel. So the full modern picture is: Ring -1 (Hyper-V VTL0/VTL1) → Ring 0 (Windows kernel) → Ring 3 (user applications). Rootkits that patch kernel memory can no longer reach into the Secure World even if they fully control Ring 0.

What happens exactly when a user-mode process tries to access kernel memory?

The hardware enforces the boundary through the page table. Each virtual memory page has protection bits set in its Page Table Entry (PTE). Kernel pages are marked as Supervisor-only (U/S bit = 0). When a Ring 3 process tries to access a virtual address that maps to a Supervisor-only page, the CPU raises a Page Fault exception (#PF, interrupt vector 14). The CPU delivers this to the kernel's page fault handler. Since the fault came from a Ring 3 process accessing a supervisor page, the handler determines this is an access violation — not a normal page fault that can be resolved by swapping in a page. The result: the kernel raises a STATUS_ACCESS_VIOLATION exception in the user-mode process, which typically results in the process being terminated with an "Access Violation" error. The kernel itself is unaffected. This is why a crashing user-mode process doesn't take down the system — the kernel catches the hardware exception, terminates the offending process, and continues normally. Contrast this with a kernel-mode bug: if Ring 0 code dereferences an invalid pointer, there's no outer safety net — the CPU faults in Ring 0, the exception dispatcher panics, and the system crashes (BSOD).

Where does csrss.exe fit in this picture?

csrss.exe (Client/Server Runtime Subsystem) is the user-mode component of the Windows subsystem. It runs in user mode (Ring 3) but has special privileges — it cannot be terminated by normal means, and it manages critical functions like console window management and process/thread creation bookkeeping for Win32 applications. Historically (Windows NT through XP), csrss.exe handled more of the Win32 subsystem, including all GUI operations. On modern Windows, much of what csrss.exe used to handle has moved into the kernel itself (win32k.sys). Today csrss.exe manages the console infrastructure and some process lifecycle events. From a security perspective: csrss.exe termination crashes the system (you'll get a BSOD or immediate reboot), which makes it a target for anti-forensic techniques. It also participates in process creation events, which is why EDR tools monitor for attempts to inject into or manipulate csrss.exe.