How CPUs Work

The Scenario

You are twelve hours into analyzing a ransomware sample. The decompiler shows two functions that scan memory: one walks a 4 KB lookup table 50,000 times, the other scans a 48 MB heap allocation once. Both perform the same number of memory reads per iteration. Your gut says the heap scan is faster — it loops far fewer times.

You attach a sandbox profiler. The 4 KB loop finishes in under 200 microseconds. The heap scan takes 600 milliseconds — 3,000 times slower. The disassembly shows equivalent instruction sequences. The CPU is executing the same operations. Nothing in Ghidra explains this.

This is the gap between the architectural model — what your disassembler shows — and the microarchitectural reality — what the silicon actually does. Closing that gap is what this chapter is about. You will also understand, by the end, exactly why the heap scan is 3,000 times slower.

The Architectural Contract

The x86-64 ISA makes a promise: instructions appear to execute sequentially, in the order they are listed, one at a time, each completing before the next begins. Every x86 program — from kernel code to your hello-world binary — can rely on this guarantee.

The silicon breaks every one of those rules internally, while carefully preserving the visible result. Modern CPUs execute multiple instructions simultaneously, in an order chosen by the hardware, before any of them have formally "completed" — and they commit the results in program order only at the final stage. The promise holds. The implementation ignores it entirely.

Before Pipelines: Sequential Execution

A minimal CPU executes instructions through five distinct stages, one after another:

In a purely sequential design, each instruction must complete all five stages before the next instruction can begin its Fetch stage. Four instructions would take 4 × 5 = 20 clock cycles. The execution unit sits idle during Fetch. The Fetch unit sits idle during Execute. Every stage is idle whenever another stage is active. That is a lot of wasted silicon.

The Pipeline: Instructions in Parallel

The solution is treating instruction execution like an assembly line. As soon as Instruction 1 moves from Fetch to Decode, Instruction 2 can begin its Fetch stage. Every stage has something to work on every cycle. Multiple instructions are "in flight" simultaneously, each at a different stage.

  CLOCK CYCLE:     1      2      3      4      5      6      7      8
                  ─────  ─────  ─────  ─────  ─────  ─────  ─────  ─────
  Instruction 1:  [IF]  [ID]   [EX]   [MA]   [WB]
  Instruction 2:         [IF]  [ID]   [EX]   [MA]   [WB]
  Instruction 3:                [IF]  [ID]   [EX]   [MA]   [WB]
  Instruction 4:                       [IF]  [ID]   [EX]   [MA]   [WB]
                  ─────  ─────  ─────  ─────  ─────  ─────  ─────  ─────
                                                                     ↑
                                                    All 4 done at cycle 8.
                                             Without pipelining: 4 × 5 = 20 cycles.

  IF  Instruction Fetch      MA  Memory Access
  ID  Instruction Decode     WB  Write Back
  EX  Execute

The pipeline reduces 20 cycles to 8. The speedup is not free — it adds complexity for managing state between stages — but it is fundamental to why modern CPUs are fast. Real microarchitectures go much further: Intel's Golden Cove has a 19-stage front-end pipeline; AMD Zen 4 is 25 stages deep on some paths. The 5-stage model is the educational foundation; the principle is identical at any depth.

Pipeline Hazards: When the Line Stalls

The pipeline assumes each stage always has something to do. Three classes of situations break that assumption, called pipeline hazards.

Data Hazard

A data hazard occurs when one instruction needs a value that a previous instruction has not yet produced. Consider this sequence:

mov rax, [rbx]      ; Stage 1: load from memory into rax
                    ;          (cache miss → might take 100+ cycles)
add rax, 1          ; Stage 2: needs rax, but it is not ready yet

The add is already in the Decode stage while the mov is still waiting for data from memory. The CPU cannot execute add until rax is valid. The pipeline inserts bubble cycles — essentially NOP slots — to stall the dependent instruction until the value arrives. On a cache miss, this stall can be 100–300 cycles long.

CPUs mitigate data hazards with operand forwarding (also called bypassing): routing the result directly from the execute unit's output to another unit's input, before it is written back to the register file. This eliminates stalls for back-to-back arithmetic but does not help when the data must come from memory.

Control Hazard

A control hazard occurs at every branch instruction. After fetching a jz .target, the CPU does not yet know whether to fetch the instruction after the jump (not taken) or the instruction at .target (taken). It will not know until the cmp/test result is computed in the Execute stage — potentially two or three cycles later. Waiting those cycles every time would make branch-heavy code dramatically slower. The solution is branch prediction.

Structural Hazard

A structural hazard occurs when two instructions in the pipeline simultaneously need the same hardware resource — for example, two load instructions competing for a single memory port. Modern CPUs with multiple execution units and multiple load/store ports are designed to minimize structural hazards; you will rarely encounter them at the instruction level, but they become relevant when analyzing throughput in vectorized code (Part 9).

Branch Prediction: The CPU's Best Guess

Rather than stalling the pipeline every time it sees a branch, the CPU guesses which direction the branch will go and speculatively fetches and begins executing instructions down the predicted path. If the prediction is correct, the speculative work is committed and the pipeline never stalled. If the prediction is wrong, the speculatively executed results are discarded, the pipeline is flushed, and execution restarts from the correct path — at a penalty of 15–20 clock cycles.

  cmp  rax, 0
  jz   .not_found          ← CPU must decide: taken or not taken?
  ...
  .not_found:

  ─────────────────────────────────────────────────────────────────
  PREDICTION CORRECT           │  PREDICTION WRONG
  ─────────────────────────    │  ─────────────────────────────────
  Speculative work committed.  │  Pipeline FLUSHED (15–20 cycles).
  Zero penalty. Continues at   │  Speculative results discarded.
  full speed.                  │  Restart from correct address.
  ─────────────────────────────────────────────────────────────────

  Modern predictor accuracy on typical code:   95–99%
  Worst case (random coin-flip branches):       ~50%  ← catastrophic

Modern branch predictors are sophisticated. They track the history of each branch — has this branch been taken the last five times? Does it alternate taken/not-taken in a pattern? They use multiple table levels, global history, and pattern-matching heuristics. A loop that runs 1000 times will be predicted "taken" for iterations 1–999 and correctly predicted "not taken" on the final exit. A linked-list traversal with random node sizes might be predicted correctly 60% of the time. A branch that checks a random boolean will be predicted correctly ~50% of the time — the CPU is doing no better than a coin flip.

The Cache Hierarchy: Speed Tiers for Memory

RAM stores everything. RAM is also far from the CPU — physically and electrically. A memory access that misses all caches and reaches main memory takes 60–100 nanoseconds. On a 3 GHz CPU, that is 180–300 clock cycles. An instruction that can execute in 1 cycle must sit idle for 300 cycles waiting for its operand. The pipeline stalls, the CPU is idle, and performance collapses.

The solution is a hierarchy of smaller, faster memory banks between the CPU and RAM.

  ┌──────────────────────────────────────────────────────────────┐
  │                        CPU CORE                              │
  │                                                              │
  │   ┌──────────────────────────────────────────────────────┐  │
  │   │         L1 Cache — 32–64 KB — ~4 cycles              │  │
  │   │   (split: 32 KB instruction + 32 KB data, per core)  │  │
  │   │                                                       │  │
  │   │   ┌───────────────────────────────────────────────┐  │  │
  │   │   │      L2 Cache — 256 KB–1 MB — ~12 cycles      │  │  │
  │   │   │                (per core, unified)             │  │  │
  │   │   └───────────────────────────────────────────────┘  │  │
  │   └──────────────────────────────────────────────────────┘  │
  └──────────────────────────────────────────────────────────────┘
                              │  shared across all cores
  ┌─────────────────────────────────────────────────────────────┐
  │           L3 Cache — 8–64 MB — ~40 cycles                   │
  │                   (last-level cache, LLC)                    │
  └─────────────────────────────────────────────────────────────┘
                              │
  ┌─────────────────────────────────────────────────────────────┐
  │           Main RAM — 8–128 GB — ~100–300 cycles             │
  └─────────────────────────────────────────────────────────────┘
                              │
  ┌─────────────────────────────────────────────────────────────┐
  │           NVMe SSD — TB range — ~50,000 cycles              │
  └─────────────────────────────────────────────────────────────┘

Cache Lines: Data Moves in 64-Byte Chunks

Memory does not move between cache levels one byte at a time. It moves in cache lines — fixed 64-byte blocks aligned to 64-byte boundaries. When the CPU reads a single byte, the entire 64-byte cache line containing that byte is loaded from the next cache level (or RAM).

This has a profound consequence for access patterns. If your code reads bytes at addresses 0, 1, 2, 3, … 63 in order, every read after the first is a cache hit — they all live in the same cache line already loaded by the first read. If your code jumps randomly through memory — reading address 0, then address 48000, then address 8, then address 32000 — every read is potentially a different cache line, potentially a cache miss, potentially 100–300 stall cycles.

Now you know why the two loops from the opening scenario behave so differently. The 4 KB loop accesses a table that fits entirely inside L1 cache (32 KB). After the first pass, every subsequent iteration reads data that is already in L1. Each read costs 4 cycles. The 48 MB heap scan accesses memory that is 4× larger than L3 cache. Every iteration reads a different region that has almost certainly been evicted since the last visit. Each read costs 200+ cycles. The instruction count is similar. The memory latency cost differs by 50×.

Out-of-Order Execution: The CPU Decides the Order

The pipeline handles temporal parallelism — multiple instructions at different stages simultaneously. Out-of-order execution (OOO) adds another dimension: modern CPUs have multiple execution units (ALUs, load units, store units, floating-point units), and they will dispatch instructions to whichever unit is idle — regardless of program order.

The CPU looks ahead in the instruction stream (typically 200–400 instructions ahead on Intel's Golden Cove), identifies which instructions have all their operands ready, and dispatches them to available execution units. A Reorder Buffer (ROB) tracks every in-flight instruction and ensures that results are committed in architectural program order — preserving the sequential contract — even if they executed in a completely different order.

add  rax, 1          ; → dispatched to integer ALU port 0
add  rbx, 2          ; → dispatched to integer ALU port 1  (same cycle)
add  rcx, 3          ; → dispatched to integer ALU port 5  (same cycle)
mov  rdx, [r8]       ; → dispatched to load unit           (same cycle)

These four instructions are completely independent — none reads a register written by another. A modern CPU with sufficient execution ports can dispatch all four in the same cycle. The disassembly lists them sequentially. The silicon executes them in parallel. The result looks identical to sequential execution from the program's perspective.

This is also why the compiler cares about instruction scheduling. When you compile with -O2, part of what GCC or Clang does is interleave independent instructions to keep all execution units busy — filling the cycles that would otherwise be stall bubbles. At -O0, the compiler emits instructions in the order you wrote them; at higher optimization levels, it is rearranging your code to exploit the CPU's OOO engine.

What Is Actually Happening Beneath the Listing

Put it together. Here is a four-instruction sequence that Ghidra shows as a simple null-pointer check, and what the CPU is actually doing with it:

  WHAT GHIDRA SHOWS                  WHAT THE CPU DOES
  ─────────────────────────────────  ─────────────────────────────────────────────────────
  cmp  rax, [rbx+0x10]          ──▶  Starts memory load from [rbx+0x10].
                                     L1 hit: 4 cycles. L3 miss: 40 cycles. RAM: 200 cycles.

  jz   .null_handler             ──▶  Branch predictor guesses "not taken" (likely correct
                                     for a fast-path check). Speculatively fetches next instr.

  mov  rcx, [rax+0x08]          ──▶  SPECULATIVELY EXECUTED before jz resolves.
                                     Cache line for [rax+0x08] may start loading now.

  lea  rdx, [rax+rcx*4]         ──▶  SPECULATIVELY COMPUTED in the ALU.
                                     Result sits in the ROB, not yet committed.
  ─────────────────────────────────────────────────────────────────────────────────────────
  When jz resolves as "not taken":   Speculative work committed. Zero penalty.
  When jz resolves as "taken":       Pipeline flushed. Cache line for [rax+0x08] remains
                                     cached — a Spectre-exploitable side channel.

The disassembly is not a script the CPU follows sequentially. It is a specification of the intended behavior. The silicon's job is to implement that behavior as fast as possible, and it will violate every sequential assumption to do so — while ensuring you never observe the difference through the architectural interface.

What You Now Know

Before moving to Chapter 2, you should be able to answer these questions from memory — if not, re-read the relevant section:

• Pipeline: Why is 4 instructions taking 8 cycles better than 20 cycles? What allows that speedup?
• Data hazard: If instruction B reads a register that instruction A writes, and A is loading from memory (cache miss), what happens inside the pipeline?
• Branch prediction: What is the cost of a misprediction? Why does a loop with random-direction branches run slower than one with predictable branches, even if the instruction count is identical?
• Cache: Why did the 4 KB loop outperform the 48 MB scan by 3,000×, given the same instruction mix?
• Cache lines: You read byte 0 of a 200-byte struct. Are bytes 1–63 now in cache? What about byte 64?
• OOO execution: Can two consecutive add instructions — operating on different registers — execute in the same clock cycle? Under what condition would they be forced to execute sequentially?
• Spectre: Why does a mispredicted branch that was rolled back still leave a side-channel trace?

In Chapter 2, we step back and look at how this architecture was built — starting from the 8086 in 1978 and the legacy decisions that still appear in every modern binary you will analyze.