Chapter 59

Anti-Disassembly Techniques

Disassemblers (IDA, Ghidra, Binary Ninja) analyze your binary by decoding instruction bytes sequentially or recursively. They assume the bytes between identified code are also instructions — which is generally true in clean binaries. Anti-disassembly techniques exploit the gap between what the CPU actually executes and what a linear or recursive-descent disassembler thinks it sees. The result: the disassembler decodes garbage, misidentifies data as code, splits functions incorrectly, or shows incorrect instruction lengths — all of which force an analyst to use a debugger instead of a disassembler, dramatically slowing analysis.

Anti-Disassembly Technique Gallery

How disassemblers work and where anti-disassembly exploits them
  Linear sweep (objdump --disassemble):
  ─────────────────────────────────────────────────────────────────────────
  Start at the first byte of the code section.
  Decode each instruction. Advance by instruction.length.
  Repeat until end of section.
  
  Problem: if a jump goes over a data byte embedded in the code section,
  the linear sweep decodes that data byte as an instruction — wrong output.
  
  Recursive descent (IDA Pro, Ghidra):
  ─────────────────────────────────────────────────────────────────────────
  Start at known entry points (EP, exported functions).
  Decode instruction. If it's a branch, add both targets to the work queue.
  If it's a call, add the call target. If it's a ret, stop this branch.
  
  Problem: if a conditional branch target can't be determined statically,
  the recursion doesn't follow. If a branch is obfuscated (JMP [EAX]),
  the disassembler doesn't know where EAX points → that code is never decoded.
  
  Anti-disassembly exploits one or both of these assumptions.
; anti_disasm.asm — Anti-disassembly techniques in x64 assembly
; NASM syntax

section .text
global anti_disasm_demo

; ── Technique 1: Junk byte after unconditional jump ──────────────────
; The CPU executes: JMP → skip the garbage byte → land at real code.
; Linear sweep: executes garbage byte as part of the next instruction.
; Recursive descent (IDA): follows the JMP to the correct target,
;   but the byte sequence between JMP and target is still SHOWN as code.
;   Some disassemblers handle this; older IDA versions get confused.
anti_disasm_demo:
    jmp .real_instruction
    db  0xE8          ; garbage byte (0xE8 = CALL opcode → confuses decoder)
.real_instruction:
    xor eax, eax
    ret

; ── Technique 2: Fake conditional jump with always-true predicate ────
; The conditional branch ALWAYS takes the same path, but the disassembler
; must analyze BOTH paths. The not-taken path starts with confusing bytes.
trick_2:
    xor  ecx, ecx       ; ecx = 0
    test ecx, ecx       ; ZF = 1 (always, since ecx is always 0 here)
    jz   .always_taken  ; always taken (ZF=1 → jump)
    ; bytes here are never executed but disassembler decodes them as code:
    db  0xFF, 0xE4      ; JMP RSP — confuses some disassemblers
    db  0x90            ; NOP (but the previous instruction ate 2+ bytes wrong)
.always_taken:
    mov eax, 1
    ret

; ── Technique 3: Self-referencing pointer trick ───────────────────────
; Embed a pointer to a nearby address. Disassembler decodes the pointer
; as instructions — 8 bytes (a 64-bit address) looks like 2-3 instructions.
; CPU never executes those bytes because a JMP skips over them.
trick_3:
    jmp  .after_pointer
    dq   trick_3       ; 8-byte pointer (= 8 bytes of "code" the disassembler sees)
.after_pointer:
    nop
    ret

; ── Technique 4: Overlapping instructions ─────────────────────────────
; A single byte can be the END of one instruction AND the START of another.
; The CPU uses one interpretation; the disassembler uses another.
;
; Example:
;   EB 02        JMP +2   (short jump, skips 2 bytes → lands at +4)
;   E8 F8        (garbage — the E8 starts a CALL decode that's wrong)
;   (landing at +4): XOR eax, eax → real instruction
;
; The F8 byte at offset 3 is decoded by a linear disassembler as the
; start of an INSTRUCTION. The CPU skips to offset 4 via the JMP.
; The "instruction" at offset 2 (E8 F8 90 xx) is fake.
trick_4:
    db  0xEB, 0x02  ; JMP +2 (lands at trick_4+4)
    db  0xE8, 0xF8  ; garbage: will decode as CALL 0xFFFFFFFB (fake call)
    ; CPU lands here (offset +4 from start of trick_4):
    xor eax, eax
    inc eax
    ret

Return Address Manipulation

; ── Technique 5: Return address manipulation ─────────────────────────
; Pop the return address from the stack, modify it, push back, and RET.
; Disassembler sees RET → thinks function ends here.
; CPU pops the modified address → continues executing at a different location.
; 
; Commonly used to hide where execution goes after a function returns:
;   CALL fake_ret_func  ; disassembler sees this as a normal call
;   [execution after ret → actually lands at modified_target, not here]

fake_ret_func:
    pop  rax            ; pop return address (would be next_instruction below)
    add  rax, 0x10      ; advance by 16 bytes (skip past data/confusing bytes)
    push rax            ; push modified return address
    ret                 ; "return" to (call_site + 16), not (call_site + 5)

; ── Technique 6: Indirect JMP via computed address ────────────────────
; IDA/Ghidra cannot determine where JMP RAX goes statically.
; They mark the target as "unknown" and stop recursive descent there.
; All code reachable only from this jump is invisible to static analysis.

indirect_jmp_demo:
    lea  rax, [rel .real_target]   ; load address of real target
    xor  rax, 0x1234               ; XOR with constant
    xor  rax, 0x1234               ; XOR again → cancels out (rax = real_target)
    jmp  rax                        ; IDA sees: JMP RAX (unknown target)
.real_target:
    mov  eax, 42
    ret

Questions & Answers

Modern IDA Pro and Ghidra handle many of these tricks. Which techniques still work against current tools?

Overlapping instructions (Technique 4) still reliably confuses Ghidra's auto-analysis — it creates "undefined" bytes in the listing view, and the decompiler often fails to produce output for the affected function. Indirect JMP via computed address (Technique 6) is effective against any tool: if the address is computed via arithmetic, the tool falls back to showing "jmp rax (unresolved)." Return address manipulation (Technique 5) defeats both linear and recursive analysis because the tool sees a RET and terminates the function, not knowing the return address was modified on the stack. The weakest techniques are simple junk-byte-after-JMP tricks (IDA's opcode fixup database handles E8 and other common confusers correctly) and obvious opaque predicates (IDA's decompiler often simplifies if (x | ~x) to true). For maximum effect: use overlapping instructions AND indirect jumps via computed values — these two together break both linear and recursive analysis.

What's the performance impact of anti-disassembly techniques on the running binary?

Minimal for most techniques. Junk bytes after unconditional jumps: the junk bytes are never executed, zero performance cost. Fake conditional branches with always-true predicates: one extra TEST + JZ instruction per use — negligible. Indirect JMP via computed address: a few XOR/LEA instructions plus one JMP instead of a direct JMP — maybe 5-10 cycles extra, negligible. The expensive technique is self-modifying code: the CPU needs to flush the instruction cache (CLFLUSH or branch predictor invalidation) after modifying code, which can cost hundreds of cycles and causes pipeline stalls. For the other techniques: anti-disassembly is essentially free at runtime because the tricks are purely about byte layout, not about adding computational work. The CPU's execution engine handles jumps and computed addresses at full speed — only the disassembler struggles with them.

How do these anti-disassembly tricks interact with control flow integrity (CFI)?

Control Flow Integrity (CFI) — specifically Microsoft's Control Flow Guard (CFG) and Clang's CFI — validates indirect call and jump targets at runtime. CFG maintains a bitmap of valid call targets; any indirect CALL that doesn't hit a valid target causes the process to terminate. Anti-disassembly tricks that use indirect JMP via computed values (Technique 6) are subject to CFG validation: the computed address must be in the bitmap of valid indirect branch targets. If your "hidden" function is reachable only through an indirect jump and you didn't declare it as a valid CFG target at build time, CFG will kill the process. Mitigation: compile your implant WITHOUT CFG enforcement (/guard:cf- in MSVC, or use GCC/MinGW which doesn't enable CFG), or ensure all your computed indirect jump targets are declared as valid. CFI is a hardening control, not a detection control — it can break your own obfuscation if you're not careful.