Malware Dev VM Setup

Why Your Daily Machine Is the Wrong Place for This

Imagine you've just written a small C program that calls VirtualAllocEx, WriteProcessMemory, and CreateRemoteThread. You compile it. You run it. A few things happen simultaneously:

• Windows Defender scans the binary the instant it lands on disk and flags it.
• If Defender doesn't catch it statically, it catches the memory-injection behavior and terminates your test process mid-run.
• Windows Error Reporting captures a crash dump and queues it for upload to Microsoft servers — including a snapshot of the process's memory at the time of termination.
• SmartScreen phones home to check the file's reputation hash.
• Telemetry services log the process creation, the loaded modules, the network connections — all sent to Microsoft's backend.

None of this is hypothetical. It is what Microsoft explicitly designed Windows to do. Your development machine actively works against you every time you compile and run a tool that looks like malware — because to Windows, it is malware.

Beyond Defender and telemetry, there is a more practical concern: offensive development produces unstable code. You will write shellcode that crashes the host process. You will test injectors that corrupt the target process's heap. You will make kernel drivers that blue-screen the machine. Doing any of this on your daily system means losing your unsaved work, rebooting mid-session, and potentially corrupting your personal files.

The solution to both problems is the same: do everything inside a virtual machine that you can snapshot, roll back, and destroy without consequence.

Choosing a Hypervisor

You have three realistic options for running Windows VMs on a development machine. Each has trade-offs that matter specifically for offensive development work.

┌─────────────────────┬──────────────────────┬──────────────────────┐
│                     │  VMware Workstation  │    VirtualBox        │
│                     │  Pro                 │    (free)            │
├─────────────────────┼──────────────────────┼──────────────────────┤
│ Cost                │ Paid (~$200 once)    │ Free                 │
│ USB passthrough     │ Excellent            │ Good                 │
│ Nested virt (VT-x)  │ Full support         │ Limited              │
│ Snapshot speed      │ Fast                 │ Slower on large disks│
│ Guest integration   │ VMware Tools         │ Guest Additions       │
│ Kernel debug attach │ Works well           │ Works                │
│ COM port for WinDbg │ Easy                 │ Possible             │
│ Detection by guests │ VMware artifacts     │ VBox artifacts       │
│ Recommendation      │ ★ Preferred          │ ★ Good enough        │
└─────────────────────┴──────────────────────┴──────────────────────┘

  Hyper-V (built into Windows Pro): avoid for offensive dev.
  It uses a Type-1 hypervisor that modifies the host kernel — once
  Hyper-V is enabled, your host OS itself runs as a VM (VBS/HVCI).
  This creates compatibility problems with low-level tools and makes
  your VM environment harder to control precisely.
        

For most work in this book, VirtualBox is sufficient and costs nothing. If you plan to build kernel drivers (Part 15) and need reliable named pipe COM port connections for kernel debugging across VMs simultaneously, VMware Workstation's COM port forwarding is noticeably smoother. Either works.

Windows VM Configuration: Silencing the OS Against You

A fresh Windows install is an adversary. Defender is scanning everything you compile. Telemetry is logging every process creation. Automatic updates are ready to re-enable every setting you change. Before you write a single line of code, you need to neutralize all of this.

Step 1 — Install Windows

Use Windows 10 21H2 or Windows 11 22H2. Get the ISO from Microsoft's official media creation tool. During installation, choose "I don't have a product key" (you can use it without activation — all features work, there's just a watermark). Do not sign in with a Microsoft account during setup — create a local account. Name it something generic (user or dev) — your username appears in build paths and debug symbols.

After the first boot, immediately take a hypervisor snapshot named "00 — Fresh Install". This is your last clean state before any configuration. If something goes badly wrong later, you can always return here.

Step 2 — Disable Windows Defender Permanently

You cannot just right-click the tray icon and turn it off. Microsoft designed Defender to re-enable itself. You need to disable it at the policy level before Defender's tamper protection can lock you out:

# Run PowerShell as Administrator

# 1. Disable tamper protection first (requires GUI — go to:
#    Windows Security → Virus & threat protection → Manage settings
#    → Tamper Protection → Off)

# 2. Disable Defender via registry policy
New-Item -Path "HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender" -Force
Set-ItemProperty -Path "HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender" `
    -Name "DisableAntiSpyware" -Value 1 -Type DWord

# 3. Disable real-time protection
Set-ItemProperty -Path "HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection" `
    -Name "DisableRealtimeMonitoring" -Value 1 -Type DWord

# 4. Disable cloud-delivered protection
Set-ItemProperty -Path "HKLM:\SOFTWARE\Policies\Microsoft\Windows Defender\Spynet" `
    -Name "SpynetReporting" -Value 0 -Type DWord

# 5. Stop and disable the service
Stop-Service -Name WinDefend -Force
Set-Service -Name WinDefend -StartupType Disabled

After running this and rebooting, open Windows Security and confirm "Virus & threat protection" shows "Threat protection is not active." If it still shows active, you may need to boot into Safe Mode to make the registry changes before Defender's self-protection loads.

Step 3 — Kill Telemetry

Even with Defender disabled, Windows still sends detailed telemetry to Microsoft — process creation events, crash dumps, network activity. For a malware dev environment, this is unacceptable.

# Disable telemetry services
$services = @(
    "DiagTrack",           # Connected User Experiences and Telemetry
    "dmwappushservice",    # WAP Push Message Routing Service
    "WerSvc",              # Windows Error Reporting
    "wercplsupport",       # Problem Reports Control Panel
    "PcaSvc"               # Program Compatibility Assistant
)
foreach ($svc in $services) {
    Stop-Service -Name $svc -Force -ErrorAction SilentlyContinue
    Set-Service  -Name $svc -StartupType Disabled -ErrorAction SilentlyContinue
}

# Block telemetry endpoints at the hosts file level
$hostsPath = "C:\Windows\System32\drivers\etc\hosts"
$blocked = @(
    "0.0.0.0 watson.microsoft.com",
    "0.0.0.0 vortex.data.microsoft.com",
    "0.0.0.0 settings-win.data.microsoft.com",
    "0.0.0.0 telemetry.microsoft.com",
    "0.0.0.0 oca.microsoft.com",
    "0.0.0.0 sqm.microsoft.com"
)
Add-Content -Path $hostsPath -Value ($blocked -join "`n")

Step 4 — Disable Automatic Updates

Automatic updates will re-enable Defender signatures and undo your policy settings. Disable them for good:

# Disable Windows Update services
$updateServices = @("wuauserv", "UsoSvc", "WaaSMedicSvc")
foreach ($svc in $updateServices) {
    Stop-Service $svc -Force -ErrorAction SilentlyContinue
    Set-Service  $svc -StartupType Disabled -ErrorAction SilentlyContinue
}

# Group Policy: disable Windows Update
$wuPath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU"
New-Item -Path $wuPath -Force | Out-Null
Set-ItemProperty -Path $wuPath -Name "NoAutoUpdate" -Value 1 -Type DWord

After all three steps, take another snapshot: "01 — Configured Base (No Defender, No Telemetry)". This is your permanent starting point. Every future snapshot branches from here.

The Essential Toolset

You need two categories of tools: tools you use to build your malware, and tools you use to observe what your malware does as it runs. Both categories are critical — you cannot write good offensive code if you can't see exactly what it's doing at the system level.

┌─────────────────────────────────────────────────────────────────┐
│                        BUILD TOOLS                              │
├──────────────────────┬──────────────────────────────────────────┤
│ MinGW-w64            │ GCC for Windows — primary compiler       │
│ MSVC (VS Build Tools)│ Microsoft compiler — for COM/DLL work    │
│ NASM                 │ x64 assembly — for shellcode stubs       │
│ Python 3 + pip       │ Scripting, analysis, quick tools         │
│ Visual Studio Code   │ Editor — C/C++/ASM with IntelliSense     │
│ CMake                │ Build system for larger projects         │
│ make (via MinGW)     │ Simple build automation                  │
└──────────────────────┴──────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                       OBSERVE TOOLS                             │
├──────────────────────┬──────────────────────────────────────────┤
│ x64dbg               │ Debugger — step through your code live   │
│ WinDbg Preview       │ Kernel debugger — required for Part 15   │
│ Process Hacker 2     │ See processes, threads, handles, memory  │
│ Procmon              │ Every FS/registry/process event live      │
│ API Monitor          │ Hook and log every Win32 API call + args  │
│ Wireshark            │ Packet capture for C2 traffic testing     │
│ Inetsim (Linux VM)   │ Fake internet (DNS, HTTP, SMTP) locally  │
└──────────────────────┴──────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                        PE TOOLS                                 │
├──────────────────────┬──────────────────────────────────────────┤
│ PE-bear              │ Best visual PE structure viewer           │
│ PEStudio             │ Threat indicators, import scoring         │
│ CFF Explorer         │ PE editing — sections, imports, headers  │
│ Ghidra               │ Disassembler/decompiler (NSA, free)      │
│ IDA Free             │ Industry standard — free 64-bit version  │
└──────────────────────┴──────────────────────────────────────────┘
        

x64dbg — Your Primary Debugger

x64dbg is the debugger you will have open constantly. Open source, actively maintained, and with a plugin ecosystem specifically built for malware analysis. The three plugins you want installed immediately:

• ScyllaHide — automatically hides the debugger from anti-debug checks. When your own code has anti-debug protections (Part 6), ScyllaHide lets you step through them without triggering them. Patches IsDebuggerPresent, NtQueryInformationProcess, heap flags, timing checks — all transparently.
• x64dbgpy — Python scripting inside x64dbg. Automate repetitive analysis tasks: "dump all strings from this allocation," "log every call to VirtualAlloc with its arguments," etc.
• Ret-Sync — synchronizes x64dbg's current instruction pointer into IDA or Ghidra. Step through code in the debugger and the decompiler follows you automatically. Invaluable once you're working on complex code.

Process Hacker 2 — Your System Observatory

Run Process Hacker before you run anything you've written. It stays open the whole time. When your injector launches, you'll see:

• The target process's memory map change — new regions appear, their protection flags change from RW to RX.
• New threads created inside the target.
• New handles opened (process handles, file handles, registry handles).
• DLLs loaded into the target's address space.
• Network connections opened.

Without a tool like this running, you're coding blind. You write something, run it, and either it works or it doesn't. With Process Hacker open, you see exactly which step succeeded and which failed.

Inetsim — Fake Internet on Your Host-Only Network

When your implant makes HTTP requests to its C2, you don't want those requests going to the real internet from a test environment. Inetsim (Internet Simulator) runs on a companion Linux VM and simulates DNS, HTTP, HTTPS, SMTP, FTP, and more. Point your Windows VM's DNS at the Linux VM's IP, and every DNS query gets a real answer (the Linux VM's IP), every HTTP GET gets a fake 200 OK response, every HTTPS connection gets a self-signed cert.

Your C2 implant thinks it's talking to its real server. You capture everything in Wireshark. Nothing leaves the host-only network.

MinGW vs. MSVC: Which Compiler and When

The choice of compiler affects not just how you build but what your binary looks like to a defender. Compiled binaries carry the fingerprints of their toolchain — import patterns, exception handling structures, runtime dependencies, section layouts. Understanding both compilers lets you choose the right one for each job and understand why defenders can sometimes tell them apart.

  MinGW-w64 (GCC for Windows)
  ─────────────────────────────────────────────────────────────────
  Best for:   shellcode, loaders, CRT-free code, cross-compilation
              from Linux, anything where you want -nostdlib
  Runtime:    msvcrt.dll (old Microsoft CRT, already in all Windows)
  Strengths:  -nostdlib works cleanly, fine control over sections,
              easy to strip completely, cross-compiles from Linux,
              NASM integrates simply
  Weaknesses: 64-bit SEH is clumsier, some COM patterns need MSVC,
              slightly non-standard C++ exception ABI

  MSVC (cl.exe / Visual Studio Build Tools)
  ─────────────────────────────────────────────────────────────────
  Best for:   DLLs loaded into Windows processes, COM-based code,
              anything using C++ exceptions, shellcode tests where
              you want identical CRT to the host process
  Runtime:    UCRT (Universal CRT) — requires vcruntime140.dll etc.
  Strengths:  Native Windows ABI, full SEH support, COM/WinRT,
              binaries look exactly like Microsoft-built software
  Weaknesses: Harder to go CRT-free, requires larger install,
              /NODEFAULTLIB setup is verbose

  In practice: Use MinGW for shellcode and loaders.
               Use MSVC for injection DLLs and COM-heavy work.
               Install both. Switch per project.
        

Installing MinGW-w64

The cleanest way on Windows is via winget or the MSYS2 installer:

# Via winget (Windows 10+)
winget install MSYS2.MSYS2

# Then inside the MSYS2 shell:
pacman -S mingw-w64-x86_64-gcc mingw-w64-x86_64-binutils nasm make

After installation, add C:\msys64\mingw64\bin to your PATH. Verify with x86_64-w64-mingw32-gcc --version. You should see something like gcc version 13.x.x.

Installing MSVC (Visual Studio Build Tools)

You do not need the full Visual Studio IDE — just the build tools:

winget install Microsoft.VisualStudio.2022.BuildTools

# During install, select:
# "Desktop development with C++"
# (this brings cl.exe, link.exe, Windows SDK, and the UCRT)

To use MSVC from the command line without the IDE, open "x64 Native Tools Command Prompt for VS 2022" — it sets all the necessary environment variables (INCLUDE, LIB, PATH) for cl.exe to find the SDK headers and libraries.

Compiler Flags That Shrink Your Footprint

A naive compile of a C program with MinGW produces a binary with a visible GCC version string, DWARF debug information, unneeded sections (.eh_frame for exception handling), and unnecessary standard library imports. None of these help you — they only make your binary larger, more detectable, and easier to attribute to a specific toolchain. Strip them all from the first build.

The MinGW Baseline Flags

CC     = x86_64-w64-mingw32-gcc
CFLAGS = -Os                          \
         -nostdlib                    \
         -nodefaultlibs               \
         -fno-ident                   \
         -fno-asynchronous-unwind-tables \
         -fno-exceptions              \
         -ffunction-sections          \
         -fdata-sections              \
         -mno-stack-arg-probe

LFLAGS = -Wl,--gc-sections            \
         -s                           \
         -Wl,--no-seh

# Usage:
# $(CC) $(CFLAGS) -o implant.exe implant.c $(LFLAGS)

What each flag does and why it matters:

  -Os
    Optimize for size. Smaller binary = less surface area for YARA rules
    to match against. Also avoids some loop-unrolling that creates
    distinctive code patterns.

  -nostdlib / -nodefaultlibs
    Do not link any standard library (libc, libgcc, libmsvcrt).
    This is how you produce a truly CRT-free binary — essential for
    shellcode and loaders that resolve everything via PEB walk.
    Without this, even a "hello world" imports printf and _start
    from the CRT, giving Defender something to chew on.

  -fno-ident
    Removes the .comment ELF section that GCC adds by default:
    "GCC: (x86_64-w64-mingw32-gcc) 13.2.0"
    Without this, anyone who runs strings on your binary instantly
    knows your compiler version. Remove it.

  -fno-asynchronous-unwind-tables
    Removes .eh_frame and .pdata sections used for C++ exception
    unwinding and stack walking. In CRT-free code you don't need these.
    Their absence slightly reduces binary size and removes a section
    that some tools use to fingerprint GCC-compiled code.

  -ffunction-sections / -fdata-sections + -Wl,--gc-sections
    Puts each function and data item in its own section, then the
    linker garbage-collects every section that nothing references.
    Removes dead code that GCC would otherwise keep. In practice
    this strips 20-40% off a small binary.

  -mno-stack-arg-probe
    Without this, GCC generates calls to __chkstk (or ___chkstk_ms)
    when stack allocations are large. That creates an import you
    don't want in a CRT-free binary. This flag eliminates it.

  -s (via linker -Wl,-s or strip --strip-all after build)
    Strips all symbols and debug info from the final binary.
    Function names, variable names, source file paths — gone.
        

The MSVC Baseline Flags

cl.exe implant.c ^
    /O1           ^
    /GS-          ^
    /DNDEBUG      ^
    /nologo       ^
    /W0           ^
    /NODEFAULTLIB ^
    /ENTRY:main   ^
    /link         ^
    /SUBSYSTEM:CONSOLE ^
    /INCREMENTAL:NO    ^
    /DEBUG:NONE

  /O1
    Optimize for minimum size (favors smaller code over speed).

  /GS-
    Disable the security cookie (stack canary). Without this, MSVC
    inserts __security_check_cookie calls that import from the CRT.
    In CRT-free code, that import doesn't resolve and linking fails.
    Disable it. (Note: in production code this is a real security
    regression — here it's acceptable because we're not defending
    our malware, we're evading detection.)

  /NODEFAULTLIB
    Don't link against MSVC's default runtime libraries (UCRT, etc.).
    Combined with /ENTRY:main to specify a custom entry point, this
    produces a CRT-free binary from MSVC just like -nostdlib in GCC.

  /DEBUG:NONE
    Ensure no PDB file is generated and no debug directory is embedded
    in the PE. Without this, MSVC embeds a path to the PDB file even
    if the PDB doesn't exist — leaking your build machine's path in
    the binary's debug directory.
        

Verifying the Result

After building with these flags, open the binary in PE-bear and check:

• Import Directory — should be empty or contain only the exact APIs you explicitly import. No CRT imports.
• Debug Directory — should be empty. No PDB path.
• Sections — .text and .data only. No .eh_frame, no .pdata, no .comment.
• Strings — run strings64.exe -nobanner implant.exe. No GCC version string, no build path, no function names.

# Quick check with Sysinternals strings
strings64.exe -nobanner implant.exe | Select-String -Pattern "(gcc|GCC|Users|Projects|mingw)"
# Should return nothing if your flags are correct

Lab Network Architecture

How your VMs are networked to each other and to the internet matters for every testing session. The wrong network configuration lets your test implants phone home to real infrastructure — or lets real malware escape your lab.

  ┌─────────────────────────────────────────────────────────────┐
  │                      HOST MACHINE                           │
  │                                                             │
  │  ┌──────────────────┐    ┌─────────────────────────────┐   │
  │  │   DEV VM         │    │   TARGET VM (clean Windows) │   │
  │  │  (this chapter)  │    │   "victim machine" for       │   │
  │  │                  │    │   injection tests            │   │
  │  │  Adapter 1:      │    │                             │   │
  │  │  Host-Only ──────┼────┼──── Host-Only               │   │
  │  │  (192.168.56.x)  │    │    (192.168.56.x)           │   │
  │  │                  │    └─────────────────────────────┘   │
  │  │  Adapter 2:      │                                      │
  │  │  NAT (for        │    ┌─────────────────────────────┐   │
  │  │  downloads only) │    │   INETSIM VM (Linux/Debian) │   │
  │  │                  │    │   Simulates: DNS, HTTP,      │   │
  │  └──────────────────┘    │   HTTPS, SMTP, FTP           │   │
  │                          │                              │   │
  │                          │   Host-Only ─────────────────┘   │
  │                          │   (192.168.56.x)                 │
  │                          └─────────────────────────────┘   │
  └─────────────────────────────────────────────────────────────┘

  Rule: Malware you are testing runs on the Target VM.
        The Target VM has ONLY Host-Only networking.
        It cannot reach the internet. It can reach Inetsim.
        Your implant's C2 calls go to Inetsim, not the real internet.

  The Dev VM has NAT for the moments you need to download tools
  or pull git repos. During actual test runs, disable the NAT
  adapter so you don't accidentally exfiltrate anything.
        

Snapshot Strategy

Snapshots are what make this workflow sustainable. Without a good snapshot strategy, every mistake that corrupts a VM means an hour of reconfiguration. With a good strategy, any mistake costs five seconds and a rollback.

  ◉ 00 — Fresh Install
  │   (OS install, local account, no Microsoft sign-in)
  │
  └─◉ 01 — Configured Base
      │   (Defender off, telemetry killed, updates blocked)
      │
      ├─◉ 02 — Tools Installed
      │   │   (FLARE / manual tool install: x64dbg, Ghidra,
      │   │    MinGW, MSVC, PE-bear, etc.)
      │   │
      │   ├─◉ 03 — Pre-Session [overwrite this before each session]
      │   │         Start here every day. Roll back here when
      │   │         something breaks.
      │   │
      │   └─◉ [Tagged snapshots for specific experiments]
      │           e.g., "After shellcode chapter 6 baseline"
      │                "After first reflective DLL working"
        

The "03 — Pre-Session" snapshot is the most important one. Before you start any coding session, take or refresh this snapshot. When your shellcode crashes the host process and leaves a corrupted registry key, when your driver BSOD's the VM, when your injector gets the target into an unrecoverable state — you roll back to Pre-Session and you're where you were five minutes ago. Zero lost work.

The Development Loop

Every chapter in this book follows the same basic cycle. Internalizing it now means you'll work faster and waste less time on frustrating dead ends.

  ┌──────────────┐
  │  Write code  │ ← VSCode or any editor on the Dev VM
  └──────┬───────┘
         │
         ▼
  ┌──────────────┐
  │   Compile    │ ← MinGW or MSVC from the terminal
  └──────┬───────┘
         │  Binary lands in your output directory
         ▼
  ┌──────────────────────────────────────────────────┐
  │  Static check                                    │
  │  PE-bear: look at imports, sections, headers     │
  │  strings: look for leaked paths and symbols      │
  └──────┬───────────────────────────────────────────┘
         │
         ▼
  ┌──────────────────────────────────────────────────┐
  │  Dynamic test on Dev VM (same machine)           │
  │  Process Hacker watching  →  run binary          │
  │  Procmon capturing        →  observe behavior    │
  │  Did it do what you expected?                    │
  └──────┬──────────────────┬───────────────────────┘
         │ YES              │ NO
         ▼                  ▼
  ┌─────────────┐    ┌──────────────────────────────┐
  │  Copy to    │    │  Attach x64dbg               │
  │  Target VM  │    │  Set breakpoint at entry      │
  │  and test   │    │  Step through instruction     │
  └──────┬──────┘    │  by instruction until you     │
         │           │  find the wrong behavior      │
         ▼           └──────────────────────────────┘
  ┌─────────────────────────────┐
  │  Test against clean target  │
  │  (rollback target VM first) │
  │  Check for artifacts:       │
  │  - Registry keys written    │
  │  - Files on disk            │
  │  - Network connections      │
  │  - Event log entries        │
  └──────────────┬──────────────┘
                 │
                 ▼
  ┌─────────────────────────────┐
  │  Roll back target VM        │
  │  Document what worked       │
  │  Snapshot Dev VM if stable  │
  └─────────────────────────────┘
        

Your First Build: The Empty Skeleton

Before writing anything meaningful, verify that your toolchain is working correctly with the flags you just configured. Here is the absolute minimum CRT-free Windows program — it does nothing except return a value, but building it correctly with no imports and no symbols proves your environment is ready.

// skeleton.c — the smallest possible CRT-free Windows program
// Compile: x86_64-w64-mingw32-gcc -Os -nostdlib -nodefaultlibs
//          -fno-ident -fno-asynchronous-unwind-tables
//          -ffunction-sections -fdata-sections
//          -o skeleton.exe skeleton.c
//          -Wl,--gc-sections -s -e main

#include <windows.h>

int main(void) {
    return 0;
}

Build it, then open it in PE-bear. What you should see:

• Import Directory: empty, or only kernel32.dll with a handful of entries (ExitProcess at minimum — the linker may add this automatically). No MSVCRT. No libgcc.
• Debug Directory: empty.
• Sections: .text only. Total file size under 3KB.
• Strings: run strings64.exe -nobanner skeleton.exe — nothing meaningful should appear.

If you see a CRT import (msvcrt.dll, __main, _start), your -nostdlib is not being applied correctly. Check that the flags appear after the source file in your command line — GCC processes arguments left to right and the position of linker flags matters.

Questions & Answers