Evasive Malware

Cucci's subject is the adversarial relationship between a sample and the person analysing it. Every technique in the book exists because an analyst did something, and every countermeasure exists because the malware author noticed. Reading it as a defender teaches you what to look for; reading it as a CTF player teaches you what the challenge author has probably done to you.

Three modules cite it - file forensics, steganography and malware - and it is one of the two most recent books on this shelf, which matters for a subject where the specific checks change every year even though the categories do not.

Why it is on this shelf

Because reversing challenges borrow directly from this material. A binary that refuses to run under a debugger, a string table that is empty until runtime, an entry point that jumps into a region that does not exist yet - these are all here, named, with the mechanism explained and the workaround given.

Chapter 2, Malware Triage and Behavioral Analysis

The first-hour chapter. Hash the sample, look at the format and the compile timestamp, read the imports, read the strings, check the section names and entropy, and only then run it in an instrumented environment and record what it touches.

The ordering is the lesson, and it is the same ordering first hour triage argues for on any unknown file: everything free comes before anything expensive. The entropy check in particular is a thirty-second measurement that answers a large question - a section whose bytes are indistinguishable from random is compressed or encrypted, and there is no point reading its disassembly until that changes.

The behavioural half - files created, registry keys written, processes spawned, network contacted - is what malware triage at scale automates, and the strings extractor and ELF and PE analyzer do the static half here.

Chapter 10, Anti-Debugging

The techniques that detect a debugger, grouped by what they actually observe. Some ask the operating system directly. Some read the flag the loader sets in the process's own memory. Some measure time across a region, because single-stepping is thousands of times slower than running. Some look for the breakpoint byte the debugger wrote into their own code. Some exploit the fact that a debugger is attached by trying to attach something else and failing.

The grouping is what makes the chapter useful rather than a list of tricks, because the defeat follows from the observation. A check that reads a flag can be patched. A check that measures time can be defeated by not single-stepping through it. A check that scans its own bytes for breakpoints is telling you to use a hardware breakpoint instead. Malware anti-analysis tricks works the same list from the CTF side.

Chapter 16, Encoding and Encryption

Cited from the steganography module, which is not where you would look for it, and it belongs there. This is the chapter on how malware hides data in plain sight: base64 and its variants, XOR with a single byte or a repeating key, custom substitution alphabets, ROT-style rotations, and layered combinations of all of them.

The reason it maps onto stego and misc challenges is that the goals are identical - make a payload survive transport while not looking like a payload - and so the recognisable artefacts are identical too. Alphabet-shaped output with an equals sign at the end. A blob whose byte histogram is flat except for one very common value, which is the XOR key showing through padding. A string that decodes to almost-English, meaning you have the right method and the wrong key.

Spot the encoding is the identification skill, the cipher identifier and XOR cipher tools do the work, and XOR crib dragging handles the repeating-key case.

Chapter 17, Packers and Unpacking Malware

What a packer actually is: the original program stored as compressed or encrypted data, plus a small stub that reconstructs it in memory and jumps to it. Everything else follows from that description, including how to detect one and how to defeat one.

Detection is the entropy and section anomalies from chapter two, plus an import table too short to do anything and an entry point outside the usual section. The general unpacking method is to let the stub do its job - run until the reconstruction is complete, find the moment control transfers to the real entry point, and dump memory there. That works regardless of which packer it is, which is why it beats hunting for a specific unpacker.

For a challenge, the practical signal is the same: if the strings are useless and the imports are three functions, you are not looking at the program yet. Reading a binary covers what to do once you are.

Where it stops

It is Windows, thoroughly. The API calls, the loader structures, the registry artefacts and the sandbox checks are all Windows-specific, and a Linux ELF challenge gets only the general principles. The appendix listing Windows functions used for evasion is excellent and entirely inapplicable elsewhere.

It is also a defender's book. It assumes you want to know what the sample does, not to extract a flag from it, so it will happily spend pages on infrastructure and persistence that a challenge does not have.

What to take into a challenge

Measure before you read. Entropy per section, import count, string count, section names: four numbers, thirty seconds, and they tell you whether the disassembly in front of you is the program or the wrapper. Reading a packed binary carefully is the single most common way to lose an hour in the reversing category.