Practical Binary Analysis

Andriesse wrote his doctorate on binary analysis and then wrote this, which is unusual in that it is genuinely a textbook that is also usable. The first part is format and structure, the second is analysis fundamentals, the third is the advanced techniques - instrumentation, taint analysis, symbolic execution - each with a chapter of principles followed by a chapter of practice with a real tool.

Three modules cite it, and between them they cover the arc a reversing player follows: learn what a binary is, learn to read one, then learn to make a machine explore it for you.

Why it is on this shelf

Because the reversing category assumes a model of compilation that most people never explicitly learn. Why are there two symbol tables and why is one missing. Why does the disassembly change when you start at a different offset. What is the difference between what the loader maps and what the linker recorded. All of that is here, in order, with the reasons.

Chapter 1, Anatomy of a Binary

The compilation pipeline, one stage at a time: preprocessing, compilation to assembly, assembly to an object file, linking to an executable, and loading. Each stage throws information away, and knowing which stage discarded what tells you what is still recoverable.

The practical consequence is the one every reversing player needs early: symbols are debugging conveniences, not part of the program. A stripped binary is not damaged - it runs identically - it has simply lost the names, and everything you do afterwards is reconstructing names from behaviour. The same goes for types, local variable names and most control-flow structure. Start with reading a binary and the ELF and PE analyzer.

Chapter 2, The ELF Format

ELF laid out completely: the executable header, the program headers that tell the loader what to map, the section headers that tell the linker what is where, and the contents of the sections that matter - .text, .rodata, .data, .bss, .plt, .got, .dynamic, .symtab and .dynsym.

Two things from this chapter earn their keep immediately. The first is that program headers and section headers describe the same file for two different audiences, which is why a binary can run perfectly with its section headers destroyed and why some challenges do exactly that. The second is the PLT and GOT relationship: calls to library functions go through a stub that reads an address from a writable table, which is the mechanism behind lazy binding and also the mechanism behind a large family of exploitation techniques. Writing shellcode and the ROP gadget finder assume you know this.

Chapter 6, Disassembly and Binary Analysis Fundamentals

The chapter that changes how you read tool output. Linear disassembly walks the bytes from the start and decodes each instruction in sequence; recursive disassembly follows control flow from known entry points. Neither is correct in general.

Linear disassembly breaks on data inlined among instructions, because it will decode a jump table as code and then stay wrong until it happens to resynchronise. Recursive disassembly misses anything reached only by an indirect jump it could not resolve. On x86, where instructions are variable-length and can be decoded starting at any byte, both failures are routine, and they are the reason obfuscated binaries are hard: the obfuscation is not hiding the code, it is making the disassembler start in the wrong place.

The rest of the chapter builds up the structures analysis is done on - basic blocks, control-flow graphs, call graphs, and the distinction between control-flow and data-flow analysis. Those are the objects your decompiler is manipulating, and knowing that makes its mistakes legible.

Chapter 12, Principles of Symbolic Execution

The clearest short explanation of symbolic execution in print. Instead of running the program on a concrete input, run it on a symbol, and carry an expression for every value plus a path constraint recording the branches taken. Reach the target block, hand the accumulated constraint to a solver, and it hands back an input that gets there.

This is precisely the tool for the classic reversing challenge: a binary that reads a serial and prints success or failure. You do not have to understand the check, you only have to constrain the outcome. The chapter is equally clear about the cost - path explosion, which is exponential in branches, and the constraints a solver cannot handle, of which a hash is the standard example. That is why concolic execution, mixing concrete and symbolic values, is what real tools do.

Dynamic binary analysis covers the practical side, and coverage-guided fuzzing is the other half of the answer: when the constraint is a checksum, a fuzzer that learns from coverage often gets further than a solver.

Where it stops

It is Linux and x86-64, deliberately and almost exclusively. PE gets a brief chapter and Windows analysis is not the subject; ARM is essentially absent, which matters for mobile and firmware challenges. Go to Practical IoT Hacking and Windows Security Internals for those.

It is also not an exploitation book. It will tell you exactly how the GOT works and never once suggest overwriting one. And the advanced half depends on specific tools - libdft, Triton, Pin - which is fine for learning and dated for doing.

What to take into a challenge

Establish the ground truth before you read a single instruction. What format, what architecture, statically or dynamically linked, stripped or not, what is imported, what strings are in it. Five minutes of that decides whether the challenge is a symbol you can look up, a check you can solve, or a program you have to read - and reading is the expensive option you should choose last.