Binary exploitation
Read a binary's protections, find an overflow offset in one crash, and build a ROP chain when the stack is not executable.
Assumes21. Reverse engineering
By the end you can
- Read NX, PIE, canary, and RELRO from a binary and say what each rules out
- Find an overflow offset with a de Bruijn pattern in a single crash
- Build a ret2libc chain, including stack alignment
- Turn a leaked pointer into a libc base and resolve arbitrary symbols
- Turn a format string into an arbitrary read and then an arbitrary write
- Name the four heap bugs that matter and the allocator behaviour each one abuses
- Write shellcode that survives the challenge's constraints - length, character set, and seccomp
1. Read
From crash to shell: stack overflows, offsets, ret2win, and ret2libc
A segfault is not an exploit. The path from an unexpected crash to a controlled instruction pointer to a shell, with the mitigation checks that decide which technique you need and the stack-alignment detail that breaks working exploits.
Integer bugs: overflow, signedness, truncation, and the off-by-one
The length check that passes because the length wrapped, the negative index that survives a bounds check, and the 16-bit truncation that turns 65,540 into 4. Where arithmetic bugs come from and how to spot them in a decompiler.
Format string bugs: when %n writes where you point
A single printf(user_input) is a read and a write primitive in disguise. How %x leaks the stack, %s dereferences arbitrary pointers, and %n turns a logging bug into a controlled memory write - the whole ladder from leak to code execution.
The glibc heap: chunks, bins, and the four bugs that matter
What malloc actually stores, why tcache made modern heap exploitation easy, and how use-after-free, double free, tcache poisoning and a one-byte overflow each turn into an arbitrary write. The menu-driven heap challenge, decoded.
Writing shellcode that fits: constraints, encoders, and seccomp
The execve stub in twenty-three bytes, and what to do when the buffer is short, the bytes must be printable, nulls are forbidden, or seccomp has taken execve away. Shellcode as a constraint-satisfaction problem.
2. Use the tools
In the order they come up while solving. Read what each one does, or go straight to the workspace tab that runs it.
- ELF, PE and Mach-O binary analyzer
Parse headers, sections, imports, and symbols from Linux, Windows, and macOS binaries, with C++ symbol demangling and gadget discovery.
Open it in the workspace - Buffer overflow offset finder
Find the exact number of bytes before the saved return address using a de Bruijn pattern and the value from a single crash.
Open it in the workspace - Cyclic pattern generator and offset finder
Generate a de Bruijn sequence and find the exact overflow offset from a crashed register value - the pwntools cyclic workflow, in the browser.
Open it in the workspace - ROP gadget finder
Search a binary for return-oriented programming gadgets, filter by the registers they touch, and exclude ones containing bad bytes.
Open it in the workspace - ROP chain and payload builder
Assemble an exploit payload from padding, addresses, and raw bytes, with a live hexdump and offset ruler - the pwntools flat() workflow.
Open it in the workspace - Libc base address calculator
Turn a leaked libc pointer into the library’s base address, then resolve any other symbol - the arithmetic every ret2libc exploit runs on.
Open it in the workspace - Format string exploit builder
Find your input’s position on the stack and build %n write primitives - turning an uncontrolled printf into an arbitrary read and write.
Open it in the workspace - Glibc heap exploitation helper
Model glibc chunk layout, bin behaviour, and tcache - the size classes and metadata that heap challenges turn on.
Open it in the workspace - Shellcode assembler and library
Assemble x86 and x86-64 shellcode, or pick a ready execve(/bin/sh) payload, with null-byte-free variants and length reporting.
Open it in the workspace - Struct pack and unpack (p32, p64, u32, u64)
Convert integers to little-endian byte strings and back at 8, 16, 32, and 64 bits - the pwntools p32/p64 helpers without the install.
Open it in the workspace - Pwntools exploit script generator
Generate a working pwntools template with the right context, process or remote connection, and the boilerplate every exploit repeats.
Open it in the workspace
3. Try one now
Generated in your browser and checked in your browser. No account, nothing to download, and a fresh one whenever you want another.
4. Practise on the real thing
Real picoCTF challenges that use these techniques, easiest first. 66 match in total - see the full index.
Common mistakes
The wrong turns this topic reliably produces. Written as the mistake rather than the rule, because the rule is easy to agree with and easy to walk straight past.
- Forgetting the stack-alignment ret gadget on 64-bit. The chain is correct and crashes inside system anyway, and students conclude the whole thing is wrong.
- Hardcoding a libc address from your own machine. The remote libc differs; the leak exists so that the base is computed, not assumed.
- Counting the offset by hand from the source. A cyclic pattern gives it in one crash and does not care about padding, alignment or the compiler's opinions.
- Testing only under a debugger. GDB disables ASLR and rewrites the environment, so the stack layout differs from the one the target actually runs with and a working exploit can still be untested.
- Building a chain without checking which bytes survive the read. scanf stops at whitespace and strcpy at a null, so the primitive that gets your payload in defines the alphabet the whole chain has to live inside.
Checkpoint
Given a 64-bit binary with NX and no PIE, produce a working ret2libc exploit and explain each entry in the chain.
Teaching note
The alignment `ret` gadget is the single most common blocker: the exploit is correct and crashes inside `system` anyway. Warn about it in advance or students will conclude their whole chain is wrong.
Go deeper
The lessons above are written to get you through a challenge. These go after the subject instead. Each one opens our notes on that chapter - what it argues, what to take from it and where it stops - so this is somewhere to read now rather than a book to buy first. Nothing here is affiliate-linked or sold by us.
Chapter 8, Buffer Overflows and Stack and Heap Manipulation
Hunting Security Bugs - Tom Gallagher, Bryan Jeffries, and Lawrence Landauer
The bug from the finder's side - where overflows come from - rather than only how to ride one.
Chapter 9, Format String Attacks
Hunting Security Bugs - Tom Gallagher, Bryan Jeffries, and Lawrence Landauer
Why a format string is a write primitive at all, stated more carefully than most references bother to.
Chapter 1, Anatomy of a Binary
Practical Binary Analysis - Dennis Andriesse
The layout and loading model every one of these exploits is manipulating.
Chapter 4, NULL Pointer FTW
A Bug Hunter's Diary - Tobias Klein
One real bug followed from discovery to working exploit, which is the arc a CTF challenge compresses.
Every book the curriculum cites has a page in the library.