Hacks, Leaks, and Revelations

Lee has worked on the Snowden archive, BlueLeaks, the Oath Keepers dumps, Parler and Epik, and this book is the working method behind that: acquire the data safely, get it into a shape you can query, and then ask questions of it rather than reading it.

It is also unusually careful about the ethics and the safety, since the datasets are real and the people in them are real. Two modules cite it - encodings and OSINT - and both cite the technique rather than the journalism.

Why it is on this shelf

Because CTF challenges have got bigger. A forensics or OSINT challenge that hands you a whole disk, a chat export, a database dump or an archive of thousands of documents is asking a question this book answers: how do you look at something too large to look at?

Chapter 2, Acquiring Datasets

Where data comes from and what it costs to touch it. Public archives, leak sites, direct sources, and in each case the questions of provenance and verification that decide whether you can trust what you have.

The verification material is what earns it a citation from the OSINT module. His questions - who produced this, how did it get here, does its internal structure match its claimed origin, do independent facts inside it agree with facts outside it - are exactly the questions an OSINT challenge is testing, and the same discipline the OSINT method applies to a person rather than a dataset. The operational-security material is the other half: the mechanics of not becoming part of the story.

Chapter 4, Exploring Datasets in the Terminal

The chapter the foundations module cites, and it is essentially a course in structured curiosity with standard Unix tools. Measure disk usage per folder to find where the mass is. Build an inventory of filenames. Count files by extension. Grep for a term, then grep with a regular expression, then grep in bulk across the tree and keep the file list rather than the matches.

The habit underneath is the one that makes it a foundations citation: look at the bytes before deciding what a file is, and look at the shape of a corpus before deciding what it contains. The extension is a claim, the directory name is a claim, and the only thing that is not a claim is the content. That is the same argument spot the encoding makes one file at a time, and the file signature identifier is the tool for it.

He also covers encrypted material inside a dataset, which in a challenge is usually the point: the archive is a haystack and the encrypted item is the needle. Working a leaked dataset is the CTF-shaped version of this whole chapter, and the regex tester helps with the patterns.

Chapter 5, Docker, Aleph, and Making Datasets Searchable

The step past grep. Grep searches text; it cannot see inside a PDF, an office document, an image of a scanned page or an email archive. Aleph, the open-source platform built by the investigative journalism community, ingests a directory tree, extracts text from every format including by optical character recognition, and gives you a full-text index over all of it.

The chapter uses Docker to run it, and the Docker material is genuinely useful on its own - containers as a way of running an awkward analysis tool without installing it, which is how most forensic tooling should be run. The Docker image inspector here is for the opposite case, when a container image is itself the artefact.

For competition purposes, the general lesson is about when indexing pays for itself. Setting up a search index is minutes of work, and it wins the moment you need to search a corpus a second time with a different term - which, in a large forensics challenge, you always do.

Where it stops

It is a journalism book. Large parts concern source protection, legal exposure, editorial judgment and publication, none of which apply to a challenge. Its later chapters walk specific real datasets in detail, which is instructive but long.

It is also not a forensics book. Filesystem artefacts, deleted data, memory and timelines are absent - for those, Practical Forensic Imaging is the right shelf.

What to take into a challenge

Ask a question the data can answer. A large artefact will absorb unlimited time if you browse it, and it will give up the answer quickly to a specific query. Write down what you are looking for before you open anything, and when a search returns nothing, change the question rather than reading more of the corpus.