Reconnaissance and OSINT
Treat open-source intelligence as a pivot loop rather than a search, and know what metadata survives which route.
Assumes1. Recognising encodings
By the end you can
- Run the pivot loop: enumerate, expand, cross-reference, record
- Predict which metadata survives a given publication route, and check the right thing first
- Narrow an image's location from visible constraints without recognising the place
- Enumerate an organisation's subdomains from certificate transparency rather than by probing
- Work a leaked dataset from the shell - shape it, index it, then query it - rather than by opening files
- State where open-source ends and unauthorised access begins
1. Read
OSINT as a method, not a lucky search
Open-source intelligence challenges reward discipline over cleverness. The pivot loop, what metadata actually survives, how to geolocate from an image without recognising the place - and where the line is.
Working a leaked dataset: OSINT on a pile of data
Some OSINT challenges hand you a dump - a CSV, a SQL export, a folder of documents - and a question buried in it. The command-line workflow for turning gigabytes of leaked data into the one record that answers the challenge.
2. Use the tools
In the order they come up while solving. Read what each one does, or go straight to the workspace tab that runs it.
- EXIF metadata viewer
Read EXIF, GPS coordinates, camera details, timestamps, and embedded comments from images - the metadata that answers OSINT challenges.
Open it in the workspace - GPS coordinate converter (DMS, decimal, UTM)
Convert between degrees-minutes-seconds, decimal degrees, and other coordinate notations - for EXIF locations and OSINT challenges.
Open it in the workspace - Timestamp converter (Unix, ISO, FILETIME, HFS+)
Convert between Unix seconds and milliseconds, ISO 8601, local time, and the non-standard epochs used by Windows, macOS, and Flask.
Open it in the workspace - Strings extractor for binaries and blobs
Pull printable ASCII and UTF-16 strings out of any file, filtered by length, with flag-pattern highlighting.
Open it in the workspace - Perceptual image hash: are these the same picture?
Compare two images by aHash, dHash and pHash in the browser - a re-encoded, resized or recompressed copy has a different checksum and nearly the same perceptual hash.
Open it in the workspace - Regex tester with match offsets and capture groups
Test regular expressions live against sample text, with every match’s offset, capture groups, and named groups broken out.
Open it in the workspace
3. Try one now
Generated in your browser and checked in your browser. No account, nothing to download, and a fresh one whenever you want another.
4. Practise on the real thing
Real picoCTF challenges that use these techniques, easiest first. 37 match in total - see the full index.
Common mistakes
The wrong turns this topic reliably produces. Written as the mistake rather than the rule, because the rule is easy to agree with and easy to walk straight past.
- Searching instead of pivoting. One good identifier - a username, a licence plate, a certificate serial - is worth more than an hour of rephrased queries.
- Trusting a stripped EXIF as proof of nothing. The route matters: a platform upload strips GPS, a direct file share does not, and the thumbnail often outlives the edit.
- Recording conclusions rather than sources. A pivot chain nobody can retrace is not intelligence, and in a real engagement it is not admissible either.
- Only ever adding evidence for the first hypothesis. A pivot chain that never tries to falsify itself identifies the wrong person eventually, so name what would rule the theory out before collecting more that confirms it.
- Probing a target for something the public record already answers. Certificate transparency, archives and registries cost nothing and leave no trace; an unauthorised scan does both, and it is the step that turns research into an offence.
Checkpoint
Given one photograph with no GPS tag, narrow its location to a city and state which visible constraint eliminated each region you ruled out.
Teaching note
The ethics half is not an aside and should be taught first, not last. OSINT technique transfers directly to real people who did not consent to being investigated, and a class that learns the pivot loop without the line learns half a skill.
Go deeper
The lessons above are written to get you through a challenge. These go after the subject instead. Each one opens our notes on that chapter - what it argues, what to take from it and where it stops - so this is somewhere to read now rather than a book to buy first. Nothing here is affiliate-linked or sold by us.
Chapter 2, Acquiring Datasets
Hacks, Leaks, and Revelations - Micah Lee
The provenance and safety questions to settle before touching a dataset, which is the part CTF practice never forces you to ask.
Chapter 5, Docker, Aleph, and Making Datasets Searchable
Hacks, Leaks, and Revelations - Micah Lee
Turns a pile of files into something queryable, which is the difference between reading a leak and working one.
Chapter 5, Web Hacking Reconnaissance
Bug Bounty Bootcamp - Vickie Li
The same loop pointed at an organisation's infrastructure, with the enumeration sources named.
Every book the curriculum cites has a page in the library.