Reconnaissance and OSINT

Before the session

• Nothing to install. Students need a browser and https://ctfpal.com. Confirm the room can reach it once; after that it works offline.
• Read the lesson yourself first - about 9 minutes.
• Have one worked example ready to paste. The classroom link builder on the instructor page turns it into a URL that opens preloaded.

Objectives

Written as things a student can do afterwards, so they can be assessed rather than asserted.

1. Run the pivot loop: enumerate, expand, cross-reference, record
2. Predict which metadata survives a given publication route, and check the right thing first
3. Narrow an image's location from visible constraints without recognising the place
4. Enumerate an organisation's subdomains from certificate transparency rather than by probing
5. Work a leaked dataset from the shell - shape it, index it, then query it - rather than by opening files
6. State where open-source ends and unauthorised access begins

Running order (90 min)

Tools used

• EXIF metadata viewer - Read EXIF, GPS coordinates, camera details, timestamps, and embedded comments from images - the metadata that answers OSINT challenges.https://ctfpal.com/?tool=exif-viewer
• GPS coordinate converter (DMS, decimal, UTM) - Convert between degrees-minutes-seconds, decimal degrees, and other coordinate notations - for EXIF locations and OSINT challenges.https://ctfpal.com/?tool=coordinate-converter
• Timestamp converter (Unix, ISO, FILETIME, HFS+) - Convert between Unix seconds and milliseconds, ISO 8601, local time, and the non-standard epochs used by Windows, macOS, and Flask.https://ctfpal.com/?tool=timestamp-converter
• Strings extractor for binaries and blobs - Pull printable ASCII and UTF-16 strings out of any file, filtered by length, with flag-pattern highlighting.https://ctfpal.com/?tool=strings-extractor
• Perceptual image hash: are these the same picture? - Compare two images by aHash, dHash and pHash in the browser - a re-encoded, resized or recompressed copy has a different checksum and nearly the same perceptual hash.https://ctfpal.com/?tool=perceptual-image-hash
• Regex tester with match offsets and capture groups - Test regular expressions live against sample text, with every match’s offset, capture groups, and named groups broken out.https://ctfpal.com/?tool=regex-tester

Reading

• OSINT as a method, not a lucky search - 5 min. Open-source intelligence challenges reward discipline over cleverness. The pivot loop, what metadata actually survives, how to geolocate from an image without recognising the place - and where the line is.
• Working a leaked dataset: OSINT on a pile of data - 4 min. Some OSINT challenges hand you a dump - a CSV, a SQL export, a folder of documents - and a question buried in it. The command-line workflow for turning gigabytes of leaked data into the one record that answers the challenge.

Practice set

Real picoCTF challenges tagged with this module’s techniques, easiest first. Assign the first three in class and the rest as homework.

1. Big Zip - picoGym Exclusive, easy
2. Bookmarklet - picoCTF 2024, easy
3. CanYouSee - picoCTF 2024, easy
4. Crack the Gate 1 - picoMini by CMU-Africa, easy
5. dont-use-client-side - picoCTF 2019, easy
6. First Find - picoGym Exclusive, easy
7. basic-file-exploit - picoCTF 2022, medium
8. Blast from the past - picoCTF 2024, medium
9. Client-side-again - picoCTF 2019, medium
10. B1g_Mac - picoCTF 2019, hard
11. Java Script Kiddie - picoCTF 2019, hard
12. Java Script Kiddie 2 - picoCTF 2019, hard

Checkpoint (gradeable)

Given one photograph with no GPS tag, narrow its location to a city and state which visible constraint eliminated each region you ruled out.

Deliberately a produced artefact rather than a quiz question: it is either there or it is not, which makes it fast to mark and hard to bluff. Every tool in ctfpal is deterministic, so two students who did the work correctly hand in the same value.

Where the room gets stuck

The ethics half is not an aside and should be taught first, not last. OSINT technique transfers directly to real people who did not consent to being investigated, and a class that learns the pivot loop without the line learns half a skill.

• Searching instead of pivoting. One good identifier - a username, a licence plate, a certificate serial - is worth more than an hour of rephrased queries.
• Trusting a stripped EXIF as proof of nothing. The route matters: a platform upload strips GPS, a direct file share does not, and the thumbnail often outlives the edit.
• Recording conclusions rather than sources. A pivot chain nobody can retrace is not intelligence, and in a real engagement it is not admissible either.
• Only ever adding evidence for the first hypothesis. A pivot chain that never tries to falsify itself identifies the wrong person eventually, so name what would rule the theory out before collecting more that confirms it.
• Probing a target for something the public record already answers. Certificate transparency, archives and registries cost nothing and leave no trace; an unauthorised scan does both, and it is the step that turns research into an offence.

If a student wants the subject, not the answer

Chapter-level references, so a student can be pointed at twenty pages rather than at a book. Nothing here is required to complete the module.

• Hacks, Leaks, and Revelations, Micah Lee. Chapter 2, Acquiring Datasets. The provenance and safety questions to settle before touching a dataset, which is the part CTF practice never forces you to ask.
• Hacks, Leaks, and Revelations, Micah Lee. Chapter 5, Docker, Aleph, and Making Datasets Searchable. Turns a pile of files into something queryable, which is the difference between reading a leak and working one.
• Bug Bounty Bootcamp, Vickie Li. Chapter 5, Web Hacking Reconnaissance. The same loop pointed at an organisation's infrastructure, with the enumeration sources named.

If you finish early

• Hand out a challenge from the cross-CTF index in this category - each one has published solutions to compare afterwards.
• Run the same input through Identify and let the class argue with the ranking. Disagreeing with a confidence score is where the technique actually lands.
• Ask a student to break their own example - construct an input that defeats the tool, and explain why.