Skip to content
All posts
miscAugust 11, 20268 min read

The first ten minutes: a triage playbook for any CTF challenge

Most challenges are lost to flailing, not to difficulty. Here is a repeatable order of operations for an unknown blob, an unknown file, and an unknown service - and the point at which you should stop guessing and start reading.

Every CTF gives you the same thing: a small artifact and no context. A base64 blob. A 40 KB PNG. A netcat address and a source file. The difference between people who solve five of these and people who solve fifteen is almost never cleverness. It is that the second group has an order of operations and does not deviate from it.

This post is that order of operations. It is deliberately boring. Boring is the point: you want the first ten minutes to be reflex, so your attention is free for the part that actually needs thinking.

Rule zero: identify before you attack

The single most common failure mode is attacking the wrong thing. Someone spends twenty minutes on a Vigenère solver against a string that was hex-encoded ciphertext, or runs a stego tool against a file that was never an image. Identification is cheap and attacks are expensive, so always spend the cheap resource first.

Concretely, before you type a single attack, you should be able to answer three questions: what *format* is this, what *category* is this (crypto, forensics, web, pwn, rev), and what is the *smallest* transformation that would produce something more readable than what you have now. If you cannot answer all three, keep identifying.

Path A: you were given text

Text is the most common starting point, and it is the one where a fixed checklist pays off the most. Run these in order.

  1. Look at the character set. The set of distinct characters is the strongest single signal you have. Only 0-9a-f means hex. Uppercase letters plus digits 2-7 with = padding means base32. Mixed case, digits, +/, and a length that is a multiple of four means base64. Only two distinct characters means binary or a substitution over a two-symbol alphabet.
  2. Look at the length. Thirty-two hex characters is an MD5. Forty is SHA-1. Sixty-four is SHA-256. A length that is not a multiple of anything interesting is a hint that you are looking at raw text, not a digest.
  3. Try the cheap decodes. Base64, hex, base32, URL, HTML entities, ROT-13. Any of these takes under a second, and a wrong guess costs you nothing because the output will be obvious garbage.
  4. Check for structure that survived encoding. Three dot-separated base64url segments is a JWT. A leading gAAAAA is a Fernet token. -----BEGIN is a PEM key. {" after a base64 decode is JSON. PK\x03\x04 is a ZIP.
  5. Only then reach for cryptanalysis. If it is plausibly English-shaped ciphertext, now you run frequency analysis, chi-squared, and index of coincidence.

A detail that trips up beginners: encodings nest. It is completely normal for a challenge to be base64 of hex of a ROT-13 of the flag. If a decode produces something that still looks encoded, decode again. A cascade of three is common; a cascade of six is a themed joke challenge and also common.

Path B: you were given a file

For files, ignore the extension entirely. Extensions are a naming convention, not a fact about the bytes. The first few bytes are the fact.

First bytesFormatWhat it usually means in a CTF
89 50 4E 47PNGImage stego, chunk tampering, or a corrupted header/IHDR
FF D8 FFJPEGEXIF data, appended payload, or a JPEG-specific stego tool
50 4B 03 04ZIP (also DOCX, XLSX, JAR, APK)Nested archive, password crack, or an Office document with macros
7F 45 4C 46ELFReverse engineering or binary exploitation
4D 5APE/EXEWindows reversing or malware triage
1F 8BgzipUnwrap it; usually another layer underneath
D4 C3 B2 A1 / 0A 0D 0D 0Apcap / pcapngNetwork forensics
25 50 44 46PDFEmbedded objects, JavaScript, or hidden text layers
The magic-byte table you will use most. file knows all of these; so does the File-mode fingerprinter.

After identification, the standard sweep for any file is: hash it, look for strings, look for *appended* data past the logical end of the format, and check the metadata. Appended data is worth calling out because it is the single most common file trick in beginner and intermediate challenges - a valid image followed by a whole ZIP archive that no image viewer will ever show you.

# The classic four, in the order they pay off
file suspicious.png
strings -n 8 suspicious.png | less
binwalk -e suspicious.png        # carve embedded/appended files
exiftool suspicious.png          # metadata, comments, GPS
If you have a shell, this is the sweep. If you do not, File mode runs the same four checks in the browser and carves what it finds.

Path C: you were given a service or a URL

Here the first ten minutes are about mapping, not exploiting. For a web target: view source, read every comment, check /robots.txt, look at the cookies, look at the response headers, and note what technology stack you are on. For a netcat service: connect, send nothing, and read what it says. Then send something obviously wrong and read what it says about *that*. Error messages are the cheapest source of information in the game.

If a source file was provided, read it before you touch the service. A provided source file is the author telling you where the bug is; declining to read it and fuzzing blindly instead is a choice to do the challenge on hard mode.

The stop-guessing threshold

Set a timer at fifteen minutes. If you have not identified the challenge by then - not solved it, *identified* it - stop attacking and go read. Read the challenge description again word by word. Read the source. Read the documentation for whatever library the source imports. Look at the file in a hex editor rather than through a tool’s summary of it.

The reason for the threshold is that flailing has a seductive property: it feels like progress because you are typing. Fifteen minutes of unstructured attempts against an unidentified artifact almost never converts into a solve, whereas five minutes of reading usually does.

Keep a log while you flail

Write down each thing you tried and what came back, even one word - b64: garbage, rot13: garbage, xor 0x20: partial ASCII. Two things happen. First, you stop repeating attempts, which is a bigger time sink than anyone admits. Second, partial ASCII is a lead you would otherwise have scrolled past, and leads are what you will hand to a teammate when you rotate off the challenge.

Know what the flag looks like

Almost every event uses a flag format like picoCTF{...}, flag{...}, or CTF{...}, and states it in the rules. Learn it at the start of the event, because it converts a hard question - is this output right? - into a trivial one. It also means you can grep for it: through strings output, through every decode candidate, through every stego extraction, all at once.

This is why automated solvers work as well as they do. They are not clever. They apply forty cheap transformations and grep the results for a known pattern. You can do exactly the same thing by hand, and when you are stuck, doing it by hand on a wider set of transformations is often the move.

The whole playbook, on one screen

  1. Read the title, the description, and the point value. Note the flag format.
  2. Identify the artifact: character set and length for text, magic bytes for files, banner and headers for services.
  3. Run the cheap transformations. Decode until the output stops looking encoded.
  4. Grep everything you produce for the flag prefix.
  5. If a source file exists, read all of it before attacking.
  6. Log every attempt in one line.
  7. At fifteen minutes with no identification, stop attacking and start reading.
  8. At forty minutes with no progress, write down your leads and rotate to another challenge. Come back with fresh eyes.

None of this is the interesting part of a CTF. That is exactly why it should be automatic - the interesting part deserves your whole attention, and it will not get it if you are still deciding whether to try base32.

Further reading

methodologytriagebeginnerencodingfile-identification

Related posts