Spot the encoding: reading base64, base32, hex and friends at a glance

Encodings are not encryption. They keep no secret; they exist to move bytes through a channel that only tolerates certain characters. That makes them the connective tissue of CTF challenges - the thing wrapped around the actual puzzle - and being able to name one on sight is the highest-leverage reflex a beginner can build.

Three properties are enough to identify nearly everything you will meet: the alphabet (which characters appear), the length (and what it is a multiple of), and the padding (what shows up at the end).

The identification table

Padding tells you the remainder, not the format

Base64 encodes three bytes into four characters. When the input length is not a multiple of three, the encoder emits = to fill the final quartet: one `=` means the last group held two bytes, two `=` means it held one, and no = means the input divided evenly. Base32 works the same way with five bytes into eight characters, which is why base32 output can end in as many as six = signs.

Two practical consequences. First, = in the middle of a string means you are not looking at one base64 blob - you are looking at several concatenated, or at a key-value string like user=YWRtaW4=. Second, missing padding is not corruption. Plenty of encoders strip it, and every sane decoder can recover it: append = until the length is a multiple of four.

Magic prefixes: knowing what is inside before you decode

Because base64 is deterministic, a known file header always produces the same leading characters. Learning a handful of them lets you read the payload type straight off the encoded string, which is genuinely useful when you are staring at a 200 KB blob and deciding whether it is worth saving to disk.

Is it text or is it bytes?

A base64 string that decodes to compressed or encrypted data looks statistically flat: byte values spread evenly across 0-255, no runs, no structure. A base64 string that decodes to text is visibly lumpy even in encoded form, because the ASCII range maps to a narrow slice of the alphabet.

This matters when a decode produces garbage. Flat, high-entropy garbage means you decoded correctly and the payload itself is compressed or encrypted - keep going, look for a gzip or ZIP header in the first bytes. Garbage with obvious ASCII fragments in it means your decode was *nearly* right: wrong variant, wrong offset, or an extra layer.

import base64, collections, math

def entropy(b: bytes) -> float:
    if not b:
        return 0.0
    counts = collections.Counter(b)
    n = len(b)
    return -sum((c / n) * math.log2(c / n) for c in counts.values())

raw = base64.b64decode(blob + "=" * (-len(blob) % 4))
print(raw[:16].hex(), f"{entropy(raw):.2f} bits/byte")
# ~4.2  -> English text
# ~6.0  -> mixed / structured binary
# ~7.9  -> compressed or encrypted; look for a container header

The traps

Hex that is secretly ASCII

A hex string whose bytes all land in 20-7e decodes to text, but a hex string of *only* 30-39 and 41-46 decodes to more hex. Challenges lean on this: hex of hex of hex, three layers deep, each one still passing a naive hex check. If your decode output is itself valid hex, decode again.

Base64 that survives ROT-13

ROT-13 applied to base64 produces valid base64 that decodes to garbage. Same for ROT-47 over the printable range. When a string looks perfectly base64 and decodes to nothing readable, try rotating the string *before* decoding rather than after.

Zero-width characters

If a challenge gives you text that looks completely ordinary and the challenge name mentions something invisible, check for zero-width joiners and non-joiners (U+200B, U+200C, U+200D, U+FEFF). They carry a binary payload that is invisible in every renderer and survives copy-paste. Count characters versus visible glyphs; a mismatch is the whole answer.

Uuencode, XXencode, and other fossils

A block that starts with begin 644 filename and whose lines all begin with the same character is uuencoded - a pre-base64 format that still shows up in email-themed challenges. Its cousin XXencode uses +-0-9A-Za-z and is rarer still. Neither is hard to decode once named, which is the entire point of learning to name things.

Build the reflex

Spend one session encoding the same short string every way you can and looking at the results side by side. flag{practice} as hex, base64, base32, base85, binary, decimal, URL-encoded, and Morse. Ten minutes of that does more for your recognition speed than reading any table, this one included - the alphabets stop being facts you recall and start being shapes you see.