Hash length extension: appending to a message you cannot read

A length extension attack lets you take a hash you were given, append data of your choosing, and produce the correct hash of the extended message - without knowing the original message or the secret prefixed to it. It is the reason H(secret || data) is not a MAC, and it appears in CTFs every time an author writes an authentication token by hand instead of using HMAC.

The attack is not a weakness in the compression function. MD5, SHA-1, SHA-256 and SHA-512 are all vulnerable to it despite MD5 and SHA-1 being broken for collisions in ways SHA-256 is not. It is a property of the *construction* those hashes share, and understanding the construction is the whole post.

Why it works: Merkle-Damgård in one diagram

A Merkle-Damgård hash processes the message in fixed-size blocks, carrying a state forward:

state = IV
for block in pad(message):
    state = compress(state, block)
return state            # <- this IS the digest, unmodified

Two facts do all the work. First, the padding is a deterministic function of the message *length* alone: a 0x80 byte, then zero bytes, then the bit length as a big-endian 64-bit integer, filling out to a block boundary. You can compute it for a message you have never seen, provided you know how long it was. Second, the final internal state is published as the digest. There is no finalisation step that destroys it.

So if someone gives you H(secret || message), they have handed you the exact internal state the hash function was in after it consumed secret || message || padding. Load that state back in, keep feeding blocks, and you are continuing a computation you were never authorised to start.

Recognising the vulnerable shape

You are looking for a signature computed as hash-of-secret-then-data, over data you partly control, where the verifier recomputes the same thing. In a web challenge it usually looks like a cookie or a query string:

GET /admin?user=guest&role=user&sig=6c1e0c... HTTP/1.1

# server side:
#   expected = sha256(SECRET + query_without_sig).hexdigest()
#   if expected != sig: 403

• The tell in the source: hashlib.sha256(secret + data), md5($secret . $data), MessageDigest.update(secret); update(data). Any concatenation where the secret comes first.
• The tell without source: a signature whose length matches a raw digest (32 hex chars for MD5, 40 for SHA-1, 64 for SHA-256) attached to data you can extend, on a parameter format that is append-friendly - a query string, a semicolon-separated cookie, a serialised blob.
• The tell that it is not this: the signature is HMAC, or the secret is appended rather than prefixed, or the digest is truncated. Any of those and the attack does not apply.

Running the attack

You need three things: the original digest, the original data, and the length of the secret. You do not need the secret's value, and you do not need to see the original message beyond the part you were already shown.

The secret length is normally unknown, and that is fine - it is a small integer. Loop it from 1 to 64, generate a forgery for each, and send all of them. One will verify, and that also tells you the secret's length for any later stage of the challenge.

# hashpump / hash_extender do the arithmetic for you.
hash_extender \
  --data 'user=guest&role=user' \
  --secret 16 \
  --append '&role=admin' \
  --signature 6c1e0c... \
  --format sha256 \
  --out-data-format=html

Doing it by hand, so you know why it worked

import struct
from sha256_pure import Sha256          # any implementation exposing its state

def md_padding(msg_len: int, block=64, endian=">") -> bytes:
    pad = b"\x80" + b"\x00" * ((block - 9 - msg_len) % block)
    return pad + struct.pack(endian + "Q", msg_len * 8)

def extend(digest_hex: str, orig_data: bytes, append: bytes, secret_len: int):
    total = secret_len + len(orig_data)
    glue  = md_padding(total)

    h = Sha256()
    h.h = list(struct.unpack(">8I", bytes.fromhex(digest_hex)))   # resume state
    h.length = (total + len(glue)) * 8                            # bytes already eaten
    h.update(append)
    return orig_data + glue + append, h.hexdigest()

Which hashes are immune, and why

The truncated-digest row is worth noticing because it also describes an accidental defence you will meet in challenges: a developer who stores md5(secret + data)[:16] has, without meaning to, made length extension impractical. When you see a signature that is *shorter* than a natural digest, this attack is off the table and you should be cracking the secret instead.

The related trick: hash collisions on the same construction

Merkle-Damgård gives you one more property CTFs use: collisions extend. If H(A) == H(B) for two equal-length blobs, then H(A || X) == H(B || X) for any suffix X, because the state after the colliding prefix is identical. That is what makes the well-known MD5 collision pairs useful - you prepend a colliding block pair to two different payloads and get two files with the same digest.

In practice this shows up as a challenge asking for two different inputs with the same MD5, or a signature check that compares digests of an uploaded file against a whitelist. The unicoll and fastcoll techniques produce these in seconds for MD5, and chosen-prefix collisions exist for SHA-1. SHA-256 has none, which is why the split between "broken for collisions" and "vulnerable to extension" matters - they are separate axes and a hash can be on either, both, or neither.

A length-extension checklist

1. Find a signature that is a bare digest over data you partly control.
2. Confirm the construction is secret-first concatenation, not HMAC and not secret-last.
3. Confirm the format tolerates appended junk - duplicate keys, trailing data, a lenient parser.
4. Generate forgeries for every secret length from 1 to 64.
5. Encode the glue bytes for the transport: percent-encoding for a URL, and check the cookie parser tolerates them.
6. Send them all; the one that verifies also tells you the secret length.
7. If the digest is truncated, or it is HMAC, or it is SHA-3 or BLAKE - it is not this attack. Go and look at how the secret is stored instead.

The reason this attack keeps appearing is that H(secret || message) looks completely reasonable if you think of a hash as a black box that mixes its input. The construction is the part that leaks, and once you have seen it once you will recognise the shape in source forever.