Hiding in text: zero-width characters, homoglyphs, and whitespace

Text steganography works because rendering discards information. A browser shows you glyphs; the bytes behind them can carry characters that occupy no space, characters that look identical to other characters, and runs of whitespace that a renderer collapses. None of it survives a screenshot, and all of it survives a copy and paste - which is exactly the property a challenge wants.

The general rule for this whole class: stop looking at the text and look at the bytes. Every technique here is invisible by construction and obvious in a hex view.

The first move, always

# What is actually in this file?
xxd chal.txt | head -40
file chal.txt                                # encoding guess
wc -c chal.txt && wc -m chal.txt             # bytes vs characters: a gap means multibyte

# List every non-ASCII codepoint with its name.
python3 -c "
import sys, unicodedata
for i, ch in enumerate(open('chal.txt', encoding='utf-8').read()):
    if ord(ch) > 127:
        print(i, hex(ord(ch)), unicodedata.name(ch, '?'))"

Zero-width characters

Unicode defines several characters with no visible width. They exist for legitimate typographic reasons - controlling ligatures, joining scripts - and they pass through most systems unmolested.

Decoding is a substitution followed by a bit reassembly. The only real question is which character maps to which bit, and there are two possibilities - try both.

text = open("chal.txt", encoding="utf-8").read()
MAP = {"\u200b": "0", "\u200c": "1"}          # try the reverse if this gives noise

bits = "".join(MAP[c] for c in text if c in MAP)
print(len(bits), "bits")

for width in (7, 8):                           # 7-bit ASCII is common in these
    out = bytes(int(bits[i:i+width], 2) for i in range(0, len(bits) - width + 1, width))
    print(width, out[:120])

Variants worth knowing: some encoders use four zero-width characters to carry two bits each, and some interleave the payload only between words rather than throughout. Neither changes the method - list the codepoints, count the distinct symbols, and the base follows from the count.

Homoglyphs

Many scripts contain characters that render identically to Latin letters. Cyrillic а (U+0430) and Latin a (U+0061) are indistinguishable in most fonts; so are Greek ο, Cherokee Ꭺ, and a long list of mathematical alphanumeric variants.

As steganography, the substitution *is* the bit: a Latin a is 0 and a Cyrillic а is 1, at each position where the letter is one of the substitutable set. As an attack, the same property is the basis of IDN homograph domains and of filter bypasses where a check compares against a Latin string and the runtime later normalises to it.

import unicodedata

text = open("chal.txt", encoding="utf-8").read()

# Which characters are not in the script you expect?
for ch in sorted(set(text)):
    if ord(ch) > 127:
        print(hex(ord(ch)), repr(ch), unicodedata.name(ch, "?"))

# NFKC folds many lookalikes toward their ASCII form. A diff against the
# original shows exactly which positions were substituted.
folded = unicodedata.normalize("NFKC", text)
bits = "".join("1" if a != b else "0" for a, b in zip(text, folded))

Bidirectional controls

Unicode's bidirectional algorithm includes override characters that reverse rendering order. U+202E (RIGHT-TO-LEFT OVERRIDE) is the well-known one: a filename exe.txt preceded by it renders as txt.exe, and the historic use was making an executable look like a document.

In a challenge these appear two ways. As stego, the presence or absence of a control character carries a bit. As a puzzle, the visible text is a scrambled rendering of the real byte order, and the answer is to strip the controls and read what is actually stored. The related "Trojan Source" technique hides code this way: source that a reviewer reads one way and a compiler reads another.

Whitespace encodings

The oldest technique in the family, and still common. Trailing spaces and tabs at the ends of lines are invisible in every editor and carry bits directly.

# Make it visible.
cat -A chal.txt | head -20          # $ marks line ends, ^I marks tabs
grep -nP '[ \t]+$' chal.txt | head  # lines with trailing whitespace

# The classic tool that produced it.
stegsnow -C chal.txt

• Space and tab as 0 and 1, read line by line or as one stream.
• Whitespace-as-a-language: the esolang Whitespace uses space, tab and newline as its entire alphabet. A file that is mostly blank and executes is that, and it is covered in esolangs and the misc pile.
• Single versus double spaces between words encodes a bit per gap and survives copy-paste into most editors.
• Line-length parity in a formatted document - odd and even lengths as bits - which survives even a re-render.

Where these turn up outside a text file

• Inside a document. A PDF's text layer, a DOCX's XML, an email body. See document forensics for getting the text out first.
• In metadata. An EXIF comment, an ID3 tag, a PNG tEXt chunk. The image workflow surfaces those.
• In a web page. Zero-width characters in HTML source, or in a CSS content property. View source, do not read the rendering.
• In a git commit message or a filename. Both accept arbitrary UTF-8, and neither is somewhere people look.
• In an LLM's output. A watermarking or exfiltration channel in an AI challenge, where invisible characters carry instructions - the text version of the indirect injection in the prompt injection guide.

The text-steganography workflow

1. Compare byte count with character count. A gap means multibyte content.
2. List every non-ASCII codepoint with its Unicode name. The names identify the technique.
3. Count the distinct invisible symbols: two means binary, three or four means a larger base or a separator.
4. Extract, map to bits, and try both polarities and both 7- and 8-bit widths.
5. If the codepoints are all letters rather than controls, it is homoglyphs - diff against a confusables table.
6. If there are no non-ASCII characters at all, look at whitespace: cat -A, trailing runs, and stegsnow.
7. If none of that produces anything, the text is a carrier for something else and the payload is in the file's structure rather than its characters.

The reason this class rewards a checklist is that every technique in it is invisible and several look identical from the outside. You are not searching for a hidden message by reading harder; you are running four mechanical tests, and one of them comes back positive.