The tests agreed with the bug

31 July 2026 · Testing 7 min read

Telechat has a large test suite, a coverage floor in CI, and three-and-a-half thousand passing tests. This week three bugs turned up in the code that formats every reply the bot sends. All three were covered. All three tests passed, before and after the fix.

The bugs are small. What they have in common is worth more than the fixes.

1. Every heading came out italic

Telegram's MarkdownV2 uses *text* for bold and _text_ for italic. Claude writes ordinary markdown, so Telechat rewrites it. The heading rule was one line:

# Convert headings to bold
result = _HEADING_RE.sub(lambda m: f"*{m.group(1).strip()}*", result)

Three passes later, the italic rule runs, and it matches a single asterisk not adjacent to another single asterisk. Which is exactly what the heading rule had just produced. Every heading became italic.

The comment says bold. The code says bold, locally. The pipeline says italic. This is the failure mode of any multi-pass text transform: each pass is correct in isolation, and the composition is not — a pass cannot tell the difference between a * the author wrote and a * an earlier pass emitted.

The fix is not a better regex, it is to stop putting a marker into text that later passes will read. The heading now goes into the same placeholder as **bold** — a sentinel no rule matches — and the asterisks are added at the very end.

2. Blockquotes rendered as a literal >

Same shape, opposite direction. The blockquote rule inserted a >, and then the escaper ran:

result = _BLOCKQUOTE_RE.sub(lambda m: f">{m.group(1)}", result)
...
result = escape_md2(result)   # '>' is on the escape list

MarkdownV2 requires > to be escaped when it is literal text — if a > b must ship as if a \> b or the message is rejected. So the escaper escapes it. Including the one the blockquote rule had just added, half a dozen lines earlier. Every quote shipped as \>quoted text, which Telegram renders as a greater-than sign followed by the text.

The same fix applies: stash the marker behind a sentinel, escape everything, put the marker back. Escaping and markup have to be separated in time, not merely in code.

3. Long replies were split three times too often

The chunker splits a reply that exceeds the message limit, and it prefers a paragraph boundary — but not one too close to the start, or you get a two-line message followed by a wall:

for match in re.finditer(r"\n\s*\n", text):
    pos = match.end()
    if pos > len(text) * 0.3 and not _is_inside_fence(...):
        return pos

Read it as a sentence and it is right: break at a paragraph boundary at least 30% in. Read it as a loop and it returns the first boundary past 30% — for a reply made of ordinary short paragraphs, the one at roughly 31%.

Measured on 7 800 characters of prose against a 4 000-character limit: five messages, sized 1 233, 1 233, 1 233, 1 233, 2 860. It should be two. Every user of every platform had been getting two to three times more notifications than necessary, for as long as the function had existed.

Three of the four rules had this shape. The fourth used rfind and was correct, which is probably why nobody looked twice at the others.

The same rule had a second bug. It could break on the ``` that closes a code block — the fence regex cannot tell an opening fence from a closing one — leaving one message holding an unterminated block and the next starting on a stray fence. Under MarkdownV2 that can fail to parse, and the whole message falls back to plain text.

What the tests said

Here is the part worth the post. This test existed, and passed, the whole time:

def test_heading_converted(self):
    result = to_markdown_v2("# My Heading")
    # Headings are converted to bold (*text*), which then passes
    # through the italic-capture pass, so the final output wraps in _ or *
    assert "My Heading" in result
    assert result.strip() != "My Heading"

Someone hit the bug, understood it precisely enough to write it down, and then wrote an assertion that accepts it. The comment is a correct description of a defect, sitting above a test that ratifies it. The blockquote test was the same: assert "quoted text" in result — true of a blockquote, and equally true of the escaped literal that shipped instead.

The chunker's tests failed differently. They were not wrong, they were vacant:

chunks = chunk_text(text, limit=30, mode="smart")
assert len(chunks) >= 2

>= 2 is true of the correct output and true of a chunker that emits one character per message. The assertion is about the function running, not about what it produced.

The tell

All three share one property: the assertion cannot distinguish the fix from the bug. That is checkable without knowing anything about the domain. Take your assertion, imagine the behaviour you are worried about, and ask whether it would still pass. If it would, the test is measuring that the code ran.

Three habits fall out of it, and they are what the replacements do:

• Assert the output, not a property of the output. == "*My Heading*" rather than "My Heading" in result. Exact equality is unfashionable because it is brittle — but brittle is the point for a pure function whose whole job is producing an exact string. When it breaks, it breaks loudly and you look.
• Assert the number that matters. The chunker's tests now say a 7 800-character reply is two messages and that non-final chunks are over 3 000 characters. That is the behaviour users experience, so it is the thing to pin.
• When you write a comment explaining surprising behaviour, stop. Understanding a defect well enough to describe it is understanding it well enough to file it. A comment is not a place to put a bug you have decided to live with — that is what an issue tracker is for, and what a failing test that you mark xfail is for.

There is a fourth, harder one. The coverage number was never the problem: all three functions were covered, and coverage is measured by lines executed, not by claims verified. Formatting bugs in particular are invisible to a coverage floor and invisible in review — \>quoted text looks fine in a diff and looks wrong only in a Telegram client. The only thing that finds them is somebody reading the output as a user, and the only thing that keeps them fixed is an assertion sharp enough to fail.