The LLM didn't truncate. It edited.

A capture came back four fifths shorter than what was sent, and nothing reported it. It wasn't a token limit and it wasn't summarisation — the model was filtering to fit the schema. Why a structured-output schema is a filter as much as a contract.

The short version

A long message forwarded into Dhaga over WhatsApp was stored as a note about a fifth of its length. The obvious hypothesis was a token limit — a response cut off mid-flight. It wasn't: the call spent 142 output tokens against a 2,048 cap. It wasn't summarisation either. The model had been handed a schema with a free-text body field sitting inside a per-person object, so it kept the clauses that were about that person and dropped the rest — the sender's own thinking, which the plan's shape offered nowhere to put.

The fix was not a bigger cap and not a better prompt. We deleted the field. The plan now says only which messages a note is made of; code joins those messages' own text. Copying a message into a note is a deterministic transform, and deterministic transforms belong in code.

The rest of this post is the deep dive: how the loss stayed invisible, why filtering and truncation are different failures that look identical from outside, and the schema shape that made it structurally impossible. File paths refer to the real code.

The failure, and the thing that hid it

Dhaga can capture over WhatsApp and Telegram. Messages accumulate into a session, and one model call turns that session into a plan: which people are involved, which messages belong to whom, which messages are notes, which are instructions addressed to the bot rather than content to store. The plan is a structured output — a Zod schema is compiled to a JSON schema and handed to the model, so what comes back is guaranteed to match the shape.

A user forwarded a long message. What got stored was two sentences.

The genuinely alarming part is the second half: nothing reported it. The pipeline has an accounting invariant — every message sequence number it was given must appear somewhere in the plan, so nothing is silently dropped. It did appear. So a message four fifths discarded passed accounting cleanly, and the bot replied with a cheerful "Added 1 note".

The monitoring was structurally correct and completely blind, because it was counting references, not content. The message was accounted for. What was inside it was not.

Ruling out the obvious cause

Every engineer's first guess here is the token cap, and it's a good guess. Our client sets max_tokens to 2,048 by default. The call in question spent 142 output tokens. There was no ceiling anywhere near it.

There's a second, structural reason truncation was never the answer, and it's worth stating because it generalises. A structured-output response that runs out of tokens doesn't parse. It stops mid-object, fails the schema, and throws in the extraction layer. Ours parsed perfectly. It was well-formed, valid, confident — and short.

That's the signature to learn. Truncation produces broken output. Filtering produces correct output with less in it.

The schema was doing the filtering

Here's the shape that caused it, reduced to the part that matters:

// Before. The plan carried a body the model composed.
const plannedNoteSchema = z.object({
  body: z.string().describe(
    "The note to store — the user's own words, lightly cleaned and joined " +
      "into readable prose. Never invented, never summarised away.",
  ),
  sourceItemSeqs: z.array(z.number().int()),
});

Read that field description again. It already begged the model not to summarise. The prompt said the same thing in its own words. Both were ignored, and not because the model was disobedient.

plannedNoteSchema sat inside a per-person object. So the field's implied subject — never written down anywhere, but unmistakable from its position — was "the note about this person". Given a message where most lines were the sender thinking out loud and a couple named the person, the model did the reasonable thing: it wrote the note about that person. The other lines weren't wrong, weren't unimportant, and weren't over any limit. They just had nowhere in the shape to go.

It helps to name the three failures separately, because teams routinely mistake one for another:

Failure	What the output looks like	How you find it
Truncation	Malformed — stops mid-object, fails the schema	Loudly, at parse time
Summarisation	Shorter, but the content is represented	Reading it and noticing compression
Filtering	Well-formed, confident, and missing whole ideas	Only by comparing against the input

Filtering is the dangerous one precisely because it produces a clean result. The model isn't failing. It's succeeding at a slightly different task than the one you meant to give it, and the schema is what redefined the task.

The fix: take the wording away from the model

The new note schema has no body field at all:

// After. The plan chooses which messages. It never writes text.
export const plannedNoteSchema = z.object({
  sourceItemSeqs: z.array(z.number().int()).describe(
    "The seq numbers of the messages this note is made of. " +
      "THEIR TEXT IS THE NOTE — you are not writing it.",
  ),
  directives: z.array(z.string()).describe(
    "Substrings of those messages addressed to YOU rather than meant to be " +
      "recorded. Copy each one EXACTLY as it appears, character for character.",
  ),
});

The body is assembled in code, in apps/web/src/lib/messaging/process-session/apply/note-body.ts:

sourceTextsFor() dedupes the sequence numbers and sorts them numerically, so the note reads in send order. The model returns seqs in whatever order it reasoned about them; that ordering is not a decision about the user's text, so code imposes the real one.
assembleNoteBody() joins those texts, then removes each quoted directive by exact first-occurrence match. Whitespace tidying only — nothing is re-wrapped, re-cased or re-punctuated, because that is editing, and editing is exactly what this file took away from the model.

The subtraction rule is deliberately brittle in the safe direction. The only thing that can be removed from a note is a string the plan quoted character for character. If the quote doesn't match, the whole message is kept and the reply says so. Matching loosely would eventually cut a sentence the sender actually wrote — and a stray line the user can delete beats a paragraph they never learn went missing.

The prompt was rewritten to match, in capitals, because the old one had been politely asking for something the shape made impossible:

YOU DO NOT WRITE NOTES. A note has no text field for you to fill: you list the message seqs it is made of. So the only question a note asks you is WHICH MESSAGES, never which parts of them.

Photos and voice notes ride the same path, since the vision and transcription passes already put their text where the assembler can find it.

Why this is a schema bug, not a prompt bug

Our engineering rulebook has a line that this incident turned from a preference into a law: use the model for judgment, not for deterministic transforms. If code can answer, code answers.

Deciding which messages belong together, and who they're about, and which sentence is an instruction to the bot — those are judgment. Copying a string from one place to another is not. We were asking a probabilistic system to perform a join, and paying for it with silent data loss.

The generalisable check is a single question to ask of every free-text field in a structured-output schema:

What happens to the input that doesn't fit here?

If the answer is "it goes in this other field", fine. If the answer is "nothing — it's dropped, and nothing downstream will notice", you have found a bug, and no amount of prompt wording will fix it. A schema is not just a contract about the output's type. It is a statement about what the output is allowed to contain, and everything outside that statement is discarded by construction.

The second, subtler half: every free-text field has an implied subject, and that subject comes from its position in the object as much as from its description. A body field nested under a person means "about the person", whatever the description says. Nesting is instruction.

Pinning it with tests that could actually fail

The regression tests live in apps/web/src/lib/__tests__/messaging-cases/verbatim-notes.test.ts, and they assert whole-string equality between the message pushed in and the note stored. There's a comment in the file explaining why, which is the best short argument for strict assertions I've written down:

Asserting the WHOLE string is the point — a length check, or a toContain on one line, would have gone green throughout the period this was broken.

That's exactly right. A toContain test looking for one line would have passed the entire time the bug was live, because the surviving fifth contained the line the test was looking for. Partial assertions are precisely blind to filtering.

The fixtures are synthetic, and deliberately so — they reproduce the structure of the reported capture (most lines being the sender's own thinking, a couple naming the person) without any of its content.

The takeaways

Truncation and filtering look identical from the outside and have opposite fixes. Truncation is a limit problem; filtering is a shape problem. Check the actual token spend before you raise a cap — ours was 142 against 2,048.
A structured-output schema is a filter as much as a contract. For every field, ask what happens to the input that doesn't fit. "It's dropped silently" is a bug you cannot prompt your way out of.
Every free-text field has an implied subject, set by where it sits. A body nested inside a person object means "about this person" no matter what the description claims.
If a transform is deterministic, don't ask a model to do it. Joining strings in order is a join, not a judgment call. Give the model the judgment and keep the wording.
Accounting that counts references is blind to content. "Every input was referenced somewhere" is a much weaker invariant than it sounds, and it will show you a green light over a four-fifths loss.
Assert whole strings in tests for anything a model touched. Partial assertions pass straight through the exact failure mode you're most likely to hit.