Why a good prompt isn’t enough to judge an email.
“Ask the model if this sounds pushy” can look convincing in a demo. A production gate needs evidence you can inspect and decisions your workflow can rely on.
Three structural flaws of unconstrained prompts.
A prompt can describe an excellent standard and still leave the whole decision inside a generative response. That is a fragile boundary when the output determines whether an automated system sends a real email. Three failure modes explain why.
1. Inherent sycophancy.
A conversational model is often inclined to be helpful to the person asking. If a writer asks whether its persuasive draft is good, the judge may mirror that framing, praise the effort or accept the implied goal instead of examining the reader’s cost.
This is not a claim that every model always agrees. It is a structural risk of using an unconstrained conversational answer as an independent permission gate. Separating writing from auditing reduces that conflict: the audit has a defined job, named boundaries and an output that must survive validation.
2. Run-to-run drift.
Language-model generation is not a fixed lookup table. Ambiguous wording can produce different candidate issues or severity assessments on different runs. A free-form “looks fine” today and “too pushy” tomorrow is hard to build a safe workflow around.
Code cannot make semantic extraction perfectly invariant. It can make the mapping from validated issues to the final verdict invariant. The same retained severity set must produce the same PASS, REVIEW or FAIL; the model does not get to override that mapping with an optimistic conclusion.
3. Hallucinated citations.
A plausible criticism can come with an implausible quote. A model may paraphrase, combine sentences or cite words that were never in the draft. In an audit, that is worse than an unhelpful explanation: it creates false evidence against your own message.
Every retained snippet must occur verbatim in the supplied email. Unsupported snippets are dropped before the final verdict is calculated. Verbatim validation establishes that the words exist; it does not by itself prove that the interpretation is correct. That remains a contextual judgment the reader can inspect.
An extraction layer, then an enforcement layer.
The model reads for meaning and proposes candidate issues: boundary, severity, exact snippet, explanation and repair. The enforcement layer checks those quotes against the original and derives the final verdict from the severities that remain.
No retained issues means PASS. Retained low or medium issues mean REVIEW. A retained high-severity issue means FAIL. The output’s friendly wording cannot overrule the severity calculation. This is a deterministic verdict calculation, not a claim that all model judgments are deterministic.
email + supplied context │ ▼ [ semantic extraction: candidate issues ] │ ▼ [ code: verify verbatim evidence ] │ ▼ [ code: derive verdict from retained severities ] │ ▼ PASS · REVIEW · FAIL + repair when available
Tiered evaluation for speed.
Not every clean draft needs the same depth of generative review. A lighter first stage screens against the same boundary framework. Only clearly clean messages skip the deeper audit; uncertain or flagged drafts continue to the more detailed evaluation.
That saves cost and latency without deliberately changing the standard between fast and deep paths. Identical repeat checks can also return the previous result from a brief memory-only cache without using another credit. Neither mechanism requires a permanent store of your email text.
These are the public architectural principles. Internal system prompts, proprietary weights and operational thresholds are not part of the public handbook.
The boundary stability stress test.
Our reported internal boundary-stability exercise used 28 ambiguous drafts and observed zero flips directly between PASS and FAIL across the repeated runs in that exercise. It asks a practical question: does a borderline message jump between “no retained issues” and “human intervention required”?
That is a bounded internal observation, not external validation or a universal zero-variance guarantee. REVIEW/FAIL movement is a separate issue, and an ambiguous draft can still receive different extracted issues on a future run. Stable code cannot eliminate all uncertainty in interpreting human language.
The useful claim is narrower: evidence is checkable, verdict logic is explicit, and instability can be measured instead of hidden behind a confident paragraph. Treat a stress test as evidence about the tested set, not permission to stop monitoring the system.