Every article our AI blogger writes goes through seven automated checks before a person sees the publish button. If any check fails, the draft is held and the owner gets the reason. Nothing publishes on its own.
We counted what those checks actually stopped. The sample is small and most of it comes from our own blog, so read the numbers as one team's log, not an industry benchmark.
The short version
- 64 drafts between September 13 and November 1, 2026. 28 of them (44%) failed at least one check.
- The fact check stopped 17 drafts. No other check came close.
- Inside those 17, the most common problem was a rule written as if it had no exceptions (13 drafts). The second was a claim about the business itself that the brief never confirmed (9 drafts).
- Humans later approved 18 of the 28 held drafts and rejected 5. 5 are still waiting.
What we measured
The sample is every article draft in our production database that has check results attached: 64 drafts from 8 workspaces. One workspace wrote 40 of the 64, and that workspace is our own blog. The other 24 come from 7 workspaces. We publish counts only, no workspace names and no client text.
The seven checks are brand rules, facts, search basics, content policy, originality against the workspace's own archive, machine readability and reading level. Two of them, facts and policy, are judged by a language model against the client's confirmed brief. The other five are code.
The numbers come from one script, scripts/qa-research.mts in our repository. It reads a fixed window, so running it again gives the same output.
Which checks stopped drafts
| Check | Drafts checked | Drafts failed |
|---|---|---|
| Facts | 64 | 17 |
| Search basics | 64 | 9 |
| Content policy | 64 | 6 |
| Brand rules | 64 | 3 |
| Reading level | 13 | 1 |
| Machine readability | 64 | 0 |
| Originality | 64 | 0 |
For search basics and machine readability, part of the check had no input on 34 and 25 drafts, usually because no target query was set. The reading-level check was added later than the others and ran on 13 drafts. 22 of the 28 held drafts failed exactly one check, 4 failed two and 2 failed three.
What the fact check found
The fact judge tags each problem it finds with a class. A single draft can carry several tags, so these add up to more than 17.
- A rule stated as universal: 13 of 17. One of our own drafts said "A social post cannot repair an unclear offer." Usually true, but not always, and a reader who knows a counterexample stops trusting the rest. Another said every social media agency also runs paid campaigns. That one is false.
- An unconfirmed fact about the business: 9 of 17. The draft describes a feature, a product or a promise that the brief does not contain. These read well, which is the problem. A model fills a gap in the brief with something plausible.
- Health or safety advice: 1 of 17.
- Other: 11 of 17. Everything the judge could not place in a named class, from a misread source to a technical claim it could not verify.
Invented business facts are the class we care about most. A wrong general claim embarrasses the writer. A wrong claim about what the business sells becomes a promise to a customer.
What the other checks found
Search basics failed 9 times. In the held drafts we can classify, the target query was missing from every H2 three times, the core answer was repeated several times in the text twice, and an internal link anchor broke the rule twice. In our view all three belong in the writing prompt, not in a check after the fact.
Content policy failed 6 times and brand rules 3 times. The policy judge tagged 2 of its 6 as an unconfirmed business fact and 2 as a universal claim, the same two classes the fact check finds most.
Is the fact judge stable?
A model judging a model is only useful if it gives the same answer twice. We keep a set of 9 test articles with known answers: an invented source, a misquoted source, a claim about the business that is not in the brief, a superlative about the product, and a few that should pass. The current judge got 27 of 27 decisions right over three runs each. The previous version got 20 of 24.
On 8 real drafts judged three times each, the current judge held 15 of 24 verdicts, and on 5 of the 8 drafts it did not give the same answer all three times. The judge is reliable on clear cases and wavers on borderline ones. That is why a held draft goes to a person instead of being thrown away.
What a hold is not
A hold is not proof the draft was wrong. People approved 18 of the 28 held drafts. We cannot tell from the data how many of those were edited first, so we do not claim the checks were right 10 times or wrong 18 times. What we can say is that a person read the reason before anything went live.
Limits of this data
- 64 drafts is a small sample. A single bad week moves the percentages by several points.
- Most drafts come from one workspace, our own, writing about marketing. Among the 17 drafts held for facts, 1 was tagged as health advice, too few to say how a clinic or a law firm would fare.
- The prompts and the judge changed during the window. This is a log of a system being built, not a controlled experiment.
- The class tags come from the judge itself. We did not hand-label them.
What we take from it
If you use AI to write for a business, the risk is not grammar. It is the confident sentence about your own company that you never said, and the rule with no exceptions that a knowledgeable reader knows is false. Those two classes made up most of what our fact check stopped. Read for them first.