Small tests liked a prompt rule. A blind test killed it.
Twelve product shots, judged blind: the plain prompt won 6, the new rule won 3, and 3 were ties. The rule went, with a condition for its return.
In this story3 sections
On 22 August 2026 Scenri’s first quality battery came back: 95 images, 73 of them reviewed by five independent AI reviewers, each scoring blind against a rubric I wrote. 63 percent of first generations passed. After refinement, 22 percent did.
Its refinement path is where the product is lost
Fixes followed in versions 0.6.9 to 0.7.4. So did blind tests of new ideas for how images are generated, and some of those ideas lost.
- Images in the first battery
- 95
- Pass rate, first shot to refined
- 22%from 63%
- Plain wins, rule wins, ties in 12 cases
- 6 · 3 · 3
- Mood board vs words, so words stayed
- 4.91 / 4.91
The rule that lost
One idea was a grounding directive: an extra block of instruction in the generation prompt. It was drafted three ways. Each draft was attacked by two hostile reviewers. A first blind A/B, seven pairs of one image each, came back as noise, and the directive was reverted as unproven.
So it was tested at full size. Twelve product cases, three takes of each, once with the plain prompt and once with the directive, judged blind by AI reviewers who did not know which was which. The bar was fixed before any image existed: four or fewer wins for the directive meant a revert. The plain prompt won 6 cases, the directive won 3, and 3 were ties. Stock prose grounded better, and the directive was removed.
The decision record from 24 August gives the reason the test had to be that big.
Smaller samples had previously suggested the opposite, which is exactly the failure this rule exists to prevent.
It also sets the condition for bringing the directive back.
Do not reland it without a blind battery of at least the size that killed it
No better means no
Whole scene could have taken a mood board instead of words. It was tested on 22 pairs. The mood board read as the set no better, 4.91 of 5 both ways. So Whole scene stays words.
A line telling the model to avoid certain colours raised rust outfits from 2 of 8 to 6 of 8. It was removed.
When something won
A tester reported a sweater product worn over the reference leggings. The capture uniform was walking into finished images. After the fix, 12 cases on the fixed branch showed zero uniform leakage, while 3 of 4 controls on main still reproduced it.
Some wins came from small batteries, and the record keeps their size. A person built from three views held identity in 6 of 6, against a pass bar of five of six. Small products were drawn two to six times too big, and about 5 of 40 passed; a two-step draw took that to 7 of 7. Ring-sized products still read about twice their size, and the design doc says so.
From scratch, a place used to be asked two things, a world and a time of day. It came out as “a niche of limestone in golden-hour sun”. Every one of the 72 curated scenes had three things more. Measured on real Codex and judged blind, four answers beat two in 4 of 4 worlds, memorable 6.0 against 4.0. The place studio now asks four.
Preservation went the same way. No reachable provider guarantees an untouched region, so Scenri composites it locally. Measured on one shot and one instruction, “add one subtle prop”, byte-identical pixels went from 1.8 percent to 86.0 percent.
