Method & caveats

What this is, and what it is not.

A small benchmark of one-shot creative and technical work. Eight tasks, two variants, one model per run. The caveats are part of the result.

The shape of the test

Each run received the same sixteen prompts: eight tasks, each in a thin variant (intent only) and a thick variant (a full brief with audience, constraints and success criteria).

Subagents were asked to return exactly one self-contained file. No network requests, no external images or libraries, no build step. The orchestrator collected the reply and saved it.

The tasks are deliberately different: visual composition, interface design, critique, motion, explanation, fiction, implementation and code review. There is no single score that makes sense across all eight.

Creative work is therefore presented without a number. The parser and code-review tasks carry a short technical note because those tasks have verifiable failure modes.

Tok/s is (output + reasoning tokens) ÷ subagent elapsed time. It includes thinking and tool time, so it is end-to-end throughput, not raw decode speed.

The eight tasks

TaskFormatBrief
01 Poster SVG The Last Ferry, 11:40pm — a poster for a crossing between San Remo and the island of Vell. Two inks, one paper tone, a condensed grotesque and a monospace timetable.
02 Interface HTML Inkwell — photograph a page of handwriting, get clean LaTeX back. The hero must show a handwritten page beside its rendered output.
03 Critique HTML A shipped landing page is supplied as a fixture. Find the five most damaging problems, repair them, and record what changed in a note of at most 150 words.
04 Motion HTML A title card for a short film about a night ferry crossing. The word must rest legibly for at least two seconds, and the loop seam must be invisible.
05 Explanation Markdown Cover the problem DNS solves, what happens between typing an address and the page appearing, why answers are remembered, and what usually goes wrong.
06 Fiction Markdown A workplace after hours, third person, one character. The clock must earn its place: something in the story depends on it.
07 Build MJS parseDuration("1h30m") returns seconds. It accepts h, m and s in any order, rejects invalid input with TypeError, and runs its own tests with node:test.
08 Code review Markdown The fixture is a small keyed limiter. The thick version asks for severity, location, problem, why it matters and a direction, in at most eight findings.

Run conditions

The runs are not identical. “Subagent tool calls” counts every tool call made by the sixteen subagents in that run. A clean run made none. Prompt time is the subagent’s start-to-finish, including tool use. It is not wall-clock: most runs dispatched in parallel batches, and HY4 ran sequentially.

RunSubagent tool callsSlowest promptMedian tok/sArtifact fidelityStatus
DeepSeek 4.1 FlashRun 1 363 19m 31s 118 16/16 saved; run used tools Tool-assistedSubagents used a browser to refine their work.
GLM 5.3Run 1 3 12m 11s 102 16/16; one CSS repair One repairOne malformed CSS line was repaired after saving.
GPT 6 LunaRun 1 30 1m 33s 73 14/16 exact; 2 post-edits Skill-assistedSubagents loaded skills; two files were edited after saving.
Claude Haiku 5.5Run 1 0 42s 210 16/16; one unfenced reply One unfenced replyOne review arrived without a fenced block.
Claude Haiku 5.5Run 2 0 32s 214 16/16 exact; one allowed retry Clean runNo subagent tools; one identical retry.
HY4 FlashRun 1 0 1m 29s 175 16/16 exact Clean runNo subagent tools; all 16 files byte-exact.
Muse Spark 1.3Run 1 5 1m 06s 88 12/16 exact; 4 trimmed Not comparableFour visual files were saved trimmed.
Muse Spark 1.3Run 2 1 1m 15s 96 9/16 near-exact; 7 trimmed Not comparableSeven files saved trimmed; three prompts re-run after wording errors.

Muse Spark run 2 also re-ran three prompts after wording errors; the saved artifacts use the corrected prompts. Its first review attempt used a different fixture and was discarded.

What we checked

Limitations

A note on labels

Every card names the model and the run.

Claude Haiku 5.5 and Muse Spark 1.3 have two runs each. The run badge is part of the result, not a detail to ignore.

Browse the results