We built a small agentic harness with one job: hand a model a song, a hard dollar budget, and a set of tools, then get out of the way and let it produce a full music video on its own. The model researches which video models exist, generates clips, watches its own footage, edits with ffmpeg, and assembles a final cut.
A few readers of our last build-off said they wanted to see how tool use actually varies between models, so we gave frontier-level models an open-ended, long-horizon task where each model decides on its own what to research, what to generate, and how to edit. We log every tool call, so you can see exactly how each one worked (full transcripts below).
We ran two models, Claude Fable 5 and GPT-5.6 Sol, each at two budgets ($25 and $100), for four runs total. Every run got the same song (Bruno Mars and Mark Ronson's "Uptown Funk"), a short text description, and a time-stamped lyric transcript.
Each model ran an autonomous tool-calling loop with six tools:
Once the budget hits zero, paid generation is refused, but the model can keep editing. Every model message, tool call, charge, and error was logged. The whole harness is open source at github.com/hershalb/music-video-arena, so you can run it yourself.
Each clip below is the model's final, self-assembled output.mp4, full length with the original song muxed in.
| Claude Fable 5 · $25 | GPT-5.6 Sol · $25 |
|---|---|
| Claude Fable 5 · $100 | GPT-5.6 Sol · $100 |
|---|---|
All four runs finished on their own (none hit a step or time limit) and all four produced a valid, full-length video with the original song muxed in.
| Model | ||||||||
|---|---|---|---|---|---|---|---|---|
| Claude Fable 5 | $25 | 39m10s | 25 | 0 | 54 | 1 | $24.30 | 1280x720 |
| GPT-5.6 Sol | $25 | 42m52s | 38 | 61 | 46 | 10 | $23.18 | 1280x720 |
| GPT-5.6 Sol | $100 | 49m39s | 34 | 0 | 70 | 2 | $36.57 | 1280x720 |
| Claude Fable 5 | $100 | 38m56s | 28 | 0 | 80 | 0 | $48.60 | 1920x1080 |
"Generation spend" is the metered FAL cost, which is what the budget caps. At $25 both models nearly exhausted it. At $100 they spent $36.57 (Sol) and $48.60 (Fable), so more budget did translate into more footage. It does not include the cost of running the model itself, which we add below.
Left to choose their own tools, the models diverged. Three of the four runs went pure text-to-video. Only GPT-5.6 Sol at $25 used an image-to-video pipeline (generating stills first, then animating them). GPT-5.6 Sol at $100 mixed three different video models in a single run.
| Run | Image model | Video model(s) | Approach |
|---|---|---|---|
| Fable 5 · $25 | none | Wan 2.5 t2v ($0.05/s) | Text-to-video only |
| Sol · $25 | FLUX schnell ($0.003/img) | Wan 2.2-5b i2v ($0.10/s) | Keyframe, then image-to-video |
| Sol · $100 | none | Wan 2.5 ($0.05/s), Veo 3.1 Lite ($0.10/s), Hailuo 2.3 Standard ($0.28/video) | Text-to-video, mixed models |
| Fable 5 · $100 | none | Seedance 1.0 Pro t2v (~$0.12/s at 1080p) | Text-to-video only |
Prices are FAL's listed rates, shown per second of output video unless noted. Hailuo 2.3 Standard is priced per video (about $0.28 per 6s clip), and Seedance 1.0 Pro is token-priced (~$0.62 per 5s 1080p clip, shown above as its effective per-second rate). Distinct clips generated per run ranged from 46 to 80.
How each run spent its tool calls (this counts attempts, including failed generation calls).
| Claude Fable 5 · $25 | GPT-5.6 Sol · $25 |
|---|---|
| Claude Fable 5 · $100 | GPT-5.6 Sol · $100 |
|---|---|
Each run's full transcript, every plan, tool call, and command, is here: Fable 5 · $25, Sol · $25, Sol · $100, Fable 5 · $100.
"Failed calls" are generation requests that returned an error (mostly transient network failures to the provider). They were not charged, but the model spent steps retrying them.
| Run | Input tokens | Output tokens | Reasoning | Cached input |
|---|---|---|---|---|
| Fable 5 · $25 | 1,476,900 | 44,341 | n/a | 0 |
| Sol · $25 | 2,956,270 | 33,220 | 9,656 | 2,558,029 |
| Sol · $100 | 2,097,572 | 31,715 | 12,330 | 1,819,050 |
| Fable 5 · $100 | 2,264,610 | 48,029 | n/a | 0 |
The budget only meters generation (FAL) spend. Adding the LLM token cost for Claude Fable 5 ($10 / $50 per 1M input/output) and GPT-5.6 Sol ($5 / $30), gives the total cost of each run.
| Run | Generation spend | LLM token cost | Total cost |
|---|---|---|---|
| Fable 5 · $25 | $24.30 | $16.99 | $41.29 |
| Sol · $25 | $23.18 | $4.27 | $27.45 |
| Sol · $100 | $36.57 | $3.25 | $39.82 |
| Fable 5 · $100 | $48.60 | $25.05 | $73.65 |
For Claude Fable 5, the tokens alone ran $16.99 to $25.05, about 30-40% of each run's total. GPT-5.6 Sol's token cost stayed near $3-4 despite similar token volume.
The arena is open source: github.com/hershalb/music-video-arena. Point it at your own song and budget, swap in whichever models you want to pit against each other, and see what they build. Issues and PRs welcome, we would love feedback on the setup.
None of the music videos were great, but watching how the models got there was pretty interesting and does show where gaps still clearly exist for frontier-level models. A few things notes:
We'll see if models can improve on more subjective/stylistic tasks as they continue to get smarter, but for now there's still a lot of room for improvement.
Every model mentioned here is available on TryAI with one account, pay-as-you-go, no subscription.
Start free