Long-Horizon Terminal-Bench

Showcase

Super Mario, played by frontier models

One of the 46 Long-Horizon Terminal-Bench tasks, super-mario, asks the agent to actually play the game — issuing moves turn by turn through the terminal for the full 90-minute budget. When the run ends, the hidden verifier replays the agent's own move log in a deterministically-seeded engine and renders the result to video. Nothing here is hand-authored: every clip below is a model's real playthrough, exactly as the grader saw it.

The number on each card is that run's normalized super-mario reward in [0, 1]: full levels cleared plus fractional progress into the current level, divided by the pass ceiling of three levels (clearing through World 1-3 = 1.0). We also show how far each agent actually got. Tellingly, only four models ever cleared World 1-1 — most die on the first stage.

Sort

17 models shown, each from a single 90-minute attempt under the shared Terminus-2 harness. Clips are re-encoded for the web (360p, muted); the raw agent_play.mp4 renders live in each run's verifier output. GLM 5.1 and Grok 4.5 are omitted (no rendered replay available yet).

Curious how the rest of the suite is built and graded? Read the report →