TestSprite launches an open-source command-line tool to help AI agents check their own work

Overview

TestSprite Inc. has open-sourced its command-line interface tool that lets AI coding agents verify their own work. The tool runs an agent's described behavior in the cloud against a live browser or live API, then returns a single failure mode with screenshots, a root cause hypothesis and a recommended fix so the agent can repair the code and rerun. Alongside the release, TestSprite launched CoderCup, a public competition that uses the CLI as a neutral referee to score frontier coding agents.

Key Takeaways

TestSprite open-sourced its CLI tool that gives AI coding agents a quality assurance loop instead of a one-time spot check.
The tool runs tests in the cloud against a live browser or live API and never uses mock protocols.
Each run returns the failing step and its neighbors, screenshots, a DOM manifest, the test source, a root cause hypothesis and a recommended fix.
Test coverage grows as the codebase grows, with dozens of new tests added each time the agent runs a phase of work.
TestSprite also launched CoderCup, a public leaderboard where frontier agents built and deployed the same app under one clock.
The CLI is released under the Apache 2.0 license and is available today.

Stats & Key Facts

#Even the best agent in TestSprite's competition broke 12% of the features that already worked.
#Installation requires Node.js 2.0 or higher.

TestSprite launches an open-source command-line tool to help AI agents check their own work

The problem TestSprite targets

Faster AI coding has produced faster code but also hidden defects.

›AI agents sometimes report a feature complete when tests failed, were incomplete, written incorrectly or skipped.
›An agent might write a function that runs on the surface but has a hidden bug that only triggers in an edge case.
›In the worst case, a change breaks another part of the codebase entirely.

Founder and Chief Executive Yunhao Jiao said the recurring cycle of fixing one thing and breaking another is what drives developers crazy, and that a verifier closes that gap.

How the verification loop works

The CLI gives coding agents a continuous test loop rather than a single check.

›The coding agent describes a behavior once.
›TestSprite runs it in the cloud the way a real user might, driving a live browser or hitting a live API.
›It returns one self-consistent failure mode that the agent reads, then fixes the code and reruns.

Because dozens of new tests are added each time the agent runs a phase of work, coverage grows alongside the codebase and acts as a safety net as application complexity changes.

Availability and installation

›The CLI is open source under the Apache 2.0 license and available today.
›It installs with npm install -g @testsprite/cli for machines with Node.js 2.0 or higher.
›Documentation and reference are available on GitHub.

CoderCup competition and leaderboard

TestSprite paired the release with a public coding battle.

›CoderCup is a public competition and leaderboard where AI agents built and deployed the same app under one clock.
›The newly open-sourced CLI acted as a neutral referee, scoring each phase and linking scores to public evidence.
›Full results and per-phase scores are published openly at codercup.ai.

The first event put several frontier agents head-to-head, including Anthropic PBC's Claude Code, OpenAI Group PBC's Codex, and Google LLC's Antigravity.

Why standard benchmarks fall short

›Jiao said most benchmarks score agents on a single number that is not what developers actually feel.
›Metrics that matter day to day include what agents get right the first time and how often they break something that used to work.
›Another key metric is whether an agent can recover on its own.

Frequently Asked Questions

What did TestSprite open-source?

TestSprite open-sourced its command-line interface tool that lets AI coding agents verify their own work, released under the Apache 2.0 license and available today.

How does the tool test code?

It runs a behavior described by the agent in the cloud against a live browser or a live API, never using mock protocols, and returns the failing step, screenshots, a DOM manifest, the test source, a root cause hypothesis and a recommended fix.

What is CoderCup?

CoderCup is a public competition and leaderboard in which AI agents build and deploy the same app under one clock, with the TestSprite CLI acting as a neutral referee and results published at codercup.ai.

Which AI agents took part in the first CoderCup event?

The first event included Anthropic's Claude Code, OpenAI's Codex, and Google's Antigravity going head-to-head.

How do I install the TestSprite CLI?

Install it with npm install -g @testsprite/cli on a machine running Node.js 2.0 or higher; documentation is on GitHub.

TestSprite positions its open-source CLI as a verification layer that lets AI coding agents catch and fix their own mistakes before users do.