Terminal-Bench (TB)

advancedwidespread

An open test series measuring how well an AI agent can independently perform real, multi-step computer tasks via the command line.

In plain terms

Terminal-Bench is to AI agents what a practical exam is to a skilled worker. They do not ask how to fix a faucet; instead, they place the agent in front of a dripping faucet and check at the end whether it still drips.

The "terminal" is the command line: the text window where the computer is controlled not by clicking but by typed commands. System administrators and programmers often work here, and most coding agents perform their work this way.

The tasks resemble real work: installing and configuring a program, fixing faulty code, processing data, starting a server. The agent does not receive step-by-step help; it must figure out itself what to type and how to fix it if it makes a mistake.

Standard chatbot tests score response quality. Here, only whether the work is actually completed at the end matters.

How it works

Each task comes with a text description and an isolated, disposable computing environment (a container). The agent can work freely inside it, because any mistakes it makes do no harm outside the environment.
The agent receives the task, issues commands, reads the result, and decides the next step based on that. This can continue for hundreds of rounds.
When the time limit expires, or the agent declares the work complete, pre-written verification programs examine the final state of the environment: does the program work, is the result correct?
A task either succeeds or fails; there are no partial scores. The final result is the percentage of successfully solved tasks.
The results are published on a public leaderboard. On this list, the model appears together with the agent framework, that is, the program that connects the model to the command line and directs its work.

An everyday example

You read a news report that a new AI model has been released, and the company boasts that it "came first on Terminal-Bench". This means that, with a particular agent framework, the model independently solved more command-line tasks than earlier systems. From this you can cautiously infer how far you can trust it with longer, independent IT work. It is also worth checking whether the company ran the measurement itself or an independent party did.

Who is affected

developerThis is one of the most frequently cited metrics when you need to decide which coding agent or model is worth using. → Code

entrepreneur, executiveIt helps assess whether an AI tool manufacturer's promises are backed by public, comparable performance. However, verify separately whether the manufacturer measured the result itself.

employeeRising scores indicate how quickly agents are becoming capable of independent IT and data processing tasks. → Work & HR

A common misconception

MythIf a model scores highly on Terminal-Bench, it can reliably do any IT task for you.

In realityThe test measures well-defined, machine-verifiable tasks in a sterile environment. Real work is full of vague expectations, human coordination, and errors that cannot simply be undone.

Limits and risks

The score reflects the combined performance of the model and the agent framework, so the same model can yield very different results in another tool.
The tasks are public, so a model may have encountered them during training, which can inflate the results.
Results reported by companies themselves are not always generated with the same settings, so when comparing, check whether the measurement is independent.
It measures only command-line work, revealing little about handling graphical interfaces, customer communication, or safe behavior.

Similar, but not the same

SWE-bench – SWE-bench measures how well models fix issues reported in open-source projects, while Terminal-Bench is broader: it also covers system administration, data processing, installation and many other command-line tasks.
coding agent – The coding agent is the software doing the work, while Terminal-Bench is the exam used to measure it.
Harbor – Harbor is the framework for running tests, while Terminal-Bench is the task collection and leaderboard itself.

Related concepts

Broader

Benchmark, AI evaluation, Agent evaluation

Narrower

Terminal-Bench 2.0

On the edge

SWE-bench, coding agent, AI-agent, Ranglista, Sandbox

A brief history

Terminal-Bench was launched in May 2025 through a collaboration between Stanford University researchers and the Laude Institute as an open-source project. In November 2025, version 2.0 appeared, containing harder, more thoroughly verified tasks, along with the Harbor framework for running the tests.

Where things stand

At the end of 2026, major AI developers regularly report their Terminal-Bench result when unveiling new models; it has become one of the core benchmarks for coding agents. The scores of the best systems are rising quickly, so the creators have already made the task set harder with version 2.0 and are working on further, harder tasks.

Status: 2026 November

Also known as

Terminal-Bench · Terminal Bench · TerminalBench · T-Bench · tbench · Terminal-Bench 2.0 · TB 2.0 · terminal agent benchmark

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.