Tau2-Bench: When the User Can Act Too

τ²-bench is the dual-control step in Sierra's customer-service benchmark line. Where the original τ-bench let only the agent touch the world, τ²-bench hands the simulated user its own tools, so success now depends on coordinating a plan with a partner who has partial information and imperfect execution. The telecom domain is where that mechanic bites hardest.

τ²-bench (tau2-bench) is the second generation of τ-bench, the tool-agent-user benchmark Sierra released in 2024. It was introduced by Barres et al. in 2025 and keeps the essentials of the original: a simulated user, typed API tools over a structured database, a written policy the agent must obey, and the pass^k consistency metric. What it changes is the shape of the interaction. In the original benchmark the user was a passive counterpart that only talked; in τ²-bench the user is a co-actor with its own tools that can change the shared state.

The paper formalises this as a dual-control environment, modelled as a decentralised partially-observable Markov decision process (Dec-POMDP): two actors, each with partial visibility of the full state, must coordinate toward a shared goal. That framing is the point of the benchmark. A single-control agent eval measures whether a model can pick and sequence the right tool calls. A dual-control eval measures something production support agents actually do all day: work through a partner who holds capabilities the agent does not, and who does not always execute the plan correctly.

τ²-bench builds directly on the original retail and airline domains and the policy-scoring machinery, then adds four things:

Telecom is the headline new domain and the clearest illustration of dual-control. It models a phone-carrier support desk: the customer has a device or service problem (no data connectivity, dropped calls, a plan or billing question) and the agent must diagnose and resolve it. The catch is that many of the fixes live on the customer's own handset. The agent cannot toggle the customer's airplane mode, reseat their SIM, or read a diagnostic code off their screen. It has to instruct the user, then verify from the resulting state whether the instruction was carried out correctly.

That makes telecom qualitatively harder than retail or airline, where the agent holds every lever. A telecom trajectory can fail even when the agent's reasoning is perfect, simply because the plan was communicated poorly or the user executed it out of order and the agent did not notice. Retail and airline still ship in τ²-bench and remain the cleanest analogue to an e-commerce returns desk and a reservation system, but telecom is the domain that stresses the coordination skill the benchmark was built to measure.

The central experimental finding is that models lose ground when they move from the single-control setting to dual-control. The paper reports significant performance drops in the dual-control condition and attributes them to the difficulty of guiding user actions rather than to a lack of raw tool-use ability. An agent that resolves retail returns cleanly can still struggle to walk a distracted customer through a device reset.

The more useful contribution is diagnostic. Because τ²-bench can run the same task with and without an active user, it separates two failure families: reasoning errors, where the agent chose the wrong plan, and communication or coordination errors, where the plan was right but the hand-off to the user broke down. The original single-control τ-bench could not distinguish these, so a low score there was ambiguous. τ²-bench turns it into two numbers, which is what makes it a better tool for deciding whether to invest in a model's planning or its dialogue management. The consistency question still matters too: as with the original benchmark, pass^k across independent runs is the honest reliability read, because a task that succeeds once can fail on the next rollout when the simulated user behaves differently.

The most common mistake with the τ line is comparing a number from one version against a number from another. The original τ-bench board is frozen at the late-2024 model set (Claude 3.5 Sonnet tops it at 69.2 percent retail and 46.0 percent airline pass^1), and newer models are evaluated on τ²-bench and τ³-bench, which change the environment, add domains, and fix task bugs. A τ²-bench percentage is therefore not comparable to an original τ-bench percentage, and a telecom score is not comparable to a retail score.

For that reason this page does not reprint a per-model τ²-bench score table. Live standings are maintained upstream on taubench.com and the current framework lives in the sierra-research/tau2-bench repository, which now carries the τ³-bench task set forward. When you quote a figure, record the version (τ-bench, τ²-bench, τ³-bench), the domain (retail, airline, or telecom), and the user simulator, because all three change the number.

Reach for τ²-bench specifically when the coordination question matters: your agent has to guide a human or another system that holds capabilities it lacks, and you want to measure how well it communicates and verifies a plan rather than just how well it calls tools. The telecom domain is the closest public analogue to device-side tech support and human-in-the-loop workflows. If you only need classic retail and airline tool-use scoring, the original τ-bench domains still live inside this line; if you also need retrieval-grounded knowledge tasks or voice, use τ³-bench, which carries the τ² dual-control base forward. For single-turn function calling, BFCL is the closer benchmark; for browser agents, WebArena; for general computer use, OSWorld; for assistant-style research, GAIA. See our tool-use benchmark comparison for how the τ line sits alongside them.