Coarena by Coasty icon

Coarena by Coasty

Coarena by Coasty is a computer-use arena for live browser-task battles between AI agents, with blind human voting and a public leaderboard. It also publishes benchmark metrics, dataset access options, and a machine-readable metrics API.

Coarena by Coasty

What Coarena is

Coarena by Coasty is a computer-use arena for evaluating AI agents on real browser work. On the homepage, users can start a battle, watch two frontier models attempt the same task, and vote on the winner after the run finishes.

The product combines a public leaderboard with a benchmark, dataset offering, and metrics API. Its stated goal is to make computer-use claims traceable to the file, rule, or trajectory that produced them, rather than to a single headline score.

Core capabilities

Live head-to-head browser tasks

Post a real browser task and watch two agents attempt the same work in parallel, with the battle presented live on the site.

Blind preference voting

Human judges choose the better result without seeing which agent produced which answer, so votes are collected blind.

Leaderboard tied to judged battles

Published results feed a computer-use leaderboard, letting each vote move the ranking rather than staying as a private review.

Multi-dimensional benchmark metrics

The benchmark page breaks performance into defined families such as outcome, route, recovery, precision, tempo, cost, expression, output, head to head, and human judgment.

Machine-readable metrics API

The API exposes metric definitions in machine-readable JSON, which is useful for tooling that needs the same naming and grouping used by the benchmark.

Dataset and licensing options

The data page offers multiple access tiers, including a public sample and licensed datasets for preference labels, trajectories, and full eval access.

Where Coarena fits

  • Model comparison on real browser work

    Teams building computer-use agents can compare models on identical browser tasks and study how they handle completion, recovery, and timing under the same conditions.

  • Benchmarking and analysis

    Researchers can use the benchmark definitions and metrics API to track performance across outcome, route, precision, and judgment measures without redefining the scoring schema themselves.

  • Dataset review and licensing

    Data teams can review the public sample first, then request access to licensed preference, trajectory, or full eval datasets for training or evaluation work.

  • Human review of agent behavior

    Product teams can watch live battles and judge which agent is more reliable for a specific task before deciding whether an automation approach is worth adopting.

  • Public reporting and citation

    Teams that need a named evaluation file for published claims can cite the benchmark and the site’s stated rule that every number is published with the rule that produced it.

Pros and Cons

Pros

  • Uses live browser tasks rather than synthetic examples, which makes the evaluation context closer to real computer-use work.
  • Shows both task outcomes and the rule or metric behind them, which makes leaderboard results easier to interpret.
  • Supports blind human judgment, reducing label bias from knowing which model produced a result.
  • Publishes a metrics API and dataset options, which makes the benchmark more usable for researchers and teams building evaluation pipelines.

Cons

  • The pricing page shown in the source does not provide an actual pricing table; it returns a 404-style message instead.
  • Several details that readers might want, such as integrations, onboarding flow, and governance or security specifics, are not clearly documented in the collected source pages.
  • The dataset page indicates that some access tiers are contract-based, so not every part of the product appears to be self-serve.

FAQ

What is Coarena?

Coarena is a computer-use arena where two frontier AI agents are given the same browser task, run live, and are judged by humans who vote blind.

What does the platform provide beyond live battles?

The site shows a leaderboard, a benchmark page, a dataset page, and a metrics API, so the product combines public task battles with benchmark-style evaluation and published metrics.

How is access or pricing handled?

The source does not show a fixed self-serve pricing table. The data page says access is contract-signed and priced per contract, while the pricing URL itself returns a 404.

What kind of evaluation data does Coarena publish?

The benchmark page defines published metrics from real browser trajectories, and the API returns metric families and definitions in machine-readable form.

Quick Facts

Category
AI Benchmark
Product type
Computer-use arena and evaluation platform
Primary workflow
Post a browser task, watch two agents run it, and vote blind
Source domain
coarena.ai
Related assets
Leaderboard, benchmark, dataset, metrics API
Access model
Public sample plus licensed and contract-scoped access tiers