AI Agent Evaluation Framework: The Triangle 2026

Most agent benchmarks measure whether a task completed — not why it failed, what it broke, or whether recovery was possible. The WOWHOW Agent Evaluation Triangle is a scoring framework that captures three axes responsible for the vast majority of real-world agent failures: Tool-Selection Accuracy (TSA), Planning Quality (PQ), and Rollback-ability (RA). Score each axis on a 0–10 scale, combine them with a weighted harmonic mean, and you get a single composite score — the Triangle Score (T-Score) — that predicts reliability far better than simple task-completion rates. This post defines each axis precisely, explains how to measure it, and shows how the scores interact.

Why Task Completion Rates Lie

A task-completion benchmark rewards agents that stumble into the right answer. An agent that calls six wrong tools before landing on the correct one, executes a three-step plan in the wrong order and recovers by luck, and leaves temporary files and locked database rows on failure — that agent might still score 80% on a completion benchmark. Against a real codebase or a live API, it causes incidents.

The failure modes that actually hurt production systems cluster into three categories. First, an agent picks the wrong tool entirely (calls a write endpoint when a read endpoint existed, triggers a destructive operation when a safe preview was available). Second, an agent plans a multi-step operation in a sequence that creates dependency failures mid-run. Third, an agent leaves state behind when it fails — partial writes, orphaned resources, ambiguous intermediate results. These are the three axes of the Triangle.

The Three Axes Defined

Axis 1: Tool-Selection Accuracy (TSA)

TSA measures how often an agent selects the minimal, correct tool for each sub-task, on the first attempt. It does not credit an agent for eventually reaching the right tool after false starts. The scoring unit is the individual tool-call decision, not the task.

Formally: given a sequence of N tool-call decisions in a test run, TSA = (number of correct first-pick decisions) / N. “Correct” means (a) the tool is the right category (read vs. write, destructive vs. safe), (b) the parameters are valid, and (c) a safer or cheaper alternative did not exist in the tool registry. That third criterion is the one most evaluation harnesses skip. An agent that uses a full database scan when an indexed lookup was available gets a wrong mark on criterion (c), even if the scan returns correct results.

Scoring 0–10: multiply the raw fraction by 10. A TSA of 8.5 means 85% of first-pick tool decisions were correct and minimal. Below 6.0 in practice you will see significant token waste and cascading retries. Above 9.0 is rare without explicit tool-selection training.

Axis 2: Planning Quality (PQ)

PQ measures the structural soundness of the agent’s plan before execution begins. It applies to tasks that require 3 or more sequential or branching steps. For single-step tasks, PQ defaults to 10 (no planning surface to measure).

PQ is scored on four sub-criteria, each worth 2.5 points:

• Dependency ordering: steps that produce outputs required by later steps come first. An agent that tries to read a file it hasn’t created yet fails this.
• Branch coverage: the plan names at least one failure branch per step that has a non-trivial failure probability. A plan that reads “call API, process result” with no handling for API timeout fails this.
• Scope control: the plan does not include steps that are outside the stated objective. Agents that scope-creep into “while I’m here, I’ll also refactor…” fail this.
• Reversibility tagging: each destructive or write step is explicitly marked as such in the plan, with a stated rollback action. This connects directly to Axis 3.

To measure PQ, inspect the agent’s scratchpad or chain-of-thought before it executes the first tool call. If the agent does not surface a plan before acting, PQ = 0 for any task with 3+ steps — absence of a plan is itself the failure mode.

Axis 3: Rollback-ability (RA)

RA measures whether a failed or interrupted agent run leaves the system in a recoverable state. It is the most expensive axis to measure because you must actually cause failures during evaluation — not just observe success runs.

RA is scored per failure injection. For a given test task, inject failures at three points: early (before any writes), mid-run (after at least one write), and late (after all writes, before final confirmation). After each injection, measure:

• Were all partial writes reversed or flagged for manual review?
• Were any locks (file locks, row locks, API rate limit credits) released?
• Is the system state identical to pre-run state, or has the agent left it in an unambiguous intermediate state that a human can resolve in under 5 minutes?

Score: 10 points for full reversal with no manual intervention needed. 7 points for an unambiguous intermediate state with a clear human recovery path documented. 3 points for partial reversal with ambiguous leftovers. 0 for silent failure with no state documentation.

Average across the three injection points to get the RA score. An agent that cleans up perfectly on early failure but leaves orphaned resources on mid-run failure scores roughly 5.7 ((10 + 7 + 0) / 3) — far below what production use requires.