The Revert Rate That Lies

The Challenge

A revert rate answers one question: how often did a swap fail on-chain? It reads like a clean reliability number, and a leaderboard that ranks venues by it looks like a fair reliability ranking. It is not one, because the number only means the same thing across venues that fail the same way.

On a router (1inch, Uniswap, KyberSwap, ParaSwap, Odos), the user signs and submits the swap. If it fails, it fails on-chain, ClearTrace sees it, and the revert rate is exactly what it looks like: the odds your transaction does not land. On a batch-auction venue like CoW Protocol, the user signs an order and a solver settles the batch in one transaction. An order that cannot be filled, from a bad price, no solver, or expiry, never becomes a transaction at all. On an intent venue like 1inch Fusion, a third-party resolver submits, and an unfilled or expired intent leaves no trace either. Their on-chain failures are invisible by construction.

So a naive cross-model ranking rewards the venues whose failures you cannot see. Measured live on Ethereum over the trailing seven days (September 2026), CoW's revert rate reads 0.53% and 1inch's Limit Order Protocol reads 0.56%, while the routers 1inch and KyberSwap read 1.11% and 1.75%. Sort that column and the two off-chain venues top the table as the most reliable. That ordering is a category error, and the direction of the error is the dangerous part: it flatters the venues a third party can measure least, which is precisely what destroys the credibility of a neutral benchmark.

What We Built

ClearTrace, our live on-chain execution-intelligence engine, encodes the distinction as a single taxonomy that every consumer imports from one module. A venue is one of four execution models:

• Router: the user submits the swap transaction, so a failed swap fails on-chain and is counted. Revert rate is comparable.
• Batch auction: the user signs an order and a solver settles it in a batch, so orders that cannot be filled never become transactions. Revert rate is structurally near zero and is not a reliability signal.
• Intent: the user signs an intent and a resolver submits it, so unfilled or expired orders never become transactions. Same shape as a batch auction.
• Unclassified: we have not established who submits the settlement transaction, or the venue runs more than one model. Never ranked on revert.

Only routers carry a comparable revert rate, and even then only against other routers. Every published row is stamped with its execution_model and a revert_comparable flag, so a downstream consumer, whether the ClearTrace dashboard or an outside script hitting the public API, can never read a batch-auction revert rate as reliability without being told otherwise. Those two fields are live on the API today.

The discriminator is who submits, not where it settles

The obvious rule, "off-chain settlement means no reverts," is wrong in a way that matters. In classic request-for-quote, the market maker signs a quote but the taker still submits the transaction carrying that signature. An expired quote or a short maker balance therefore does revert on-chain, in the user's own transaction. A venue like that is router-shaped for measurement purposes even though its price came from off-chain. The question is not where the price originated. It is whether a user's failed swap becomes a transaction anyone can count.

The taxonomy, and the guard that enforces it

The rule lives in one place so it cannot drift. A short excerpt of the module every consumer imports:

# Execution models
ROUTER, BATCH_AUCTION, INTENT, UNCLASSIFIED = "router", "batch_auction", "intent", "unclassified"

# The ONLY model whose tx-level revert rate measures user reliability, because it
# is the only one where the transaction that fails belongs to the user.
REVERT_COMPARABLE_MODELS = frozenset({ROUTER})

class CrossModelComparison(ValueError):
    """Raised when code tries to rank/compare revert rates across execution models."""

def rank_by_revert(rows, rate_key="revert_rate_pct", project_key="project", strict=True):
    """Rank venues by revert rate, best first, WITHIN ONE EXECUTION MODEL.
    Refuses to return a list that mixes models, because such a list ranks nothing."""
    rankable, refused = [], []
    for r in rows:
        (rankable if is_revert_comparable(r.get(project_key)) else refused).append(r)
    if strict and refused:
        raise CrossModelComparison(
            "refusing to rank revert rates across execution models: these venues do "
            "not submit the user's transaction, so their on-chain revert rate is not "
            "comparable to a router's.")
    rankable.sort(key=lambda r: r.get(rate_key))
    return rankable

A compare page does not print a suppressed head-to-head as a blank. It replaces the number with the reason: a plain-English sentence explaining that the two venues settle differently, that the off-chain venue's on-chain revert rate is therefore structurally near zero regardless of how reliably it fills, and that putting that number next to a router's would overstate its reliability rather than measure it. The refusal is legible, not silent.

Under-claiming is the default

A venue ClearTrace has not established is unclassified, and unclassified never ranks on revert. Absence of evidence is not evidence. A venue that runs both models is also unclassified: Bebop settles taker-submitted quotes, where a failed fill reverts in the user's own transaction, and solver-submitted intents, where failures never land. A rate computed over that mixed population answers no question, so ClearTrace does not rank Bebop until the two are split. The same discipline applies to a venue like Tokenlon, whose very low on-chain rate would top any reliability ranking, which is exactly why it needs evidence about who submits rather than an assumption.

Why the guardrail exists

This is not a hypothetical. ClearTrace once shipped the sentence "CoWSwap's revert rate is 0.04% versus ParaSwap's 1.41%" to production, in body copy and in indexed FAQ schema. That is a 35x reliability claim, and it was purely an artifact of settlement: CoW's number was structurally low because unfillable orders never became transactions, not because it executed 35 times more reliably than a router. A flattering wrong number costs a neutral benchmark its credibility exactly as a damaging one does. The taxonomy, the CrossModelComparison guard, and a pre-send claim gate that blocks any cross-model revert claim before it leaves the building are the response to that specific mistake.

What It Shows

The rare skill here is not a model or a scraper. It is knowing which comparison is invalid and building the refusal into the pipeline, so a metric that looks universal is only ever computed over the population where it means something. Finding signal in adversarial data includes declining to manufacture signal that is not there, even when the invalid version would make the product look more complete.

What It Proves to a Client

That we build data infrastructure that is honest under its own load. Anyone can publish a leaderboard. The harder and more valuable thing is a benchmark whose numbers survive contact with the teams they describe, because the one comparison that would embarrass it is blocked in code rather than caught in review. For a foundation, a grant program, or a protocol team spending real money against a metric, that is the difference between a dashboard and a source of record. ClearTrace runs today as a free dashboard and public API, with paid per-chain integrity reports and standing monitoring engagements.