One index
or the fan-out

The question

A coding agent searches the web dozens of times a day. Each search goes to one index, and any one index sees about half the web. Jevlin's answer is to ask several indexes in one call and return one list. The obvious objection is that the best single index might already be good enough, and that the fan-out just costs more. We built the cheapest honest test we could think of: take questions whose answer lives on one known page, send the same query to each index on its own and to the router, and count how often the known page comes back in the first ten results.

No model reads the results. No judge scores them. A page either is the gold page or it is not, after the same URL normalisation on both sides. That makes the test cheap enough to run on hundreds of questions three times over, and it makes the numbers hard to dispute. It also limits what the numbers mean, which we come back to at the end.

The panel

485 questions from three public sets, each with a gold page the question's own authors fixed:

• 30 coding tickets from OpenBenchmarks' coding-agent board (SAP, ServiceNow, Workday integration tasks). The gold page is the documentation or forum page the reference solution cites.
• 201 developer-docs questions from Firecrawl's DevDex benchmark, each answered by one page of a project's documentation, matched by DevDex's own rule.
• 254 hard factual questions from SealQA's Seal-Hard split, where the gold is a specific page and the question is built to confuse a search engine.

Each question was searched three ways. The first phrasing is the question as an agent would type it; the second and third were written by Claude Opus 5.5 from the question text alone, with any words from the gold page's URL removed, and are published with the data. Every search was run three times, each time as a fresh fan-out, so trial-to-trial variance is inside the intervals.

Five indexes were measured on their own: Exa, Exa lite, Brave, Parallel and Perplexity. Each was called through the Jevlin router pinned to that single provider, so the request path, the result normalisation and the timing are identical for everyone. Two live Jevlin tiers were measured as the agent gets them: fast (Exa lite, Brave and Perplexity) and balanced (Exa, Brave, Parallel and Perplexity). That is 7 arms, 3 phrasings, 3 trials, and about 36,000 router calls, all of them in the published data.

Each index on its own

With one query, the gold page was in the first ten results this often:

Exa is the best single index on this panel, and it is not close for the rest: Exa lite and Brave trail by eight points, Parallel and Perplexity by twelve. Latency is from a separate quiet pass with six calls in flight, measured end to end from the client. Cost is the provider's list price as the router meters it.

One index, two, three, four

The interesting number is not the router's top 10. It is what the router had before it cut to ten. The light bars below pool everything the providers returned, starting from Exa and adding one index at a time. The dark bars cut those same pooled lists back to a single top 10 with reciprocal-rank fusion, the textbook merge.

Every added index adds reach: +16.1 points, 95% interval +12.9 to +19.4, 84 items won and 0 lost for all four against Exa alone. But cut back to ten slots, the four lists score 62.1%, which is where Jevlin's live top 10 lands too. The merge is as good as the fusion rule. What costs the points is the window: the router is holding 30.1 pages with the gold page in them 74% of the time, and showing ten. Against the pool of its own arms, balanced's top 10 is -11.5 points, 95% interval -14.2 to -8.8, 1 items won and 70 lost.

The same call records show what a wider window buys. We asked the router for twenty merged pages and scored the first ten; score all twenty and balanced reaches 69.4%, fast 65.4%. Twenty pages at today's excerpt length is about 21k characters, roughly what ten pages at twice the length would cost.

Questions where the gold page arrived only once a second, third or fourth index was added:

Found only with provider 2 (38 questions)

• how do you connect a custom sum calculation to a specific data stream in opentelemetry
  gold page: https://github.com/open-telemetry/opentelemetry.io/blob/HEAD/content/en/docs/lan
  rank per arm: Exa —, Exa lite #1, Brave #1, Parallel #1, Perplexity —, Jevlin fast #4, Jevlin balanced #4
• how do you get the absolute value of a column in polars using a query
  gold page: https://github.com/pola-rs/polars/blob/HEAD/docs/source/user-guide/sql/select.md
  rank per arm: Exa —, Exa lite —, Brave #3, Parallel —, Perplexity #3, Jevlin fast #2, Jevlin balanced #3

Found only with provider 3 (28 questions)

• Patch file.py so run(base_url) creates the tutorial's business-account record and references the associated person that was created previously. Use the documented SAP Cloud Application Programming Model OData v2 collection and sample parent payload instead of creating another person record.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-sap/sap-cloud-application-
  rank per arm: Exa —, Exa lite —, Brave —, Parallel #1, Perplexity —, Jevlin fast —, Jevlin balanced #8
• how does jax handle drawing values that are typically considered unpredictable in other libraries
  gold page: https://github.com/jax-ml/jax/blob/HEAD/docs/new_docs/101/random.md#chunk0
  rank per arm: Exa —, Exa lite —, Brave —, Parallel #1, Perplexity #2, Jevlin fast #7, Jevlin balanced #4

Found only with provider 4 (12 questions)

• Update the existing SAP OData request so run retrieves only user records associated with the United States through the article's custom read-only operation instead of reading the ordinary user collection. Keep the request as GET and return the decoded JSON response.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-members/let-s-code-crudq-a
  rank per arm: Exa —, Exa lite —, Brave —, Parallel —, Perplexity #5, Jevlin fast —, Jevlin balanced —
• Patch the ServiceNow Advanced Work Assignment request in file.py so run retrieves the set of availability modes an agent may use and indicates which modes represent availability. Preserve the existing httpx structure and agent identifier substitution.
  gold page: https://www.servicenow.com/docs/r/api-reference/rest-apis/agent-api.html
  rank per arm: Exa —, Exa lite —, Brave —, Parallel —, Perplexity #1, Jevlin fast #8, Jevlin balanced #9

Why the fast tier has three indexes

The fast tier is the free default, so its three arms have to earn their place. Here is the pooled reach as arms are added, from the best and the worst starting index; the order does not matter, because all three together land at the same place.

Three cheap indexes together reach 67.4% of gold pages, nine points more than Exa alone at a third of Exa's price per call. The live fast top 10 shows 60.3% of them and the top 20 shows 65.4%.

The third arm also answers the latency objection. A fan-out is as slow as its slowest index, and the obvious fix is to stop waiting. We replayed every search under four wait policies, using each arm's own measured time:

Returning without the slowest arm saves 0.66 seconds at the median on the fast tier and loses 5.3 points of reach. A hard 1.5-second deadline loses 20. The slow index is often the one carrying the page. In 35 questions, only the slowest of the three fast arms had the gold page at all:

• Update run() to call the released S/4HANA Cloud Public operation that confirms receipt for an inbound delivery whose item quantity has already been adjusted. Use the documented HTTP method and endpoint, retain httpx, and return the response.
  gold page: https://community.sap.com/t5/enterprise-resource-planning-q-a/cannot-post-goods-
  rank per arm: Exa #5, Exa lite #3, Brave —, Parallel —, Perplexity —, Jevlin fast —, Jevlin balanced —
  arm times: Exa lite 6.7 s, Brave 2.1 s, Perplexity 2.8 s
• Update the script to retrieve one SAP S/4HANA Cloud customer together with linked postal-address records so the integration can identify the correct address before making a change.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-members/updating-customer-
  rank per arm: Exa #2, Exa lite #5, Brave —, Parallel —, Perplexity —, Jevlin fast #2, Jevlin balanced #3
  arm times: Exa lite 6.1 s, Brave 2.9 s, Perplexity 3.4 s
• Update the existing SAP OData request so run retrieves only user records associated with the United States through the article's custom read-only operation instead of reading the ordinary user collection. Keep the request as GET and return the decoded JSON response.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-members/let-s-code-crudq-a
  rank per arm: Exa —, Exa lite —, Brave —, Parallel —, Perplexity #5, Jevlin fast —, Jevlin balanced —
  arm times: Exa lite 1.1 s, Brave 2.6 s, Perplexity 3.3 s

Three phrasings beat one, for everyone

Agents rarely get the first query right, and the router can take several phrasings in one call. We measured what three phrasings do for each arm on its own, then fused back to ten.

Three phrasings lift every index by twelve to sixteen points of reach, and no index lost a single question. Exa goes from 57.5% to 71.2% pooled, 64.7% fused back to ten; Jevlin balanced from 62.1% to 73.9% pooled, 68.2% in ten. This is the largest single lever in the whole study, and it is the agent's to pull with any search API. The router's search_queries field takes up to four phrasings in one request; each is billed as a search.

Questions Exa missed with the first phrasing and found with the second or third (66 of them):

• Update the httpx script to create a purchase requisition in SAP S/4HANA Cloud through the standard procurement API described by the integration example.
  gold page: https://community.sap.com/t5/enterprise-resource-planning-blog-posts-by-sap/s-4h
  Exa rank by phrasing 1 / 2 / 3: — / #1 / —
  the three phrasings

  1. SAP S/4HANA Cloud create purchase requisition API
  2. how to create a purchase requisition through the OData API in SAP S/4HANA Cloud integration
  3. API_PURCHASEREQ_PROCESS_SRV purchase requisition header POST payload
• Update the httpx request to retrieve flight booking details for carrier AA and connection 19 using the tutorial's custom read operation.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-members/odata-function-imp
  Exa rank by phrasing 1 / 2 / 3: — / #1 / —
  the three phrasings

  1. SAP tutorial flight booking custom read operation carrier connection
  2. how to read a flight booking by carrier ID and connection ID with a custom OData function in SAP
  3. SAP RAP or CAP custom function import flight booking carrier AA connection 19
• Update the existing SAP OData request so run retrieves only user records associated with the United States through the article's custom read-only operation instead of reading the ordinary user collection. Keep the request as GET and return the decoded JSON response.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-members/let-s-code-crudq-a
  Exa rank by phrasing 1 / 2 / 3: — / #10 / —
  the three phrasings

  1. SAP OData custom read-only function filter users by country
  2. how to define a custom read-only operation that returns users from the United States in an SAP OData service
  3. CAP OData function GET users country filter custom handler

Longer pages, but not fewer

The balanced tier shows about 1,200 characters per page. The router can show 600, 2,000 or 4,000. On the 455 questions that carry an answer string, we checked whether the answer appears in the text the agent is shown, at ten pages of each length and at fewer, longer pages for the same character budget.

At ten pages, 2,000 characters beats 1,200 by +2.9 and 4,000 by +6.0 (interval +4.0 to +8.2). At a fixed budget the order flips: three pages at 4,000 lose -3.3 against ten at 1,200, six at 2,000 are a wash, five at 4,000 are a wash. Longer pages put more answers in front of the agent as long as the number of pages stays the same. Spend the budget on pages first, then on length.

Questions whose answer appears only in the 4,000-character text (19 of them at 2,000):

• how does posthog calculate the mrr breakdown when showing monthly numbers instead of daily ones
  gold page: https://github.com/posthog/posthog.com/blob/HEAD/contents/docs/revenue-analytics
  answer: If you're looking at a timeframe where we display monthly data then we'll display the aggregated data for the whole month.
  …uct name - initial coupon - customer id - country - transaction amount because your revenue data is exposed as [managed if you're looking at a timeframe where we display monthly data then we'll display the aggregated data for the whole month. ## gross revenue - [using revenue data in product analytics](#using-revenue-data-in-product-analytics) posthog computes…
• in diffusers, where is it better to keep the variable that tracks noise levels to prevent extra delays during image generation
  gold page: https://github.com/huggingface/diffusers/blob/HEAD/docs/source/en/optimization/f
  answer: In general, the `sigmas` should stay on the CPU to avoid the communication sync and latency.
  …of the communication sync between the cpu and gpu. it becomes more evident when the denoiser has already been compiled. in general, the `sigmas` should stay on the cpu to avoid the communication sync and latency.…

By kind of question

The router's edge over Exa is largest on the hard factual questions and on developer docs. On the 30 coding tickets the three arms are within a few points, and thirty tickets cannot separate them; what the tickets do show is that Brave and Parallel on their own find the gold page on fewer than one ticket in five, so an agent routed to either alone would miss most of the time.

Questions where balanced found the gold page and Exa did not (43 of them; 23 went the other way):

• Patch file.py so run(base_url) creates the tutorial's business-account record and references the associated person that was created previously. Use the documented SAP Cloud Application Programming Model OData v2 collection and sample parent payload instead of creating another person record.
  gold page: https://community.sap.com/t5/technology-blog-posts-by-sap/sap-cloud-application-
  rank per arm: Exa —, Exa lite —, Brave —, Parallel #1, Perplexity —, Jevlin fast —, Jevlin balanced #8
• Patch the ServiceNow Advanced Work Assignment request in file.py so run retrieves the set of availability modes an agent may use and indicates which modes represent availability. Preserve the existing httpx structure and agent identifier substitution.
  gold page: https://www.servicenow.com/docs/r/api-reference/rest-apis/agent-api.html
  rank per arm: Exa —, Exa lite —, Brave —, Parallel —, Perplexity #1, Jevlin fast #8, Jevlin balanced #9
• how does jax handle drawing values that are typically considered unpredictable in other libraries
  gold page: https://github.com/jax-ml/jax/blob/HEAD/docs/new_docs/101/random.md#chunk0
  rank per arm: Exa —, Exa lite —, Brave —, Parallel #1, Perplexity #2, Jevlin fast #7, Jevlin balanced #4
• how do you connect a custom sum calculation to a specific data stream in opentelemetry
  gold page: https://github.com/open-telemetry/opentelemetry.io/blob/HEAD/content/en/docs/lan
  rank per arm: Exa —, Exa lite #1, Brave #1, Parallel #1, Perplexity —, Jevlin fast #4, Jevlin balanced #4

What this does not show

It does not show that agents answer better. A page in the top 10 is not a page read, and on OpenBenchmarks' multi-hop company board, where the metric is the agent's final answer, the same router ties the best single index. That board measures a different thing with a different judge, and we are not claiming any number from it here.

It does not show speed. The fast tier's median was 2.8 seconds in the quiet pass against 2.2 for its fastest arm alone; a fan-out waits for its slowest index, and the section above says what the wait buys.

It does not rank the indexes. Each one was measured through our router on our request shape; a vendor's own numbers on their own harness will differ, and the per-call records are published so anyone can check the comparison on their terms.

What changes in the router

Three things follow from the data. The window is the lever: the balanced tier will show twenty merged pages to agents rather than ten, because the pages are already fetched and the gold page is in the second ten for one question in fourteen. Page count comes before page length when the budget is fixed. And the client should send several phrasings per call, which the Jevlin skill now does.

The data

Everything is in the supersearch-tests dataset on Hugging Face: the 485 questions with their gold pages, the three phrasings per question, every successful call record with the exact request and the router's merged results, the scored results, the per-question examples used above, and the scripts that build the panel, run it, score it and render this page. The item sets are CC-BY-4.0, MIT and Apache-2.0 from their authors; our data is CC-BY-4.0. Rerunning the panel needs a router key and about $300 of provider spend.