> ## Documentation Index > Fetch the complete documentation index at: https://docs.mcprouter.dev/llms.txt > Use this file to discover all available pages before exploring further. # Benchmarks > Understand the qualified 0.0.1 filtering result, percentile calculation, workload, and evidence boundaries. # Benchmarks > **Documentation for v0.0.1.** Additional guides and API reference coverage are being expanded. Measurements below apply only to the identified artifact, workload and environment, not a universal performance guarantee. A benchmark measures how long a specific piece of work takes under recorded conditions. mcprouter's local filtering benchmark asks: **how quickly can the router find relevant tools in a catalogue of 1,000 tools?** It measures the default, model-free [tool filtering and ranking](https://docs.mcprouter.dev/integrate/filter.md) path. It does not measure a whole AI conversation or how quickly an external tool responds. The published `0.0.1` core archive has a qualified warm-query **p95 of 24.893034 ms**, meeting the **p95 ≤ 50 ms** target for the exact run described under [current release evidence](#current-release-evidence). That run used an explicitly accepted hosted-VM venue exception, not verified dedicated physical hardware. ## What does p95 ≤ 50 ms mean? **The target is p95 ≤ 50 ms: at least 950 of the 1,000 measured queries must finish in 50 milliseconds or less.** * **p95** means the 95th percentile, not the average. * **≤** means “less than or equal to.” A p95 of exactly 50 ms meets the target. * **50 ms** is 0.05 seconds, or one twentieth of a second. To calculate p95, sort the 1,000 measured durations from fastest to slowest and take the **950th duration**. This is the *nearest-rank* method: `ceil(0.95 × N)`, counted from one. For example, if the 950th duration is **42 ms**, the percentile meets the target. If it is **51 ms**, it misses the target. These are **illustrative examples, not measured results**. Up to 50 queries can be slower than 50 ms while this percentile target still passes. It is not a maximum-latency guarantee, a promise for every machine, or a production service-level agreement. A numerical pass alone also does not establish release qualification. ## What the test does | Part | Fixed workload | | - | - | | Catalogue | 1,000 synthetic, locally registered tools | | Ranking | Default lexical ranking; no model or embeddings | | Results | Five ranked tools returned per query | | Cold work | Router construction, registration, and the first query recorded separately | | Warmup | 100 distinct queries before the measured sample | | Measured sample | 1,000 distinct queries, separate from warmup queries | | Concurrency | One query at a time; no concurrent benchmark suites | | Query cache | Disabled; this is not a cached-answer speed test | | Side effects | No tool handlers or model providers called | Each timed query includes source capture, bounds and snapshot checks, eligibility checks across all 1,000 tools, lexical ranking, sorting, and building the returned results. Timing uses `performance.now()` around the awaited filtering operation. The 100 warmup queries are excluded from the measured p95. Cold construction, registration, and the first query are also excluded from that percentile and retained as separate costs. Post-call assertions and report serialization are outside the timed filtering operation. ## What it does not measure * **AI response time:** there are no inference requests, model repairs, or generated conversations in this workload. * **Network or MCP discovery latency:** the catalogue is already registered locally; the test does not contact a remote MCP server. * **Tool execution time:** filtering returns descriptors and never invokes the registered handlers. * **Semantic ranking quality:** lexical filtering speed does not establish whether a model or embedding ranker chooses the right tool. * **Concurrent production traffic:** a single-query workload does not establish throughput or latency under load. Real model evaluation is a separate check of routing outcomes and request accounting. Passing this latency benchmark cannot substitute for it. Likewise, a fast ranking result is not [permission to execute a tool](https://docs.mcprouter.dev/guides/security-and-trust.md). ## Three different limits | Limit | What it means | | - | - | | **p95 ≤ 50 ms** | The latency target for the 1,000 measured filtering queries | | **Five minutes (300,000 ms)** | The timeout for the whole benchmark workload process, not a per-query target | | **30 minutes** | The job ceiling for the hosted benchmark workflow, including setup and result handling | Finishing within five minutes does **not** mean the 50 ms target passed. The workload can finish within its timeout and still have a p95 above 50 ms. A timed-out, interrupted, or incomplete run cannot supply a passing result. The job ceiling is not time spent inside every query and does not guarantee exclusive physical hardware. ## How to read a benchmark report A useful report ties the measurements to the **exact candidate archive**, dependency and harness inputs, Node version, operating system, architecture, and CPU/RAM allocation. Hardware and image-store compatibility checks support reproducibility; they are not latency measurements themselves. Look for: 1. All raw warmup and measured durations, with the required sample counts and unique queries. 2. The nearest-rank p95, mean, population variance, and standard deviation—not only the fastest run. 3. Cold costs reported separately, plus the query-cache, concurrency, and zero-dispatch checks. 4. The command, exit status, network isolation, cleanup result, and any failed or incomplete attempts. 5. Review of the execution venue: dedicated hardware evidence or an explicitly scoped exception. A hosted VM must not be described as verified dedicated physical hardware. Equivalent required checks can run in local Docker or CI. Local testing is useful for diagnosis; CI helps reproduce and retain results. **Neither location automatically proves performance.** Evidence applies only to the version, workload, and environment actually tested. Do not silently retry until a favourable result appears or reuse an old exception for a new run. ## Current release evidence **The October 24, 2026 timed run was reviewed and qualified for the exact `0.0.1` core archive subsequently published on npm.** The measured warm-query p95 is **24.893034 ms**, below the **50 ms** target. | Recorded property | Qualified result or boundary | | - | - | | Package | [`@surf-lab/mcprouter`, version `0.0.1`](https://www.npmjs.com/package/@surf-lab/mcprouter/v/0.0.1) | | Candidate archive SHA-256 | `621e27364027e28de25ddddf06f2193287f1e77690a0b43dcca6475511e68147` | | Workload | 1,000 tools, 100 distinct warmups, 1,000 distinct measured queries; serial, query cache disabled | | Warm-query nearest-rank p95 | **24.893034 ms** | | Environment | Ubuntu 24.04 ARM64 hosted VM; observed Ubuntu 24.04.5, Node 24.21.0, Docker 28.0.4 with overlay2 | | Allocation | 2 logical CPUs, approximately 7.73 GiB RAM; CPU model unknown | | Venue | Explicit, run-scoped hosted-VM exception; **`dedicatedHardwareVerified=false`** | | Safety | Zero provider requests or handler dispatch; network disabled, no published ports, read-only filesystem, nonroot execution | | Completion | Self-test, measurement and assessment exited successfully; cleanup and artifact binding confirmed | | Cold registration | **12,835.822073 ms**, excluded from warm-query p95 | | First query | **56.540540 ms**, excluded from warm-query p95 | The cold costs make clear why the result is a **warm-query** qualification, not a cold-start guarantee or production SLA. The venue exception applies only to this run; it neither certifies dedicated hardware nor grants another hosted run. Other artifacts, dependency graphs, Docker stores, machines and concurrent workloads need their own evidence. Earlier untimed adapter self-tests passed in a local containerd-backed environment and on Docker 28.0.4 with a classic VFS store in a disposable Debian 13 ARM64 VM. Those remain image-binding/self-test compatibility evidence, **not measured p95 results**. Earlier development measurements retain their own artifact/environment identities and do not replace the qualified result above. ### Separate model qualification The original three complete model trials scored **120/120**, **120/120** and **119/120**, with **360 settled requests**, no unknown requests and no forbidden dispatches. The third trial's miss returned `needs_input` without dispatch; it was counted, not dropped or retried. These are finite routing-evaluation results, not filtering latency or universal provider accuracy. Those live measurements retain their **original dependency-graph identity**. The published replacement patches include bundled Undici `7.30.1` and use Zod `4.6.6` in the selected profile. Their bounded qualification carries the original model evidence forward through reviewed source-impact analysis and a **40-case offline differential comparison on each of Node 22 and Node 24**, not a fresh live-model run on the replacement graph. That comparison does not establish universal behavior, wire-byte identity or latency equivalence. No new provider request is authorized by these results. For API usage, see [Filter tools](https://docs.mcprouter.dev/integrate/filter.md). For the difference between filtering and execution, see [Choose an integration](https://docs.mcprouter.dev/start/choose-integration.md). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.