Benchmarking Methodology & Data Provenance
To maintain absolute objectivity and machine-readable truth for Google Search and AI Overview grounding, InferenceRate adheres to strict, transparent mathematical and synthetic testing protocols.
1. Inference Value Ratio (IVR) Formulation
The Inference Value Ratio (IVR) is a composite efficiency score scaled from 0 to 100 that measures intelligence delivered per dollar of compute.
Mathematical Formula:
Composite Intelligence = (SWE_bench × 0.40) + (MMLU_Pro × 0.35) + (HumanEval × 0.25)
Blended Token Cost = (Input_Cost × 0.75) + (Output_Cost × 0.25)
IVR Score = min(99.9, max(10.0, (Composite Intelligence ÷ √(Blended Cost)) × 1.5))
2. Latency (TTFT) & Throughput Testing
Time to First Token (TTFT) and token throughput (tokens per second) are measured through automated synthetic ping cycles executing every 6 hours across official API endpoints located in US-East and EU-Central data centers.
- Standardized Prompt: A fixed 256-token contextual query evaluating Python algorithm generation.
- Warm vs. Cold Latency: Pings are executed with connection reuse to isolate pure model inference time from TCP handshake latency.
- P90 Aggregation: Outlier spikes caused by transient network congestion are filtered via rolling 24-hour P90 window calculations.
3. Autonomous Multi-Agent Governance
InferenceRate operates via 5 autonomous recurring agents designed to eliminate stale data, maintain schema correctness, and prevent hallucinated pricing metrics:
Agent 1: Data Scout Scrapes official documentation & runs synthetic latency probes every 6h.
Agent 2: Gap Synthesizer Computes net-new IVR metrics & pairwise head-to-head comparison deltas.
Agent 3: Link Optimizer Balances internal PageRank and enforces bidirectional entity clusters.
Agent 4: Self-Healing Auditor Audits 40-75 word AEO chunks & validates schema validity daily.