Local per-call measurement
Measures provider-reported usage, pricing evidence, request and response metadata, status, timing, retry evidence, and detector signals for calls the proxy actually sees.
inferock-bench
inferock-bench is a local diagnostic proxy for LLM API traffic that records per-call usage, failures, and timing, then generates receipts for billing-integrity review.
Local LLM cost-tracking proxy
inferock-bench is a local diagnostic proxy for metered LLM API traffic. It routes requests through localhost, records per-call events, and turns those events into receipts that summarize usage, timing, failures, and billing-related signals.
The project is aimed at people who need to inspect AI spend and billing integrity at the call level. The README frames it as a way to audit an AI or LLM bill, check token usage locally, and investigate whether a failed API call may still have been billed.
Core capabilities
Failure and billing-integrity evidence
Records delivery failures that can affect billing or audit trails, including billed-empty output, refusals, truncation, token-recount mismatches, duplicate request IDs, cache-discount-at-risk evidence, and provider-fault retries.
Surface coverage state
Labels each surface as watched-clean, signal, or not-openable so users can see when a check is unopened instead of assuming a clean result.
Receipt generation
Generates receipts from stored event records using the shipped @inferock/measure grading code and The Inferock Standard.
Local-first handling of keys and receipts
Keeps provider keys local to the proxy workflow and does not send them to Inferock; receipts remain local unless shared by the user.
Measured-provider scope
Targets four measured provider planes while remaining extensible for other surfaces that are not yet measured today.
Common ways to use it
Audit AI spend at the request level
Use the proxy when you want a per-call record of what was billed, what failed, and what timing or retry evidence was attached to the request.
Check token usage locally
Use it to inspect token usage locally for provider calls routed through your own machine before you rely on invoice totals or dashboard summaries.
Investigate billing-integrity questions
Use the receipt output when a failed request may still have affected billing, especially for cases involving empty output, refusals, truncation, or retry behavior.
Review what was and was not checked
Use the coverage-state labels to see whether a surface was watched cleanly, produced signals, or was not openable in the measured run.
Keep diagnostic evidence local
Use the local records and receipts when you need evidence you can keep in your own environment rather than sending provider keys or raw billing data to a third party.
Pros and Cons
Pros
- Captures per-call usage, timing, retry, and pricing evidence in one local workflow.
- Highlights billing-related failure modes such as truncation, duplicate request IDs, and cache-discount-at-risk signals.
- Keeps receipts local unless the user chooses to share them.
- Makes unopened checks visible through surface coverage states instead of silently treating them as clean.
Cons
- It only measures traffic routed through the local proxy, so bypassed calls are not covered.
- The README says it cannot explain a monthly bill without the matching invoice.
- Coverage is limited to the measured provider planes and other surfaces are described as extensible-by-design, not measured today.
FAQ
What does inferock-bench do?
It is a local diagnostic proxy that measures calls routed through localhost with your provider API key, then renders receipts from the recorded events. The README describes it as a local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints.
Which providers does it support?
The README names OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints as the measured provider planes. It also says the OpenRouter coverage includes observed hosts such as meta-llama, deepseek, mistral, moonshot/kimi, z-ai/glm, and qwen.
Who is this for?
The project is meant to help you audit an AI or LLM bill, measure token usage locally, and investigate whether a failed API call may still have affected billing. It is especially focused on per-call receipts and billing-integrity questions.
What kind of output does it produce?
The benchmark writes local event records and produces receipts using the shipped @inferock/measure grading code and The Inferock Standard. The README emphasizes that receipts are local unless you choose to share them.
Are there any important limitations?
No. The README says it cannot audit traffic that bypasses the local proxy, cap provider spend across unseen calls, or explain a monthly bill without the matching invoice. It also notes that anything outside the measured surfaces is extensible-by-design, not measured today.
Quick Facts
- Category
- Developer Tool
- Primary use
- Local AI bill and token-usage auditing
- Supported providers
- OpenAI, Anthropic, Gemini Developer API, pinned OpenRouter endpoints
- Workflow
- Route calls through localhost, record events, render receipts
- Source domain
- github.com
- Pricing
- Public GitHub repository; no product pricing disclosed on the project page
inferock-bench Alternatives
ByteAsk
ByteAsk is a terminal-first AI coding agent for C and C++ that edits repos and verifies changes with compilers, debuggers, sanitizers, and tests.
Manta AI
Manta AI is an autonomous web app testing tool that maps app behavior, catches regressions, and generates tests from a URL, no scripts or selectors needed.
PromptScout
PromptScout tracks how ChatGPT, Gemini, Google AI Overviews, and Perplexity mention your brand or competitors, with source analysis and website audits.
CreateOS Sandbox
CreateOS Sandbox is an isolated compute environment for running code and agent workloads in Firecracker micro-VMs with private networking and SDK, CLI, or MCP control.
Sleek Analytics
Sleek Analytics is a privacy-friendly web analytics tool with real-time visitor tracking, Core Web Vitals, and revenue attribution.
hob
hob is an independent workspace for coding agents, with local control over sessions, terminals, history, routing, and follow-up work.