Single gateway endpoint
ARBR exposes one OpenAI-compatible endpoint that sits in front of multiple providers, so applications can keep their existing client pattern while the gateway handles provider selection.
ARBR
ARBR is an open-source AI gateway and control plane for one OpenAI-compatible endpoint for routing, observability, budgets, and governance across model providers.
Overview
ARBR is an open-source AI gateway and control plane for routing, observability, budgets, and governance across model providers. It presents a single OpenAI-compatible endpoint so teams can place one operational layer between their apps and the models they already use.
The product is built to classify, route, meter, and govern each request before it reaches a provider, then return analytics and recommendations back to the stack. The site also describes support for local dashboards or OpenTelemetry export, plus deployment as one standalone container with the gateway, admin API, and dashboard together.
Platform capabilities
Routing and fallback controls
The request path can apply difficulty-aware selection, cost guardrails, rules, and fallback behavior before a model is chosen.
Governance in the request path
The platform can enforce budgets, rate limits, output guardrails, prompt-injection checks, and kill switches before inference happens.
Operational observability
Every request is turned into a structured event with cost, latency, tokens, routing decisions, app, team, model, and user attribution.
Live evaluation loop
ARBR can score live traffic with an LLM judge, then route future requests to the cheapest model that still clears the chosen quality bar.
Controlled model deployment
The docs describe canary and shadow deployments for models, with rollback and regression gates to help control rollout risk.
Use cases
Add a control layer without changing client code
Point existing OpenAI SDK, agent framework, or chat UI traffic at ARBR’s base URL so requests keep working while the gateway adds routing, logging, authentication, and budget enforcement.
Route tasks to different models
Use routing rules, cost guardrails, and fallback behavior to choose a model per request based on task difficulty, latency, or price.
Monitor AI usage and spend
Track spend, latency, tokens, and routing decisions by app, team, model, and user, then review the data in the dashboard or export it to OpenTelemetry-compatible tools.
Enforce governance on live requests
Apply budgets, rate limits, output guardrails, prompt-injection checks, and kill switches before inference to reduce operational risk.
Roll out models more carefully
Run canary or shadow tests for new models on real traffic, then use regression gates and rollback if evaluation results do not meet the bar.
Pros and Cons
Pros
- OpenAI-compatible base URL makes it easier to integrate with existing SDKs and apps.
- Covers routing, governance, observability, and evaluation in one layer.
- Supports many providers and also OpenAI-compatible or self-hosted endpoints.
- Deployment docs provide concrete production guidance, including Docker Compose, TLS, reverse proxy, and GCP walkthroughs.
- Data can be exported to local dashboards or OpenTelemetry-compatible tools.
Cons
- Public pricing is not available from the provided pages.
- The docs note that traffic capture into training data is only on the roadmap and not yet shipped.
FAQ
How do you connect an existing app to ARBR?
ARBR is designed to sit in front of existing OpenAI-compatible clients. The docs show Python, JavaScript, and cURL examples that point an SDK or app to ARBR’s base URL, then keep using the same chat-completions style requests.
How is ARBR deployed in production?
The deployment docs say ARBR runs as one container on one port, with the gateway, admin API, and dashboard together. In production, the docs recommend putting TLS and a reverse proxy in front of it and using the production Docker Compose profile.
What can be configured in the dashboard?
The homepage says ARBR can route, govern, observe, evaluate, and deploy models, and the deployment docs add that runtime behavior such as routing mode, required API keys, budgets, default model, rules, and AI policy are managed in the dashboard and stored in MongoDB.
Which model providers does ARBR support?
The homepage lists major providers such as OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Azure OpenAI, Vertex AI, Groq, DeepSeek, Moonshot, xAI, Mistral, LiteLLM, and NVIDIA NIM, and says ARBR also works with any OpenAI-compatible or self-hosted endpoint.
Does ARBR have public pricing information?
The source pages do not provide published pricing. The pricing URL fetched returned a GitHub Pages 404, so no plan details were available from the provided evidence.
Quick Facts
- Category
- AI gateway / control plane
- License
- MIT licensed
- Deployment model
- Single container on one port
- Compatibility
- OpenAI-compatible endpoint
- Source domain
- projectarbr.org
- Primary users
- Teams operating multiple model providers
ARBR Alternatives
ByteAsk
ByteAsk is a terminal-first AI coding agent for C and C++ that edits repos and verifies changes with compilers, debuggers, sanitizers, and tests.
CreateOS Sandbox
CreateOS Sandbox is an isolated compute environment for running code and agent workloads in Firecracker micro-VMs with private networking and SDK, CLI, or MCP control.
hob
hob is an independent workspace for coding agents, with local control over sessions, terminals, history, routing, and follow-up work.
Ably Chat
Ably Chat is a chat API platform for custom realtime chat apps, with rooms, typing indicators, presence, reactions, message updates and usage-based pricing.
MacSpoof
MacSpoof is a macOS MAC address changer that lets you change or randomize your Wi‑Fi MAC to reconnect and limit device logging on public Wi‑Fi.
Manta AI
Manta AI is an autonomous web app testing tool that maps app behavior, catches regressions, and generates tests from a URL, no scripts or selectors needed.