Unified routing for multiple model sources
Send requests to public model providers, custom endpoints, and self-hosted models through a single hosted gateway.
ngrok.ai
ngrok.ai is a hosted AI gateway that routes, secures, and manages traffic to cloud or local LLMs through a single URL.
Overview
ngrok.ai is a hosted AI gateway for routing, securing, and managing traffic to large language models. The site describes it as one gateway for every model, covering public providers, custom endpoints, and models you run yourself.
Its main purpose is to let teams point their apps at a single URL, keep using the SDKs they already know, and add access control, observability, routing, and fallback behavior without rebuilding infrastructure.
Features
SDK-friendly setup
Use the gateway as the base URL in supported SDKs and route requests by swapping the API key rather than reworking your app.
Private local-model connectivity
Connect to local models privately without public IPs or inbound ports, keeping them reachable through the gateway.
Scoped access control
Assign separate access keys to apps or developers and restrict which providers and models each key can call.
Observability across calls
Track tokens, latency, errors, and spend across routed traffic so you can see how the gateway is being used.
Fallback and retry handling
Define healthy alternatives and reroute or retry requests when a model or key slows down or fails.
Use cases
Standardize model access in an app
Point an application at a single gateway URL and route requests through supported SDKs instead of wiring each model provider separately.
Build local-to-cloud fallback paths
Send traffic to a self-hosted model first, then fall back to public providers when the local model is unavailable or unsuitable.
Connect local models privately
Keep a private model reachable through the gateway without exposing public IPs or opening inbound ports.
Manage team access by scope
Create separate access keys for apps or developers and limit which models or providers each one can call.
Monitor routed usage and cost
Review tokens, latency, errors, and spend to understand how routed traffic behaves and where it is costing money.
Pros and Cons
Pros
- Consolidates public providers, custom endpoints, and self-hosted models behind one gateway.
- Supports a simple base-URL swap workflow in common SDKs.
- Includes scoped access control for separate apps or developers.
- Surfaces usage details such as tokens, latency, errors, and spend.
- Supports private connectivity to local models without public IPs or inbound ports.
- Adds reroute and retry behavior to help keep requests flowing when a model or key has issues.
Cons
- The provided pricing page returns 404, so the site does not currently expose a full pricing page in the collected sources.
- The pages reviewed mention key capabilities, but they do not provide a complete docs set for integrations, limits, or implementation details.
- Some advanced behavior is described at a high level rather than with explicit configuration examples or operational constraints.
FAQ
How do you start using ngrok.ai?
It is set up as a hosted gateway. The source says you change your base URL to https://gateway.ngrok.ai and swap your API key, then route requests through the gateway.
What integrations are shown on the site?
The source shows support for OpenAI SDK, Anthropic SDK, and Vercel AI SDK examples, and says the gateway can be called from APIs, Terraform, CLI, or custom tooling. It does not publish a full integration directory on the pages provided.
Can it work with local models as well as cloud providers?
The gateway can route to public providers and to models you run yourself. The source also mentions local LLMs that are reachable from the gateway without public IPs or inbound ports.
What does the platform let you monitor and control?
The product description says you can see tokens, latency, and errors rolled up across routed calls. It also says provider dashboards show spend, while the gateway adds app-, developer-, and model-level access control.
How is ngrok.ai priced?
The pricing text says there is one flat fee for routing, observability, and the rest, with $0.05 per million tokens plus the cost of inference when using ngrok keys. It also says there are no subscriptions or commitments.
Quick Facts
- Category
- AI gateway / developer tool
- Platform
- Hosted web service
- Primary users
- Developers and teams building AI products
- Source domain
- ngrok.ai
- Deployment model
- Single hosted gateway with API access
- Pricing model
- Flat usage fee with credits; no subscriptions or commitments stated
ngrok.ai Alternatives
AakarDev AI
AakarDev AI helps teams manage AI provider access, project setup, logs, and analytics in one dashboard. BYOK support included.
ByteAsk
ByteAsk is a terminal-first AI coding agent for C and C++ that edits repos and verifies changes with compilers, debuggers, sanitizers, and tests.
CreateOS Sandbox
CreateOS Sandbox is an isolated compute environment for running code and agent workloads in Firecracker micro-VMs with private networking and SDK, CLI, or MCP control.
hob
hob is an independent workspace for coding agents, with local control over sessions, terminals, history, routing, and follow-up work.
Ably Chat
Ably Chat is a chat API platform for custom realtime chat apps, with rooms, typing indicators, presence, reactions, message updates and usage-based pricing.
Manta AI
Manta AI is an autonomous web app testing tool that maps app behavior, catches regressions, and generates tests from a URL, no scripts or selectors needed.