oMLX is a native macOS inference server built on MLX for local models on Apple Silicon Macs, with SSD-backed KV caching and OpenAI/Anthropic endpoints.
oMLX is a native macOS inference server for local LLM use on Apple Silicon Macs. It is built on MLX and is designed to reduce the repeated recomputation that happens when coding agents invalidate context during a session.
Its main differentiator is paged SSD KV caching, which persists cache blocks to disk so previously used prefixes can be restored quickly across requests and even after server restarts. The product also adds continuous batching, multi-model serving, and OpenAI- and Anthropic-compatible APIs for local tools and clients.
oMLX stores KV cache blocks on SSD in safetensors format instead of keeping everything only in RAM. Hot blocks stay in memory, cold blocks move to disk, and previously seen prefixes can be restored across requests and server restarts.
The server handles concurrent requests through mlx-lm's BatchGenerator so multiple generations do not have to wait behind a single request. The homepage reports speedups up to 4.14× at 8× concurrency.
oMLX includes a native macOS menu bar app for starting, stopping, and monitoring the server. The companion dashboard supports model management, chat, and real-time metrics, and the app is signed, notarized, and auto-updating.
The dashboard can serve multiple model types at once, including LLMs, VLMs, embeddings, and rerankers. It also supports LRU eviction when memory is tight and model browsing/downloading from the interface.
oMLX exposes both OpenAI-compatible and Anthropic-compatible APIs, including /v1/chat/completions and /v1/messages. The site says it can work as a drop-in backend for Claude Code, OpenClaw, Cursor, and other OpenAI-compatible clients.
The product supports major tool-calling formats such as JSON, Qwen, Gemma, GLM, and MiniMax, plus MCP integration and configurable tool-result trimming for oversized outputs.
Use oMLX as the local backend for coding assistants that repeatedly revisit earlier context. The SSD-backed cache is designed to preserve and restore prior prefixes instead of recomputing them every time the agent revises its path.
Connect Claude Code, OpenClaw, or Cursor to a local macOS inference server using the drop-in API support. The site says the dashboard can generate the exact configuration command for each tool.
Serve more than one model type from the same machine, such as an LLM for chat, a VLM for image-aware work, and embedding or reranker models for retrieval tasks. The dashboard lets you browse, download, and manage models in one place.
Use the menu bar app and web dashboard to start the server, monitor status, and inspect real-time metrics without switching to a separate desktop app. This is useful for people who want local inference to stay available while they work.
Reuse models already present in Hugging Face cache folders or LM Studio directories, then serve them locally without another download. That makes oMLX a fit for users who already keep a local model library on disk.
oMLX is a native macOS inference server built on MLX. It exposes OpenAI-compatible and Anthropic-compatible endpoints so tools such as Claude Code, OpenClaw, and Cursor can connect to a local backend.
The source says oMLX supports Apple Silicon Macs running macOS 15+. The minimum RAM is 16GB, and 64GB or more is recommended for comfortable use with larger models.
Yes. The homepage says oMLX works as a drop-in backend for Claude Code, OpenClaw, and Cursor, and it can also be used with any OpenAI-compatible client on localhost:8000.
No. The FAQ says oMLX reads the standard Hugging Face cache, so models already downloaded in shared cache locations can be reused without re-downloading. It can also pick up LM Studio folders and custom directories.
The site says any MLX-format model from HuggingFace is supported, including LLM, VLM, embedding, and reranker models, with automatic handling for several reasoning-model formats.
ByteAsk is a terminal-first AI coding agent for C and C++ that edits repos and verifies changes with compilers, debuggers, sanitizers, and tests.
CreateOS Sandbox is an isolated compute environment for running code and agent workloads in Firecracker micro-VMs with private networking and SDK, CLI, or MCP control.
hob is an independent workspace for coding agents, with local control over sessions, terminals, history, routing, and follow-up work.
Manta AI is an autonomous web app testing tool that maps app behavior, catches regressions, and generates tests from a URL, no scripts or selectors needed.
SonOf connects to your repo and PM tool, audits your codebase, and turns approved work into shipped tickets with senior engineering review.
Ghost is a terminal-based AI assistant for chatting, code generation, and CLI tasks. Includes free models, supports Linux, macOS, Windows, and is open source.
