oMLX is a native macOS inference server built on MLX for running local models on Apple Silicon Macs. It supports SSD-backed KV caching, continuous batching, and OpenAI- and Anthropic-compatible endpoints for tools such as Claude Code, OpenClaw, and Cursor.
oMLX is a native macOS inference server for local LLM use on Apple Silicon Macs. It is built on MLX and is designed to reduce the repeated recomputation that happens when coding agents invalidate context during a session.
Its main differentiator is paged SSD KV caching, which persists cache blocks to disk so previously used prefixes can be restored quickly across requests and even after server restarts. The product also adds continuous batching, multi-model serving, and OpenAI- and Anthropic-compatible APIs for local tools and clients.
oMLX stores KV cache blocks on SSD in safetensors format instead of keeping everything only in RAM. Hot blocks stay in memory, cold blocks move to disk, and previously seen prefixes can be restored across requests and server restarts.
The server handles concurrent requests through mlx-lm's BatchGenerator so multiple generations do not have to wait behind a single request. The homepage reports speedups up to 4.14× at 8× concurrency.
oMLX includes a native macOS menu bar app for starting, stopping, and monitoring the server. The companion dashboard supports model management, chat, and real-time metrics, and the app is signed, notarized, and auto-updating.
The dashboard can serve multiple model types at once, including LLMs, VLMs, embeddings, and rerankers. It also supports LRU eviction when memory is tight and model browsing/downloading from the interface.
oMLX exposes both OpenAI-compatible and Anthropic-compatible APIs, including /v1/chat/completions and /v1/messages. The site says it can work as a drop-in backend for Claude Code, OpenClaw, Cursor, and other OpenAI-compatible clients.
The product supports major tool-calling formats such as JSON, Qwen, Gemma, GLM, and MiniMax, plus MCP integration and configurable tool-result trimming for oversized outputs.
Use oMLX as the local backend for coding assistants that repeatedly revisit earlier context. The SSD-backed cache is designed to preserve and restore prior prefixes instead of recomputing them every time the agent revises its path.
Connect Claude Code, OpenClaw, or Cursor to a local macOS inference server using the drop-in API support. The site says the dashboard can generate the exact configuration command for each tool.
Serve more than one model type from the same machine, such as an LLM for chat, a VLM for image-aware work, and embedding or reranker models for retrieval tasks. The dashboard lets you browse, download, and manage models in one place.
Use the menu bar app and web dashboard to start the server, monitor status, and inspect real-time metrics without switching to a separate desktop app. This is useful for people who want local inference to stay available while they work.
Reuse models already present in Hugging Face cache folders or LM Studio directories, then serve them locally without another download. That makes oMLX a fit for users who already keep a local model library on disk.
oMLX is a native macOS inference server built on MLX. It exposes OpenAI-compatible and Anthropic-compatible endpoints so tools such as Claude Code, OpenClaw, and Cursor can connect to a local backend.
The source says oMLX supports Apple Silicon Macs running macOS 15+. The minimum RAM is 16GB, and 64GB or more is recommended for comfortable use with larger models.
Yes. The homepage says oMLX works as a drop-in backend for Claude Code, OpenClaw, and Cursor, and it can also be used with any OpenAI-compatible client on localhost:8000.
No. The FAQ says oMLX reads the standard Hugging Face cache, so models already downloaded in shared cache locations can be reused without re-downloading. It can also pick up LM Studio folders and custom directories.
The site says any MLX-format model from HuggingFace is supported, including LLM, VLM, embedding, and reranker models, with automatic handling for several reasoning-model formats.
ByteAsk is a terminal-first AI coding agent for C and C++ that edits repositories and verifies changes with the real compiler, debugger, sanitizers, and tests before showing a diff. It offers a free tier plus paid plans, with editor connectors and zero-retention handling described in the source.
CreateOS Sandbox is an isolated compute environment for running code and agent workloads inside Firecracker micro-VMs. It is designed for workflows that need machine-level isolation, private networking between sandboxes, and programmatic control through SDK, CLI, or MCP.
hob is an independent workspace for coding agents that keeps agent sessions, terminals, history, and follow-up work organized around the tools and providers you already use. It is aimed at developers who want local control over routing, history, and workspace structure rather than a bundled model stack.
Manta AI is an autonomous web app testing tool for teams that want to map application behavior, catch regressions, and generate tests without writing scripts or maintaining selectors. It works from a URL and supports plain-English test flows, run results with screenshots, and scheduled or deployment-triggered checks.
SonOf connects to your repo and PM tool, audits the codebase and surrounding product context, and turns approved work into shipped tickets with senior engineering review. It is aimed at founders and engineering leaders who need backlog help without hiring a full team immediately.
Ghost es un asistente de IA para terminal para chatear, generar código y ejecutar tareas en la línea de comandos. Incluye modelos gratuitos, es compatible con Linux, macOS y Windows, y es de código abierto.
