schemafit icon

schemafit

schemafit is a local-first CLI that lints JSON Schema and structured-output specs against major LLM provider constraints to catch issues before CI fails.

schemafit

Overview

schemafit is a local-first CLI for checking structured-output, JSON Schema, and tool-definition payloads against the constraint surfaces of major LLM providers. The page positions it as a way to catch provider-specific schema failures before they reach production.

Instead of waiting for an API 400, schemafit flags the exact JSON-Pointer path, keyword, and reason for each violation, then exits non-zero so CI can fail the change. It also offers a best-effort repair pass, a provider listing command, and an optional live-verify mode for checking whether the static rule pack has drifted from current provider documentation.

Features

Static schema linting

Checks schemas against provider-specific constraint surfaces and reports violations with the exact JSON-Pointer path, violated keyword, and reason.

Versioned rule packs

Encodes provider rules as declarative, versioned packs so the checks are reviewable and tied to documented constraints rather than ad hoc logic.

CI enforcement and SARIF output

Fails with a non-zero exit code and can emit SARIF so violations can break CI and surface in GitHub code scanning.

Local-first, offline operation

Runs offline with no API key, no model calls, and zero runtime dependencies, which makes it easy to vendor and audit.

Schema repair pass

Includes a repair command that emits a best-effort provider-valid variant by stripping or rewriting offending keywords.

Provider listing and drift detection

Provides a providers command and an optional live-verify mode to compare static packs with provider docs and detect drift.

Use Cases

  • Pre-merge schema validation

    Use schemafit in a build pipeline to catch schemas that would 400 on a target provider, so the pull request fails before the broken schema ships.

  • Multi-provider schema maintenance

    Run linting across a set of schemas when supporting several LLM vendors, so you can spot provider-specific keyword conflicts early.

  • Best-effort schema repair

    Use the repair command to produce a provider-valid variant when a schema needs to be adapted for a particular API surface.

  • Code-scanning workflows

    Generate SARIF output when you want schema violations to appear in GitHub code scanning or the Security tab.

  • Offline verification

    Run the demo or lint commands from a fresh clone in a network-restricted environment, since the core workflow is offline and keyless.

Pros and Cons

Pros

  • Runs locally without API keys, model calls, or runtime dependencies.
  • Targets multiple major providers from one schema linting workflow.
  • Surfaces the exact failing path, keyword, and reason, which should make debugging more direct.
  • Can fail CI before a bad schema reaches production.
  • Offers SARIF output for code-scanning workflows and an optional live-verify drift check.

Cons

  • The source does not provide a full support matrix, installation matrix, or platform limitations beyond the CLI being static and offline.
  • The repair pass is described as best-effort, so it is not guaranteed to produce a complete or semantically perfect fix for every schema.

FAQ

What does schemafit do?

It is a local-first CLI that lints schemas against the documented constraint surfaces of OpenAI, Anthropic, Gemini, Mistral, and Cohere. The source describes commands for linting, repairing, listing supported providers, and running a hermetic demo.

Does schemafit need API keys or model calls?

The source says it is static and offline, with no API key required and zero runtime dependencies. It can run from a fresh clone or after a single pip install.

Can schemafit be used in CI?

Yes. The product description says lint exits with a non-zero code so CI can fail the build before a schema reaches production. It also mentions SARIF output for GitHub code scanning and the Security tab.

Which providers does schemafit cover?

The page explicitly lists OpenAI, Anthropic, Gemini, Mistral, and Cohere as supported providers. It also says provider rule packs are versioned and grounded in documented constraints.

Does schemafit only report problems, or can it repair schemas too?

The page says a repair pass can emit a best-effort provider-valid variant of a schema by stripping or rewriting offending keywords while trying to preserve intent. It also notes an optional live-verify mode for drift detection in v0.5.

Quick Facts

Category
Developer Tool
Product type
CLI
License
MIT
Operation
Static, offline
Supported providers
OpenAI, Anthropic, Gemini, Mistral, Cohere
Source domain
danmercede.com