Agent infrastructure
evals
openai/evals
OpenAI's framework for evaluating LLMs and LLM systems, with an open registry of benchmarks and tooling for custom eval creation.
Related projects
cc-switch152.6kA cross-platform desktop manager for coding agents that switches API providers and centralizes MCP, skills, and prompts.
OmniRoute86.2kA free MIT AI gateway that exposes hundreds of providers and models through one OpenAI-compatible endpoint, with quota-aware fallback, token compression, routing strategies and MCP/A2A support.
OmniRoute85.6kA free self-hosted AI gateway that routes one OpenAI-compatible endpoint across hundreds of providers with fallback and compression.
dsh-our-free-model72.7kThis DeepSeek Harness plugin adds free, no-key access to several upstream model pools and exposes a local OpenAI/Anthropic-compatible gateway. It includes fallback routing, usage dashboards, token-saving compression and in-app updates.
free-claude-code68.1kLocal proxy gateway letting eleven coding agents share sixty-plus providers with fallback routing, token optimization, cost tracking and admin UI.
dsh-our-free-model58.9kA DeepSeek Harness plugin that adds free and bring-your-own-key model providers, a local OpenAI/Anthropic-compatible gateway, routing, usage dashboards, and token compression.