Agent infrastructure
inference
xorbitsai/inference
A unified model-serving library to deploy LLMs and multimodal models locally or in the cloud via OpenAI-compatible API.
Topics
Related projects
nano-vllm16.3kA lightweight vLLM reimplementation in about 1,200 lines of Python delivering fast, readable offline LLM inference.
LMCache12.3kA vendor-neutral KV cache management layer for LLM inference that reuses cached context to cut latency and boost throughput.
ml-engineering19.7kAn open book on machine learning engineering, collecting practical guides, scripts, and troubleshooting for training and serving large models.
kornia12.3kA differentiable computer vision library built on PyTorch for image processing, geometric vision, and augmentation pipelines.
AI_Tutorial3.7kA daily-updated curated knowledge hub of industry AI practices, top papers, and hands-on engineering lessons.
awesome-ai-agents-20262.1kA curated awesome-list of 340+ AI agents, frameworks and tools across 20+ categories, updated monthly.