Agent infrastructure
project-maya
mw00/project-maya
Project Maya is a local LLM inference engine that runs large GLM, DeepSeek, and MiniMax models on your own GPUs, with quantized weights, a web dashboard, and OpenAI/Anthropic-compatible APIs.
Topics
Related projects
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
freetoken16.8kFreeToken is an edge-native Mixture-of-Experts inference engine that runs large open-weight models on consumer hardware, with a desktop app, CLI, and OpenAI/Anthropic-compatible APIs.
TensorRT-LLM15kNVIDIA's open-source library for optimizing LLM inference with custom kernels and a high-performance runtime.
LMCache12.3kA vendor-neutral KV cache management layer for LLM inference that reuses cached context to cut latency and boost throughput.
lmdeploy8.1kA toolkit for compressing, deploying, and serving LLMs with fast inference engines, quantization, and distributed serving.
lemonade6.2kA local AI server that runs optimized LLMs on your own GPU and NPU and serves them to apps via standard OpenAI-compatible APIs.