Best Llama CPP Tools & Frameworks

Open-source AI engine to run LLMs, vision, voice, image, video on any hardware, no GPU required.

Intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI with semantic caching and auto-failover.

Portable offline AI assistant desktop app running GGUF models locally via llama.cpp.

Self-hosted mixed-vendor GPU inference cluster manager with speculative decoding proxy for faster LLM serving.

Run Hermes Agent and Claude Code locally on llama.cpp with zero API costs.