Best Local LLM Tools & Frameworks

Open-source AI engine to run LLMs, vision, voice, image, video on any hardware, no GPU required.

Open-source Rust terminal coding agent that works with any model, hosted or local.

93+ production-ready AI projects with tutorials on LLMs, RAG, and agents.

Self-hosted AI answering engine with privacy, local LLM support, and cited sources.

QwenPaw: Your personal AI assistant, deployable locally or in the cloud, with memory, multi-agent support, and multi-cha

AI-powered news radar generating daily bilingual briefings in English & Chinese.

Open-source AI coding assistant for VS Code with local or hosted models, chat, edits, and reviews.

Semantic code intelligence MCP server with RAG, AST, spectral clustering, and memory graph for large codebases.

Intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI with semantic caching and auto-failover.

Portable offline AI assistant desktop app running GGUF models locally via llama.cpp.

Self-hosted mixed-vendor GPU inference cluster manager with speculative decoding proxy for faster LLM serving.

Run Hermes Agent and Claude Code locally on llama.cpp with zero API costs.

Self-hosted AI workspace for local & BYOK LLMs with agents, MCP tools, and deep research.

Local AI load balancer for Ollama and MLX fleets with auto-discovery, smart routing, and OpenAI-compatible API.

Private, offline AI coding agent for local Qwen models via Ollama with zero dependencies and real tools.

Open-source local AI workspace for chat, agents, coding, RAG and fine-tuning on your own machines.

Run IBM Granite 4.0 locally on Raspberry Pi 5 with Ollama.This is a privacy-first AI. Your data never leaves your device