worldmonitor
MCPAGPL-3.0
Open-source real-time global intelligence dashboard with AI news aggregation, geopolitical monitoring, and infrastructur
83k·MCP Servers·Open source
Alternative to
olla
Open sourceHigh-performance LLM proxy and load balancer with intelligent routing, failover, and model discovery.
292Apache-2.0
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
Pros
- +Native support for multiple inference backends (llama.cpp, vLLM, Ollama, etc.)
- +Dual proxy engines for simplicity or high performance
- +KV-cache-aware sticky sessions for efficient multi-turn conversations
- +Automatic failover and health checks with circuit breakers
Cons
- −Requires understanding of LLM infrastructure concepts
- −May be overkill for simple single-backend setups
- −Documentation may be limited for advanced configurations
Target audience: Developers and DevOps engineers managing self-hosted LLM inference infrastructure who need reliable routing and load balancing.
Related tools
Other open-source tools that share tags with olla.
netdata
MCPGPL-3.0
Open-source, real-time infrastructure monitoring with per-second metrics, ML-powered anomaly detection, and zero-config
80k·LLM Ops·Open source
Alternative to
LocalAI
MCPMIT
Open-source AI engine to run LLMs, vision, voice, image, video on any hardware, no GPU required.
49k·Self-Hosted AI·Open source
Alternative to
siyuan
MCPAGPL-3.0
Privacy-first, self-hosted knowledge workspace with block-level editing, AI agents, and MCP support.
46k·Self-Hosted AI·Open source
Alternative to
kong
MCPApache-2.0
Kong is a cloud-native API, LLM, and MCP gateway with high performance and extensible plugins.
44k·LLM Ops·Open source
Alternative to
go-micro
MCPApache-2.0
Go Micro is a Go agent harness and service framework for building AI agents with MCP and A2A support.
23k·AI Agents·Open source