Tool Ecosystem

Tools

35

Checked 05/27/2026, 01:45 AM UTC

Visible:

P OpenRouterAPI aggregation and routingCross-model API aggregation and routing platform with public model ranking and pricing surfaces.GitHub No reliable public source P0 AI LiteLLMAPI aggregation and routingModel access layer for OpenAI-compatible APIs, routing, budgets, and gateway operations.Stars 48,319Forks 8,364Issues 3,379 2026-05-26 AI PortkeyAPI aggregation and routingLLM gateway, routing, observability, and governance platform.Stars 11,858Forks 1,085Issues 173 2026-05-25 AI HeliconeObservability and cost monitoringLogging, analytics, cost, and gateway observability platform for LLM APIs.Stars 5,733Forks 589Issues 102 2026-05-18 AI LMSYS / Chatbot ArenaModel evaluationModel evaluation and leaderboard platform based on public battles and votes.Stars 39,485Forks 4,788Issues 1,036 2026-05-01 AI OpenCompassModel evaluationOpen-source evaluation framework and leaderboard tooling for large models.Stars 7,031Forks 780Issues 460 2026-05-25 AI InspectModel evaluationOpen-source evaluation framework for model capability and safety assessments.Stars 2,128Forks 526Issues 166 2026-05-26 AI EleutherAI lm-evalModel evaluationLanguage model evaluation harness maintained by EleutherAI.Stars 12,701Forks 3,288Issues 823 2026-05-11 AI RagasModel evaluationEvaluation framework for RAG and LLM applications.Stars 14,072Forks 1,442Issues 415 2026-02-24 AI LangChainAgent frameworksDevelopment framework for LLM applications, agents, tool use, and orchestration.Stars 137,689Forks 22,808Issues 587 2026-05-26 AI LlamaIndexAgent frameworksDevelopment framework for data connection, RAG, agents, and knowledge workflows.Stars 49,680Forks 7,462Issues 398 2026-05-20 AI AutoGenAgent frameworksMicrosoft open-source framework for multi-agent applications.Stars 58,424Forks 8,815Issues 861 2026-04-15 AI CrewAIAgent frameworksFramework for multi-agent orchestration and automated workflows.Stars 52,224Forks 7,253Issues 362 2026-05-26 P OpenAI Agents SDKAgent frameworksOfficial OpenAI Agents SDK for tool-using, handoff-capable, observable agent applications.Stars 26,676Forks 4,105Issues 110 2026-05-26 P Claude CodeCoding agentsAnthropic interactive coding agent tool and a priority monitored developer workflow object.Stars 126,759Forks 20,783Issues 9,599 2026-05-27 P Codex ecosystemCoding agentsOpenAI Codex-related coding agent and developer tool ecosystem, tracked as a priority object.Stars 85,970Forks 12,551Issues 5,196 2026-05-27 P OpenClawPrompt / workflow / skill managementTool object for agent skills, workflow packaging, and reusable execution units, tracked as a priority object.Stars 374,883Forks 78,108Issues 6,909 2026-05-27 AI vLLMInference and deploymentHigh-throughput LLM inference and serving framework.Stars 81,052Forks 17,264Issues 5,039 2026-05-26 AI SGLangInference and deploymentInference serving framework for language and vision-language models.Stars 28,295Forks 6,150Issues 3,557 2026-05-26 AI TensorRT-LLMInference and deploymentNVIDIA TensorRT-LLM tooling for optimized large-model inference and deployment.Stars 13,734Forks 2,410Issues 1,304 2026-05-26 AI OllamaLocal model platformsLocal model runtime and management platform.Stars 172,351Forks 16,297Issues 3,275 2026-05-25 AI LM StudioLocal model platformsLocal model download, runtime, chat, and local API platform.GitHub No reliable public source P0 AI llama.cppInference and deploymentC/C++ LLM inference project for local and edge environments.Stars 113,167Forks 18,806Issues 1,730 2026-05-26 AI LangfuseObservability and cost monitoringLLM tracing, evaluation, prompt management, and observability platform.Stars 27,987Forks 2,877Issues 643 2026-05-26 AI PromptLayerObservability and cost monitoringPrompt management, LLM observability, and evaluation platform.GitHub No reliable public source P0 AI LangSmithObservability and cost monitoringDiagnostics, evaluation, and monitoring platform for LLM applications in the LangChain ecosystem.GitHub No reliable public source P0 AI MCPPrompt / workflow / skill managementModel Context Protocol for connecting model applications with external tools and data context.Stars 86,262Forks 10,824Issues 522 2026-05-21 AI DifyPrompt / workflow / skill managementLLM application development, agent workflow, and knowledge-base platform.Stars 142,731Forks 22,452Issues 800 2026-05-26 AI CozePrompt / workflow / skill managementPlatform for building AI bots, agents, and workflows.GitHub No reliable public source P0 AI FlowisePrompt / workflow / skill managementLow-code LLM workflow and agent orchestration tool.Stars 53,090Forks 24,403Issues 876 2026-05-26 AI JanLocal model platformsLocal-first AI assistant and model runtime platform.Stars 42,683Forks 2,878Issues 398 2026-05-26 AI QdrantData and RAG toolsVector database and similarity search engine.Stars 31,580Forks 2,294Issues 557 2026-05-26 AI MilvusData and RAG toolsOpen-source vector database commonly used for RAG and semantic search.Stars 44,452Forks 4,026Issues 916 2026-05-26 AI WeaviateData and RAG toolsVector database and retrieval platform for AI applications.Stars 16,237Forks 1,296Issues 538 2026-05-26 AI ChromaData and RAG toolsOpen-source embedding database for AI applications.Stars 28,096Forks 2,273Issues 637 2026-05-26

Sources and coverage

Tools35

Priority monitored5

With GitHub metrics30

Categories9

Tool filter boundaryRecord typereference_snapshotRegistry snapshot05/27/2026, 01:45 AM UTCRow timestampsPer rowMissing-data statenull / unknown