LLM & Inference
SDKs, local runtimes, gateways, chat UIs and fine-tuning kits for large language models.
Use these when you need to call, serve, run locally, fine-tune or chat with a language model.
Ranked by vibe score
ollamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.★ 187k
transformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.★ 169k
open-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)★ 158k
langflowLangflow is a powerful tool for building and deploying AI-powered agents and workflows.★ 157k
vllmA high-throughput and memory-efficient inference and serving engine for LLMs★ 96.8k
lobehub🤯 LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.★ 85k
unslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.★ 79.3k
litellmThe fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]★ 63.1k
llama.cppLLM inference in C/C++★ 134k
LocalAILocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.★ 50.3k
aiThe AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents★ 27.9k
openai-nodeOfficial JavaScript / TypeScript library for the OpenAI API★ 11.3k
anthropic-sdk-typescriptAccess to Anthropic's safety-first language model APIs in TypeScript★ 2.2kanthropic-sdk-python★ 4.1kAlso useful here
AutoGPTAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.★ 188k
langchainThe agent engineering platform.★ 149k
OpenHands🙌 OpenHands: AI-Driven Development★ 93.2kopenai-cookbookExamples and guides for using the OpenAI API★ 77k
llama_indexLlamaIndex is the document processing platform for AI★ 52.9k
zodTypeScript-first schema validation with static type inference★ 44.3kopenai-agents-pythonA lightweight, powerful framework for multi-agent workflows★ 30.6k
E2BOpen-source, secure environment with real-world tools for enterprise-grade agents.★ 14.8ksmolagents🤗 smolagents: a barebones library for agents that think in code.★ 30.3kclaude-cookbooksA collection of notebooks/recipes showcasing some fun and effective ways of using Claude.★ 54.5kchatbotA full-featured, hackable Next.js AI chatbot built by Vercel★ 21k
aideraider is AI pair programming in your terminal★ 50.1k
bolt.newPrompt, run, edit, and deploy full-stack web applications. -- bolt.new -- Help Center: https://support.bolt.new/ -- Community Support: https://discord.com/invite/stackblitz★ 16.6kWhat are the best llm & inference repositories?
By vibe score (popularity, freshness and hygiene), the top LLM & Inference picks are ollama/ollama (187k★), huggingface/transformers (169k★), open-webui/open-webui (158k★), langflow-ai/langflow (157k★), vllm-project/vllm (96.8k★).
When do I need llm & inference?
Use these when you need to call, serve, run locally, fine-tune or chat with a language model.