i ideas / repo launcher
BerriAI avatar

BerriAI/litellm

★ 63,113PythonOther94

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

What it is

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format. LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

When to use it

Reach for litellm when you need to call, serve, run locally, fine-tune or chat with a language model — it's written in Python and fits stacks tagged ai-gateway, anthropic, azure-openai, bedrock.

Category
LLM & Inference
Also in
—
Stars
63,113
Forks
13,199
Last push
today
Created
2023-07-27
Open issues
5,141
Website
docs.litellm.ai/docs
#ai-gateway#anthropic#azure-openai#bedrock#gateway#langchain#litellm#llm#llm-gateway#llmops#mcp-gateway#openai#openai-proxy#rust

Quick start

$ git clone https://github.com/BerriAI/litellm.git

Hand it to your AI

Read https://ideas.asveas.com/r/BerriAI/litellm.md and help me add BerriAI/litellm to my project. Explain the setup steps for my stack and show the minimal code to get it working.
LLM & InferenceMarkdown .mdOpen on GitHub ↵

Alternatives to litellm

All LLM & Inference →
ollama avatarollamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.★ 187khuggingface avatartransformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.★ 169kopen-webui avataropen-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)★ 158klangflow-ai avatarlangflowLangflow is a powerful tool for building and deploying AI-powered agents and workflows.★ 157kvllm-project avatarvllmA high-throughput and memory-efficient inference and serving engine for LLMs★ 96.8klobehub avatarlobehub🤯 LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.★ 85k

Pairs well with

langchain-ai avatarlangchainThe agent engineering platform.★ 149kvercel avataraiThe AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents★ 27.9kSignificant-Gravitas avatarAutoGPTAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.★ 188kOpenHands avatarOpenHands🙌 OpenHands: AI-Driven Development★ 93.2kunslothai avatarunslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.★ 79.3klangchain-ai avatarlanggraphBuild resilient agents.★ 44.8k

FAQ

What is litellm?

BerriAI/litellm is a LLM tool on GitHub: The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] Open Source AI Gateway for 100+ LLMs.

When should I use litellm?

Reach for litellm when you need to call, serve, run locally, fine-tune or chat with a language model — it's written in Python and fits stacks tagged ai-gateway, anthropic, azure-openai, bedrock.

What are the best alternatives to litellm?

Popular alternatives in LLM & Inference: ollama/ollama (187k★), huggingface/transformers (169k★), open-webui/open-webui (158k★), langflow-ai/langflow (157k★), vllm-project/vllm (96.8k★).

Is litellm actively maintained?

BerriAI/litellm looks very actively maintained: the last push was today (2026-11-11), with 63,113 stars and 5,141 open issues.

What license does litellm use?

BerriAI/litellm is published under the Other license. Check the repository's LICENSE file before commercial use.