i ideas / repo launcher
vllm-project avatar

vllm-project/vllm

★ 96,755PythonApache-2.096

A high-throughput and memory-efficient inference and serving engine for LLMs

What it is

🔥 We have built a vLLM website to help you get started with vLLM. Please visit vllm.ai to learn more. For events, please visit vllm.ai/events to join us. Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has grown into one of the most active open-source AI projects built and maintained by a diverse community of many dozens of academic institutions and companies from over 2000 contributors.

When to use it

Reach for vllm when you need to call, serve, run locally, fine-tune or chat with a language model — it's written in Python and fits stacks tagged amd, blackwell, cuda, deepseek.

Category
LLM & Inference
Also in
—
Stars
96,755
Forks
24,689
Last push
today
Created
2023-02-09
Open issues
10,244
Website
vllm.ai
#amd#blackwell#cuda#deepseek#deepseek-v3#gpt#gpt-oss#inference#kimi#llama#llm#llm-serving#model-serving#moe

Quick start

$ git clone https://github.com/vllm-project/vllm.git

Hand it to your AI

Read https://ideas.asveas.com/r/vllm-project/vllm.md and help me add vllm-project/vllm to my project. Explain the setup steps for my stack and show the minimal code to get it working.
LLM & InferenceMarkdown .mdOpen on GitHub ↵

Alternatives to vllm

All LLM & Inference →
ollama avatarollamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.★ 187khuggingface avatartransformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.★ 169kopen-webui avataropen-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)★ 158klangflow-ai avatarlangflowLangflow is a powerful tool for building and deploying AI-powered agents and workflows.★ 157klobehub avatarlobehub🤯 LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.★ 85kunslothai avatarunslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.★ 79.3k

Pairs well with

Significant-Gravitas avatarAutoGPTAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.★ 188klanggenius avatardifyBuild Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.★ 161kOpenHands avatarOpenHands🙌 OpenHands: AI-Driven Development★ 93.2kmudler avatarLocalAILocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.★ 50.3ke2b-dev avatarE2BOpen-source, secure environment with real-world tools for enterprise-grade agents.★ 14.8kf avatarprompts.chatf.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.★ 177k

FAQ

What is vllm?

vllm-project/vllm is a LLM tool on GitHub: A high-throughput and memory-efficient inference and serving engine for LLMs 🔥 We have built a vLLM website to help you get started with vLLM.

When should I use vllm?

Reach for vllm when you need to call, serve, run locally, fine-tune or chat with a language model — it's written in Python and fits stacks tagged amd, blackwell, cuda, deepseek.

What are the best alternatives to vllm?

Popular alternatives in LLM & Inference: ollama/ollama (187k★), huggingface/transformers (169k★), open-webui/open-webui (158k★), langflow-ai/langflow (157k★), lobehub/lobehub (85k★).

Is vllm actively maintained?

vllm-project/vllm looks very actively maintained: the last push was today (2026-11-11), with 96,755 stars and 10,244 open issues.

What license does vllm use?

vllm-project/vllm is published under the Apache-2.0 license. Check the repository's LICENSE file before commercial use.