i ideas / repo launcher
ggml-org avatar

ggml-org/llama.cpp

★ 134,082C++MIT93

LLM inference in C/C++

What it is

ggml / ops / maintainer PRs%20sort%3Aupdated-desc) / dev stats / lib llama API / llama-server REST API Visit https://llama.app and follow the instructions Run with Docker - see our Docker documentation Download pre-built binaries from the releases page Build from source by cloning this repository - check out our build guide

When to use it

Reach for llama.cpp when you need to call, serve, run locally, fine-tune or chat with a language model — it's written in C++ and fits stacks tagged ggml, llm, cpp.

Category
LLM & Inference
Also in
—
Stars
134,082
Forks
25,405
Last push
today
Created
2023-03-10
Open issues
2,600
Website
llama.app
#ggml#llm#cpp

Quick start

$ git clone https://github.com/ggml-org/llama.cpp.git

Hand it to your AI

Read https://ideas.asveas.com/r/ggml-org/llama.cpp.md and help me add ggml-org/llama.cpp to my project. Explain the setup steps for my stack and show the minimal code to get it working.
LLM & InferenceMarkdown .mdOpen on GitHub ↵

Alternatives to llama.cpp

All LLM & Inference →
ollama avatarollamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.★ 187khuggingface avatartransformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.★ 169kopen-webui avataropen-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)★ 158klangflow-ai avatarlangflowLangflow is a powerful tool for building and deploying AI-powered agents and workflows.★ 157kvllm-project avatarvllmA high-throughput and memory-efficient inference and serving engine for LLMs★ 96.8klobehub avatarlobehub🤯 LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.★ 85k

Pairs well with

Significant-Gravitas avatarAutoGPTAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.★ 188klangchain-ai avatarlangchainThe agent engineering platform.★ 149kOpenHands avatarOpenHands🙌 OpenHands: AI-Driven Development★ 93.2kunslothai avatarunslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.★ 79.3kBerriAI avatarlitellmThe fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]★ 63.1krun-llama avatarllama_indexLlamaIndex is the document processing platform for AI★ 52.9k

FAQ

What is llama.cpp?

ggml-org/llama.cpp is a LLM tool on GitHub: LLM inference in C/C++ ggml / ops / maintainer PRs%20sort%3Aupdated-desc) / dev stats / lib llama API / llama-server REST API Visit https://llama.app and follow the instructions Run with Docker - see our Docker documentation Download pre-built binaries from the releases page Build from source by cloning this repository - check out our build guide

When should I use llama.cpp?

Reach for llama.cpp when you need to call, serve, run locally, fine-tune or chat with a language model — it's written in C++ and fits stacks tagged ggml, llm, cpp.

What are the best alternatives to llama.cpp?

Popular alternatives in LLM & Inference: ollama/ollama (187k★), huggingface/transformers (169k★), open-webui/open-webui (158k★), langflow-ai/langflow (157k★), vllm-project/vllm (96.8k★).

Is llama.cpp actively maintained?

ggml-org/llama.cpp looks very actively maintained: the last push was today (2026-11-11), with 134,082 stars and 2,600 open issues.

What license does llama.cpp use?

ggml-org/llama.cpp is published under the MIT license. Check the repository's LICENSE file before commercial use.