i ideas / repo launcher
unclecode avatar

unclecode/crawl4ai

★ 87,226PythonApache-2.095

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

What it is

Crawl4AI Cloud is a hosted web scraping, extraction, and search API for AI agents and data pipelines. Turn pages into clean Markdown for RAG, extract structured JSON, or search the live web with one API key. Crawl4AI runs the browsers and manages the proxies. Get a free API key · 1,000 free credits every month. No credit card required.

When to use it

Reach for crawl4ai when you need semantic search, embeddings, scraping or turning documents into LLM-ready data — it's written in Python and fits stacks tagged ai, ai-agents, crawl4ai-cloud, crawler.

Category
RAG, Search & Data
Also in
—
Stars
87,226
Forks
9,090
Last push
yesterday
Created
2024-05-09
Open issues
331
Website
crawl4ai.com/?ref=github-about
#ai#ai-agents#crawl4ai-cloud#crawler#data-extraction#llm#markdown#mcp#playwright#python#rag#scraper#web-crawling#scraping

Quick start

$ git clone https://github.com/unclecode/crawl4ai.git

Hand it to your AI

Read https://ideas.asveas.com/r/unclecode/crawl4ai.md and help me add unclecode/crawl4ai to my project. Explain the setup steps for my stack and show the minimal code to get it working.
RAG, Search & DataMarkdown .mdOpen on GitHub ↵

Alternatives to crawl4ai

All RAG, Search & Data →
firecrawl avatarfirecrawlSupercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥★ 201kinfiniflow avatarragflowRAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.★ 94kdocling-project avatardoclingGet your documents ready for gen AI★ 70.4kmem0ai avatarmem0The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.★ 68.8kmeilisearch avatarmeilisearchA lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.★ 59.8krun-llama avatarllama_indexLlamaIndex is the document processing platform for AI★ 52.9k

Pairs well with

Shubhamsaboo avatarawesome-llm-apps100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.★ 144kbrowser-use avatarbrowser-useOpen-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.★ 121kSignificant-Gravitas avatarAutoGPTAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.★ 188klanggenius avatardifyBuild Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.★ 161kopen-webui avataropen-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)★ 158kunslothai avatarunslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.★ 79.3k

FAQ

What is crawl4ai?

unclecode/crawl4ai is a retrieval tool on GitHub: Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it. Crawl4AI Cloud is a hosted web scraping, extraction, and search API for AI agents and data pipelines.

When should I use crawl4ai?

Reach for crawl4ai when you need semantic search, embeddings, scraping or turning documents into LLM-ready data — it's written in Python and fits stacks tagged ai, ai-agents, crawl4ai-cloud, crawler.

What are the best alternatives to crawl4ai?

Popular alternatives in RAG, Search & Data: firecrawl/firecrawl (201k★), infiniflow/ragflow (94k★), docling-project/docling (70.4k★), mem0ai/mem0 (68.8k★), meilisearch/meilisearch (59.8k★).

Is crawl4ai actively maintained?

unclecode/crawl4ai looks very actively maintained: the last push was yesterday (2026-11-10), with 87,226 stars and 331 open issues.

What license does crawl4ai use?

unclecode/crawl4ai is published under the Apache-2.0 license. Check the repository's LICENSE file before commercial use.