Agent tools
firecrawl
firecrawl/firecrawl
A developer API service that gives AI agents real-time web search, scraping, and crawling with LLM-ready structured output.
Topics
Related projects
Scrapegraph-ai32.7kA Python web-scraping library that uses LLMs and graph pipelines to turn websites and documents into structured data.
crw1.2kA fast Rust web scraping, crawling and search API with an MCP server that gives AI agents clean markdown and structured data.
crawl4ai87.1kAn open-source web crawler built for LLMs and agents that turns any website into clean, LLM-ready Markdown.
llm-scraper7kA TypeScript library that turns any webpage into typed structured data using LLMs on top of Playwright.
anakin3.9kA self-hosted web-scraping API that turns any website into LLM-ready markdown and structured JSON for RAG pipelines and AI agents.
spider2.8kA concurrency-first Rust crawling engine that streams pages locally or at scale for search and RAG feeds.