Agent tools
Scrapegraph-ai
ScrapeGraphAI/Scrapegraph-ai
A Python web-scraping library that uses LLMs and graph pipelines to turn websites and documents into structured data.
Topics
Related projects
firecrawl199.5kA developer API service that gives AI agents real-time web search, scraping, and crawling with LLM-ready structured output.
crawl4ai87.1kAn open-source web crawler built for LLMs and agents that turns any website into clean, LLM-ready Markdown.
crw1.2kA fast Rust web scraping, crawling and search API with an MCP server that gives AI agents clean markdown and structured data.
spider2.8kA concurrency-first Rust crawling engine that streams pages locally or at scale for search and RAG feeds.
agent-memory4.3kA long-term memory runtime for AI agents that keeps Markdown as the source of truth, adds local ranked retrieval, and shares one store across Claude Code and Codex.
brightdata-mcp2.7kAn MCP server giving AI agents reliable web search, scraping, structured extraction, and browser automation at scale.