Libraries & frameworks
scrapy-rotating-proxies
TeamHG-Memex/scrapy-rotating-proxies
A Scrapy downloader middleware that rotates a pool of proxies, detects dead or banned proxies, and adjusts crawling concurrency and retry backoff per proxy.
Topics
Related projects
http-proxy-middleware11.1kA Node.js HTTP proxy middleware that forwards requests to a target with a single configuration, working with Express, Connect, Next.js, Hono, Fastify and other server frameworks.
valtio10.2kValtio is a proxy-based state management library for React and vanilla JavaScript, offering mutable state objects, snapshot subscriptions and fine-grained re-rendering with minimal boilerplate.
scrapy-redis5.6kRedis-based components for Scrapy that enable distributed crawling and post-processing by letting multiple spiders share a single Redis queue.
ProxyManager4.9kA PHP library that generates and manages OOP proxy classes, enabling lazy loading, virtual proxies and other proxy patterns in application code.
spring-cloud-gateway4.9kAn API gateway framework built on Spring Boot offering dynamic routing, route-scoped filters and Spring Cloud discovery integration for microservice architectures.
chucker4.6kAn Android OkHttp interceptor library that records HTTP(S) requests and responses inside the app and provides a built-in UI for inspecting and sharing them.