Directory
Book Reviews
Posts
Tech Explore
Categories
Tags
agent-frontend-ux agentic-browser-automation Blogging Books code-first-agent-frameworks code-sandboxes distributed-runtimes document-parsing-pdf fine-tuning gpu-cloud-platforms graphrag-knowledge-graphs inference-infrastructure llm-programming-gateways llm-security-red-teaming llmops-observability local-runtimes-chat-uis managed-cloud-ai markdown-agents-assistants Meta ml-lifecycle-data-versioning Reviews scraping-crawling speech-voice-agents standards-protocols Tech Explore testing-evaluation vector-databases visual-agent-platforms workflow-orchestration
Table of Contents
Table of Contents
55 words
1 minute
Scrapy
Scrapy
Category: Scraping & Crawling · GitHub · Docs ⭐ 64,100 · 2.18.0 · BSD-3-Clause · snapshot 2026-08-31 Full profile (deep dive, contenders, matrix): [[AI Tools & Platforms Landscape]]
One-liner: Battle-tested Python crawling framework for large structured crawls
What problem does it solve?
How does it work?
When would I reach for it?
Related tools
- [[Crawlee]]
- [[Colly]]
- [[Crawl4AI]]
My exploration notes
Blog post checklist
- Outline
- Draft → Blog/posts/tech-explore/scrapy/
Some information may be outdated
Directory
Book Reviews
Posts
Tech Explore
Categories
Tags
agent-frontend-ux agentic-browser-automation Blogging Books code-first-agent-frameworks code-sandboxes distributed-runtimes document-parsing-pdf fine-tuning gpu-cloud-platforms graphrag-knowledge-graphs inference-infrastructure llm-programming-gateways llm-security-red-teaming llmops-observability local-runtimes-chat-uis managed-cloud-ai markdown-agents-assistants Meta ml-lifecycle-data-versioning Reviews scraping-crawling speech-voice-agents standards-protocols Tech Explore testing-evaluation vector-databases visual-agent-platforms workflow-orchestration
Table of Contents