Web source mapping

Central registry for [web:NNN] tags used in a2c ADRs and docs under docs/.

Every [web:NNN] in scoped markdown must have a matching ### web:NNN entry here. Every entry here must be referenced at least once in scoped docs. See ADR-0011 and rules/15-web-source-tags.md.

Entry format

### web:NNN — Short human label
- https://canonical.example/primary
- https://alternate.example/mirror   # optional; first bullet is canonical

Allocate IDs sequentially from web:001; do not skip numbers.

Standards and specs

web:001 — Keep a Changelog 1.1.0

• https://keepachangelog.com/en/1.1.0/

Web crawlers and focused fetching

web:002 — Building an ethical web crawler (AWS prescriptive guidance)

• https://docs.aws.amazon.com/prescriptive-guidance/latest/web-crawling-system-esg-data/building-crawler.html

web:003 — Best open-source web crawlers (Firecrawl overview)

• https://www.firecrawl.dev/blog/best-open-source-web-crawler

web:004 — Scrapy documentation

• https://docs.scrapy.org/en/latest/

web:005 — Crawl4AI documentation

• https://docs.crawl4ai.com/

web:006 — Web crawler system design overview

• https://www.hellointerview.com/learn/system-design/problem-breakdowns/web-crawler

web:007 — Open-source web crawlers (CrawlCrawl)

• https://crawlcrawl.com/blog/open-source-web-crawlers

web:008 — Best web crawlers for 2026 (DataImpulse)

• https://dataimpulse.com/blog/the-best-web-crawlers-for-2026/

web:009 — Web crawling tools (LibHunt topic)

• https://www.libhunt.com/topic/web-crawling

web:010 — Top web crawler tools (Scrapfly)

• https://scrapfly.io/blog/posts/top-web-crawler-tools