Directory

Self-hosted, open-source web archiving tool that saves URLs as HTML, PDF, screenshot, WARC, readable text and media snapshots with a web UI and CLI

Nonprofit open repository of petabyte-scale web crawl data since 2008, published as WARC, WAT and WET files on AWS Open Data with URL and columnar indexes