Providers
Providers
Every source, one class shape. Pick an archive to see its endpoint, its options and the traps.
Every provider answers the same calls: snapshots and, where the archive can replay original responses, content. What changes is the index it reads, the options it needs before it can answer, and what its _meta carries. The pages are short on purpose; the interesting part of each one is the gotcha list.
providers.all() covers the ones that need nothing from you. The rest need a collection, a user, a key, or would duplicate requests an aggregator already makes.
| Index | What it talks to | ||
|---|---|---|---|
| Wayback Machine | CDX | CDX API with collapse and filter. Bodies replayed under id_, the original bytes. | list · read · diff |
| Arquivo.pt | CDX | Public CDX index of the Portuguese web archive. Raw replay through noFrame/replay. | list · read · diff |
| Webarchiv Österreich | CDXJ | Austrian National Library CDXJ index. One exact URL per query, id_ replay for bodies. | list · read · diff |
| Vefsafn | CDX | Icelandic web archive at the National Library. One exact URL per query, id_ replay for bodies. | list · read · diff |
| OSZK Webarchívum | CDX | Hungarian web archive at the National Széchényi Library. Lists a whole domain, id_ replay for bodies. | list · read · diff |
| New Zealand Web Archive | CDX | New Zealand's national web archive. Lists a whole domain; the replay hides behind a bot check. | list only |
| Archive.today | Memento TimeMap | Memento TimeMap on archive.is. Bodies are the rendered wrapper page, not the original bytes. | list · read · diff |
| Common Crawl | CDX + WARC | CDX index per crawl, bodies read from the WARC byte range on data.commoncrawl.org. | list · read · diff |
| WebCite | none | No API to list a domain. Answers unsupported with the reason; no new archives since about 2019. | nothing to list |
| Memento | MemGator TimeMap | ODU MemGator JSON TimeMap across several archives. Outside all to avoid duplicate requests. | list · read · diff |
| Archive-It | CDX/C | CDX/C index of one numbered collection. Needs a collection; reads bodies inside it. | list · read · diff |
| Conifer | CDX | Search inside an existing public collection. Needs a user and a collection; no bodies. | list only |
| Perma.cc | REST | REST API behind an API key. Exact URL lookup, metadata only, what the key can see. | list only |