Wayback Machine

factory

providers.wayback()

index

CDX

reads bodies

yes

provider=all

included

Load it

import { createArchive, providers } from "@agntn/archives";

const archive = createArchive(providers.wayback());
const yearly = createArchive(providers.wayback({ collapse: "timestamp:4" }));
const okOnly = createArchive(providers.wayback({ filter: "statuscode:200" }));

Or import WaybackProvider from @agntn/archives and construct it yourself.

What it lists

snapshots("example.com") asks web.archive.org/cdx/search/cdx for example.com/*, so a domain lists everything under it. A full URL is looked up as is. Rows come back as original, timestamp and statuscode; _meta keeps the raw stamp of fourteen digits and the status.

collapse defaults to timestamp:4, one capture per year, which keeps a domain with millions of captures answerable. Pass collapse: "timestamp:8" for one per day, or collapse: "digest" for one per distinct body. filter is passed through untouched (statuscode:200, mimetype:text/html, !statuscode:3..).

from and to narrow the CDX query itself, so a window is cheap here.

Reading bodies

const capture = await archive.getContent("https://example.com", { timestamp: "2002" });
capture.snapshot; // https://web.archive.org/web/20021129054348id_/http://www.example.com:80/

Bodies are read under the id_ modifier, which replays the original response without the toolbar or rewritten links. The capture is the newest at or before the timestamp, or the closest after it when nothing older exists. A snapshot URL from a listing works as the target too.

Gotchas

• The CDX index answers oldest first. A merged providers.all() listing sorted newest first can cut every Wayback row when other archives hold newer captures; ask Wayback alone for early history.
• CDX queries for busy domains take seconds, sometimes tens of seconds. Keep timeout generous and cache aggressively.
• collapse: "timestamp:4" is a default, not a fact about the archive. Drop it when you count captures.

Where it lives

src/providers/wayback.ts. CDX row mapping is shared with Archive-It and Common Crawl through mapCdxRows in src/utils/_utils.ts; id_ playback through readPlaybackCapture in src/utils/_content.ts.