Getting Started

Why this exists

Every web archive has its own idea of an index. The Wayback Machine speaks CDX with a collapse grammar, Arquivo.pt speaks a slightly different CDX, Webarchiv Österreich answers CDXJ for one exact URL, Archive.today only has a Memento TimeMap, Common Crawl points into WARC files, Perma.cc is a REST API behind a key, and WebCite cannot list anything at all. Pulling one client per archive into a project means seven response shapes and seven ways to misread a date.

@agntn/archives puts ten providers behind one class shape. Same calls, same option object, same page type. The provider decides how to ask; you decide what to ask for.

Install

pnpm add @agntn/archives

First listing

import { createArchive, providers } from "@agntn/archives";

const archive = createArchive(providers.wayback());
const response = await archive.snapshots("example.com", { limit: 100 });

if (response.success) {
  for (const page of response.pages) {
    console.log(page.timestamp, page.snapshot);
  }
}

providers.wayback() returns a promise, and createArchive accepts it as is: the provider module is imported the first time the archive needs it, so a project that only ever asks the Wayback Machine never ships the other nine.

Every archive at once

const archive = createArchive(providers.all());
const response = await archive.snapshots("example.com");

response.pages; // merged, newest first, one shape
response._meta?.unsupportedProviders; // [{ provider: "webcite", reason: "WebCite has no list-by-domain API. …" }]

providers.all() fans the query out to Wayback, Arquivo.pt, Webarchiv Österreich, Archive.today, Common Crawl and WebCite in parallel and merges what comes back. A provider that has no endpoint for the call says so in _meta; a provider that fails names its error. Neither is silently dropped. The Timeline does exactly this call against the docs worker.

Read what a page said

const capture = await archive.getContent("https://example.com", { timestamp: "2002" });

capture.timestamp; // "2002-11-29T05:43:48Z", the capture it actually found
capture.mime; // "text/html"
capture.content; // the decoded body

Listing says when a page existed. Reading says what it contained. Reading captures covers how a timestamp resolves and which providers can serve bodies.

What ships

Each provider has its own page under Providers with the endpoint, the options, and the traps.

The three operations

Every archive instance answers the same three questions.

1. Which captures exist

const response = await archive.snapshots("example.com", { from: "2010", to: "2012" });
const pages = await archive.getPages("example.com"); // same call, throws instead of returning success: false

2. What did one capture contain

const response = await archive.content("https://example.com/about", { timestamp: "20150601" });
const capture = await archive.getContent("https://example.com/about"); // newest capture, throws on failure

3. What changed between two captures

import { diffArchivedContent } from "@agntn/archives";

const before = await archive.getContent("https://example.com", { timestamp: "2024" });
const after = await archive.getContent("https://example.com", { timestamp: "2026" });
const diff = diffArchivedContent(before, after);

diff.patch; // unified diff of the visible text
diff.additions; // 3
diff.deletions; // 8

The get* variants throw on failure; the plain variants return { success: false, error } and leave the decision to you.

One page shape

interface ArchivedPage {
  url: string; // original URL, as the archive recorded it
  timestamp: string; // ISO 8601
  snapshot: string; // direct link to the archived version
  _meta: Record<string, unknown>; // provider, status, digest, mime, …
}

_meta.provider names the source of every page in a merged listing. The rest of _meta is whatever the archive knew: Wayback and Arquivo.pt add status, Arquivo.pt and Webarchiv Österreich add mime, digest and length, Common Crawl adds collection, Memento adds the upstream archive host, Archive.today adds its hash.