Listing Captures

The call

import { createArchive, providers } from "@agntn/archives";

const archive = createArchive(providers.wayback());
const response = await archive.snapshots("example.com", { limit: 50 });

snapshots returns an ArchiveResponse. getPages is the same call that returns ArchivedPage[] and throws when the response is not a success. Use the first when a failed provider is a normal outcome, the second when it is a bug.

A target can be a bare domain or a full URL. Wayback, Arquivo.pt, Archive-It and Common Crawl treat a domain as a prefix and list everything under it; Webarchiv Österreich and Perma.cc look up one exact URL, so give them the page you mean.

A time window

await archive.snapshots("example.com", { from: "2019", to: "2019" }); // the whole of 2019
await archive.snapshots("example.com", { from: "2010-06-01", to: "20101231235959" });

from and to accept Wayback digits (YYYY through YYYYMMDDhhmmss) or ISO 8601 dates. Both edges are inclusive, and a partial stamp covers the period it names: to: "2019" reaches the last second of that year. Providers whose index can narrow the query do so; the rest are filtered after the fact, so the window holds when every archive is asked at once.

Limits

limit caps the pages one call returns. On a merged listing it caps the merged list, after every provider answered and the pages were sorted newest first. A provider that lists oldest first, which is what the Wayback CDX does by default, can contribute nothing to a short merged listing when other archives hold newer captures. Ask that provider on its own, or narrow the window, when you need its early history.

Every archive at once

const archive = createArchive(providers.all({ concurrency: 3, timeout: 10_000, retries: 1 }));
const response = await archive.snapshots("example.com");

providers.all() queries Wayback, Arquivo.pt, Webarchiv Österreich, Archive.today, Common Crawl and WebCite. Memento is not in it because MemGator already aggregates several of those archives and would double the requests; Archive-It, Conifer and Perma.cc need a collection, a user, or a key before they can answer.

Pages that match on URL and timestamp across providers are deduplicated. Each page keeps _meta.provider, so a merged list can still be split by source. The landing panel does that with groupByProvider in the docs app.

Any archive can grow after creation:

const archive = createArchive(providers.wayback());
await archive.use(providers.archiveToday());
await archive.useAll([providers.commoncrawl(), providers.memento()]);

Performance options

They can be set on the factory, on createArchive, or per call; the nearest one wins.

When a provider cannot answer

Not every provider implements every operation. WebCite has no endpoint that lists a domain, Perma.cc lists only what your key can see. Instead of an empty array, such a provider returns success: false with unsupported: true and a reason.

In a merged response the unsupported providers always appear under _meta.unsupportedProviders. The flags at the top level follow stricter rules:

const response = await createArchive(providers.all()).snapshots("example.com");

if (!response.success && response.unsupported) {
  // nobody could even try
}
response._meta?.errors; // ["commoncrawl: [GET] … fetch failed"], when a provider failed

getPages throws UnsupportedOperationError, with the provider records attached, when the whole call was unsupported. That is a structural answer, distinct from a fetch that failed.

Caching

Every listing is cached in memory for seven days by default, keyed by provider, target and the options that change the result set. fromCache: true marks a replayed response. Turn it off per call with { cache: false }, or set a driver and TTL once through configuration.