Listing Captures

The call

import { createArchive, providers } from "@agntn/archives";

const archive = createArchive(providers.wayback());
const response = await archive.snapshots("example.com", { limit: 50 });

snapshots returns an ArchiveResponse. getPages is the same call that returns ArchivedPage[] and throws when the response is not a success. Use the first when a failed provider is a normal outcome, the second when it is a bug.

A target can be a bare domain or a full URL. Wayback, Arquivo.pt, OSZK Webarchívum, the New Zealand Web Archive, Archive-It and Common Crawl treat a domain as a prefix and list everything under it; Webarchiv Österreich, Vefsafn and Perma.cc look up one exact URL, so give them the page you mean.

A time window

await archive.snapshots("example.com", { from: "2019", to: "2019" }); // the whole of 2019
await archive.snapshots("example.com", { from: "2010-06-01", to: "20101231235959" });

from and to accept Wayback digits (YYYY through YYYYMMDDhhmmss) or ISO 8601 dates. Both edges are inclusive, and a partial stamp covers the period it names: to: "2019" reaches the last second of that year. Providers whose index can narrow the query do so; the rest are filtered after the fact, so the window holds when every archive is asked at once.

Limits

limit caps the pages one call returns. On a merged listing it caps the merged list, after every provider answered and the pages were sorted newest first. A provider that lists oldest first, which is what the Wayback CDX does by default, can contribute nothing to a short merged listing when other archives hold newer captures. Ask that provider on its own, or narrow the window, when you need its early history.

The latest captures first

Most indexes keep a page's history oldest first. So limit: 3 on Wayback hands you 2020, 2021 and 2022 for https://sive.rs/about. And the recent ones? Still in there, unread. Flip it:

await archive.snapshots("https://sive.rs/about", { limit: 3, order: "newest" });

Same page, now 2026, 2025 and 2024. The archive does the flipping, not your loop. Wayback gets a negative limit. Arquivo.pt, OSZK Webarchívum, the New Zealand Web Archive, Archive-It and Common Crawl get sort=reverse. Webarchiv Österreich and Vefsafn get reverse=true. Archive.today, Memento and Conifer send their whole list anyway, so it's sorted before the cap. Perma.cc is the weak spot. It sorts only the page its API sent.

order: "oldest" does the same from the other end. On a merged listing it also decides how the merge runs. And with newest, every archive sends its latest rows, so the top of the merge really is the newest capture anyone holds.

It takes one page, though. A CDX index files a domain by URL first and by time second. The "newest" end of example.com would just be the last page in the alphabet. So snapshots() throws instead of pretending. Write the page with its scheme, https://example.com/ for a home page. Or leave order out and narrow a domain with from and to.

Leave order out and nothing changes. A single provider answers in its own order, a merged listing runs newest first.

Every archive at once

const archive = createArchive(providers.all({ concurrency: 3, retries: 1 }));
const response = await archive.snapshots("example.com");

providers.all() queries Wayback, Arquivo.pt, Webarchiv Österreich, Vefsafn, OSZK Webarchívum, the New Zealand Web Archive, Archive.today, Common Crawl and WebCite. Memento is not in it because MemGator already aggregates several of those archives and would double the requests; Archive-It, Conifer and Perma.cc need a collection, a user, or a key before they can answer.

Pages that match on URL and timestamp across providers are deduplicated. Each page keeps _meta.provider, so a merged list can still be split by source. The landing panel does that with groupByProvider in the docs app.

Any archive can grow after creation:

const archive = createArchive(providers.wayback());
await archive.use(providers.archiveToday());
await archive.useAll([providers.commoncrawl(), providers.memento()]);

Performance options

They can be set on the factory, on createArchive, or per call; the nearest one wins.

When a provider cannot answer

Not every provider implements every operation. WebCite has no endpoint that lists a domain, Perma.cc lists only what your key can see. Instead of an empty array, such a provider returns success: false with unsupported: true and a reason.

In a merged response the unsupported providers always appear under _meta.unsupportedProviders. The flags at the top level follow stricter rules:

const response = await createArchive(providers.all()).snapshots("example.com");

if (!response.success && response.unsupported) {
  // nobody could even try
}
response._meta?.errors; // ["commoncrawl: [GET] … fetch failed"], when a provider failed

getPages throws UnsupportedOperationError, with the provider records attached, when the whole call was unsupported. That is a structural answer, distinct from a fetch that failed.

Caching

Every listing is cached in memory for seven days by default, keyed by provider, target and the options that change the result set. fromCache: true marks a replayed response. Turn it off per call with { cache: false }, or set a driver and TTL once through configuration.