Custom Providers

The contract

interface ArchiveProvider {
  name: string;
  slug?: string;
  readonly options?: ArchiveOptions;
  cacheKey?: (options?: ArchiveOptions) => string | undefined;
  snapshots: (domain: string, options?: ArchiveOptions) => Promise<ArchiveResponse>;
  content?: (url: string, options?: ArchiveContentOptions) => Promise<ArchiveContentResponse>;
}

snapshots is the only required method. content is optional: a provider that serves captures only through its own UI leaves it out, and the aggregator reports the missing method the same way it reports an explicit unsupported response.

Extend the base class

import { BaseProvider, type ArchiveOptions, type ArchiveResponse } from "@agntn/archives";

interface MyArchiveOptions extends ArchiveOptions {
  collection?: string;
}

export class MyArchiveProvider extends BaseProvider<MyArchiveOptions> {
  readonly name = "My Archive";
  readonly slug = "my-archive";

  /** Options that change the result set belong in the cache key. */
  override cacheKey(options?: Readonly<MyArchiveOptions>): string | undefined {
    const collection = options?.collection ?? this.options.collection;
    return collection ? `collection=${collection}` : undefined;
  }

  async snapshots(domain: string, requestOptions?: Readonly<MyArchiveOptions>): Promise<ArchiveResponse> {
    const options = await this.resolveOptions(requestOptions);
    try {
      const rows = await fetchMyIndex(domain, options); // your HTTP call, honouring options.timeout and options.signal
      return {
        success: true,
        pages: rows.map((row) => ({
          url: row.original,
          timestamp: new Date(row.captured).toISOString(),
          snapshot: `https://my-archive.example/${row.id}`,
          _meta: { provider: "my-archive", status: row.status },
        })),
        _meta: { source: "my-archive", provider: "my-archive" },
      };
    } catch (error) {
      return {
        success: false,
        pages: [],
        error: error instanceof Error ? error.message : String(error),
        _meta: { source: "my-archive", provider: "my-archive" },
      };
    }
  }
}

resolveOptions merges the factory options, the options of the call and the resolved config, in that order of precedence. this.options holds what the constructor received.

Three rules the built-in providers follow, and a review would ask of yours:

• Timestamps are ISO 8601. Convert at the boundary; the merged listing sorts by timestamp and the window filter parses it.
• _meta.provider on every page. A merged listing has no other way to say where a page came from.
• A missing capability is a response, not an exception. Return { success: false, unsupported: true, unsupportedReason } for a call your archive cannot make, so providers.all() can list it under unsupportedProviders instead of failing.

Serve bodies

Add content() when the archive can replay original responses:

async content(url: string, requestOptions?: Readonly<ArchiveContentOptions>): Promise<ArchiveContentResponse> {
  const options = await this.resolveContentOptions(requestOptions);
  const capture = await pickCapture(url, options.timestamp); // newest at or before, else closest after
  const body = await readBody(capture, options.maxBytes);
  return {
    success: true,
    content: {
      url: capture.original,
      timestamp: capture.iso,
      snapshot: capture.rawUrl,
      content: body.text,
      mime: body.mime,
      bytes: body.bytes,
      truncated: body.truncated,
      _meta: { provider: "my-archive" },
    },
    _meta: { source: "my-archive", provider: "my-archive" },
  };
}

Read from an endpoint that returns the original response, never a page that wraps it. The diff helper compares whatever content says, and a replay banner would show up as a change.

Use it

import { createArchive, providers } from "@agntn/archives";
import { MyArchiveProvider } from "./my-archive";

const archive = createArchive(new MyArchiveProvider({ collection: "news" }));
await archive.use(providers.wayback());

const response = await archive.snapshots("example.com");

An instance and a lazy Promise<ArchiveProvider> are both accepted, so a custom provider sits next to the built-in ones without a wrapper.

Where the built-ins live

src/providers/ holds one file per archive; wayback.ts is the template. The shared helpers in src/utils/ map CDX rows, select captures, read playback and WARC bodies, and build the response objects, so a new built-in provider is mostly the index mapping. src/providers/index.ts registers the lazy factory.