Guide

Agent tests that hit the live model are flaky, paid, and nondeterministic — a temperature wobble or a provider hiccup turns a green suite red, and every CI run burns tokens. tapedeck records that call once against the real API, commits the result as a cassette, and replays it on every run after: deterministic, offline, free, and stream-accurate.

tapedeck is a Vercel AI SDK companion. It plugs in at the wrapLanguageModel middleware layer, so it is provider-agnostic and stream-aware by construction: no HTTP proxy, no mock to hand-write, no infra. Both language-model specs are typed, so the same middleware object drops into wrapLanguageModel under ai@6 (spec v3) and ai@7 (spec v4) with no cast. Switch behaviour with one env var; nothing else in your code changes.

Install

npm install -D @nkwib/tapedeck
# or
pnpm add -D @nkwib/tapedeck

Requires the ai peer (>=6.0.0 <8), so ai@6 and ai@7 are both supported. tapedeck has zero runtime dependencies beyond that peer: the provider packages are type-only dev dependencies.

Quickstart

Wrap your model once and read the mode from an env var. That is the whole integration:

import { openai } from '@ai-sdk/openai';
import { generateText, wrapLanguageModel } from 'ai';
import { cassetteMiddleware } from '@nkwib/tapedeck';

const model = wrapLanguageModel({
  model: openai('gpt-4o'),
  middleware: cassetteMiddleware({
    mode: process.env.CASSETTE_MODE ?? 'live', // record | replay | compare | live
    cassetteDir: './cassettes',
    redact: ['apiKey', 'authorization', /token/i],
  }),
});

// CASSETTE_MODE=record → hits the live API, writes a cassette.
// CASSETTE_MODE=replay → offline, deterministic, free.
const { text } = await generateText({ model, prompt: 'Say hi' });

The recommended workflow: live in development, record to capture a fixture once, replay in CI, and compare on a schedule to catch the model drifting away from the fixture. Run the test against the live API a single time, commit the cassette, and flip CI to replay.

Modes

In record mode the call flows out to the provider and the response is captured on the way back:

In replay mode the cassette is resolved by hash and served straight back — the provider is never touched:

Drift detection

A cassette is a static fixture, and a static fixture rots quietly: the provider retunes the model, the recorded trajectory stops being what the model actually does, and replay keeps passing because it never asks the model anything. compare is the check for that. It calls the live model, loads the recorded cassette, reports how the two diverged, and hands your code the live result. It never writes, so the cassette you committed is byte-identical after the run.

cassetteMiddleware({
  mode: 'compare',
  cassetteDir: './cassettes',
  onCompare(result) {
    if (!result.equal) reports.push(result); // fail the suite at the end
  },
});

Three signals, all explainable, with no similarity score to argue with:

result.equal is true when the trajectory and the finish reason match and the text is no worse than normalized: model prose is not a contract, the trajectory is. result.text.status is in every report, so a stricter policy is one onCompare handler away. The reasoning has its own decision page.

Drift never passes silently. onCompare fires once per compared call, drift or not; with no handler registered the first diverging call throws CassetteDriftError carrying the rendered report. That is the CI path, and tapedeck compare propagates the child process's exit code:

A cassette miss in compare mode is a miss, not a pass: it throws CassetteMissError exactly as replay does.

cassetteMiddleware

cassetteMiddleware(options?) returns a TapedeckMiddleware, which satisfies wrapLanguageModel under both supported ai majors. It intercepts both doGenerate (one-shot) and doStream (streaming).

tapedeck also exports lower-level primitives for direct use — hashing (computeCassetteHash, stableStringify, normalizeTools), cassette I/O (loadCassette / saveCassette, parseCassette / serializeCassette, isMultiCassette), drift comparison (compareCassetteResponses, summarizeResponse, formatCompareResult), diff/merge (diffCassettes, diffCassetteFiles, mergeCassetteDirs), storage (fileCassetteStore, memoryCassetteStore), telemetry (withSpan), and the constants CASSETTE_VERSION, MULTI_CASSETTE_VERSION, cassetteFilename(hash), REDACTED, DEFAULT_REDACT. See the API reference for all of them.

Spec v3 and v4

ai@6 takes a language-model spec v3 middleware; ai@7 takes spec v4. The two concrete types are mutually unassignable, so a middleware annotated with either one forces consumers of the other major to write as unknown as LanguageModelMiddleware at the call site. As of 0.4.0 tapedeck types the middleware boundary structurally instead: TapedeckMiddleware declares only the fields tapedeck genuinely reads (prompt, tools, the sampling params, the model's provider and modelId) and stays generic in the result type, so whatever doGenerate hands in is what wrapGenerate hands back.

import { wrapLanguageModel } from 'ai'; // 6 or 7, same code
import { cassetteMiddleware } from '@nkwib/tapedeck';

const model = wrapLanguageModel({
  model: openai('gpt-4o'),
  middleware: cassetteMiddleware({ mode: 'replay' }), // no cast under either major
});

Nothing about a request is interpreted, only hashed and serialized, so nothing is lost by describing the shape rather than importing the version. It is the same technique tapedeck already uses for the OTel tracer and the toolroute router. specificationVersion stays 'v3' because v4 hosts accept any string while v3 hosts accept only 'v3', and test/types.test-d.ts asserts assignability to both spec surfaces on every CI run, in both legs of the matrix. See Compatibility for the dated table.

Streaming

Streaming is first-class — not a non-goal. In record mode tapedeck drains the live stream, captures the ordered stream parts, and re-serves them so your code still receives the response. In replay mode the recorded parts are replayed as a genuine ReadableStream via the SDK's own simulateReadableStream, so streamText, UI message streams, and tool-call streaming all see the surface they would live.

import { streamText } from 'ai';

const { textStream } = await streamText({ model, prompt: 'Tell me a story' });
for await (const delta of textStream) process.stdout.write(delta);
// Identical output whether the model is live or replayed from a cassette.

Cassette format

Cassettes are pretty-printed JSON, keyed by a stable hash, designed to diff cleanly in PRs:

{
  "version": "tapedeck@0.1.0",
  "hash": "sha256:abc123…",
  "recordedAt": "2026-06-10T12:00:00Z",
  "request": {
    "modelProvider": "openai",
    "modelId": "gpt-4o",
    "prompt": [ ],
    "tools": [ ],
    "temperature": 0.7
  },
  "response": {
    "type": "stream",
    "chunks": [
      { "type": "text-delta", "id": "0", "delta": "I'll" },
      { "type": "text-delta", "id": "0", "delta": " help" },
      { "type": "tool-call", "toolCallId": "call_123", "toolName": "search", "input": "{\"query\":\"t-shirts\"}" }
    ]
  }
}

A one-shot generateText produces a "type": "generate" response holding the recorded content array, finish reason, and usage instead of chunks.

Named cassettes (from withCassette / cassetteName) use the v2 multi-interaction format: one file holding every call the test makes, keyed by hash — generate and stream interactions can mix freely:

{
  "version": "tapedeck@0.3.0",
  "recordedAt": "2026-06-10T12:00:00Z",
  "interactions": [
    { "hash": "sha256:abc…", "request": { }, "response": { "type": "generate" } },
    { "hash": "sha256:def…", "request": { }, "response": { "type": "stream", "chunks": [ ] } }
  ]
}

Legacy v1 single-interaction named cassettes still replay (served as-is); hash-addressed cassettes always use the single format.

Hash algorithm

The hash is a SHA-256 of the canonicalized, sorted JSON of:

{ modelProvider, modelId, prompt, toolSchemas, maxOutputTokens, temperature, topP }

Tool schemas are normalized (descriptions stripped, keys sorted) so cosmetic doc changes don't invalidate a cassette — but a changed prompt, tool input schema, or sampling param does. That is the point: a behavioural change fails CI loudly instead of replaying stale data.

Secret redaction

Redaction is key-name based and runs at record time, so secrets never reach disk:

• Default matchers: apiKey, authorization, x-api-key, bearer, token (case-insensitive).
• Configurable via redact: (string | RegExp)[] — strings match field / header names case-insensitively; RegExps test the raw key. Your matchers are merged with the built-in defaults.
• Replaying a cassette that still contains a value a matcher would strip throws CassetteSecretError — a committed secret fails the build instead of leaking.

cassetteMiddleware({
  mode: 'record',
  redact: ['apiKey', 'authorization', /secret/i],
});

Vitest helper

@nkwib/tapedeck/vitest exports withCassette(name, testFn, options?), which pins a test to a named cassette and forces replay mode for its duration:

import { describe, it, expect } from 'vitest';
import { withCassette } from '@nkwib/tapedeck/vitest';

describe('checkout agent', () => {
  it('runs the checkout flow', async () => {
    await withCassette('checkout-flow.json', async () => {
      const result = await runAgent({ prompt: 'buy a t-shirt' });
      expect(result.steps).toHaveLength(3);
    });
  });
});

Any cassetteMiddleware instance active inside the callback picks up the named cassette automatically — via an AsyncLocalStorage context — and tears down on exit, so there is no global setup/teardown to wire up. Pass options.mode to override the forced replay, or options.cassetteDir to point at a different directory.

The named cassette is multi-interaction: if the agent above makes three model calls, all three are recorded into checkout-flow.json keyed by request hash, and each call replays its own response — in any order. Re-recording a test starts the file fresh, so stale interactions never linger.

@nkwib/tapedeck/vitest also exports the toFollowRoute() matcher: pair with toolroute to assert that the replayed trajectory only makes transitions your router allows (expect(result.steps).toFollowRoute(router) after expect.extend({ toFollowRoute })).

CLI

The package ships a tapedeck bin for the record/replay workflow:

npx tapedeck record ./scripts/demo.mjs    # run with CASSETTE_MODE=record
npx tapedeck replay ./scripts/demo.mjs    # run with CASSETTE_MODE=replay
npx tapedeck compare pnpm test            # run with CASSETTE_MODE=compare (drift gate)
npx tapedeck record pnpm test             # non-file args run as commands on PATH

npx tapedeck ls ./cassettes               # kind, model, recordedAt per cassette
npx tapedeck diff a.json b.json           # semantic field-level diff (exit 1 on difference)
npx tapedeck merge ./from-ci ./cassettes  # merge directories; --force overwrites conflicts

record, replay, and compare all run the given script (or command on PATH) with CASSETTE_MODE set and propagate its exit code, so tapedeck compare pnpm test is a drift gate a scheduled job can fail on. diff pinpoints which fields diverged and ignores recordedAt; merge skips identical files and fails on conflicts unless --force is passed.

Telemetry

Pass any OTel-compatible tracer and every record/replay emits a span — typed structurally, so tapedeck keeps zero runtime dependencies:

import { trace } from '@opentelemetry/api';

cassetteMiddleware({ mode: 'replay', tracer: trace.getTracer('tapedeck') });

Spans (tapedeck.generate / tapedeck.stream) carry mode, hash, cassette path, model, hit/miss, and chunk-count attributes; a miss records the exception with an error status, so a failing CI replay shows up in traces.

Storage & edge runtimes

Cassette I/O goes through a CassetteStore (read/write/list). The default is the filesystem (loaded lazily); pass memoryCassetteStore() — or a KV/R2-backed store — on edge runtimes. The core never imports node:fs, node:path, or node:crypto statically; the one remaining Node builtin is node:async_hooks, which Cloudflare Workers provides under the nodejs_compat flag. See Compatibility for the caveats.

Errors

All extend CassetteError, so you can catch the whole family with one instanceof CassetteError.

Roadmap

Everything deferred from the first cut — OTel spans, the CLI, diff/merge tooling, the edge-safe core, the toFollowRoute() matcher, and multi-interaction named cassettes — has shipped as of 0.3.0. Still ahead:

• Deployed Cloudflare Workers smoke test in CI — edge support is designed-for, not yet CI-verified.
• Interaction-level merge for multi-cassettes (merge is file-level today).

See the changelog for the full release history.