Replay runner

The replay Seamward runs for a repair proposal executes Seamward's bounded mapping in an isolated sandbox. It never runs your application. The replay runner closes that gap: it feeds the same fixtures to the code you will ship, on your machine or in your pipeline, and reports which fixtures passed.

Use it to answer one question before you approve or merge a fix: does my handler accept the payload shape that caused this incident?

Before you start

• A replay dataset. One is built when a repair proposal is requested for an incident. List an incident's datasets with seamward replays datasets INCIDENT_ID --connection CONNECTION_ID.
• A signed-in CLI with the report scope. seamward login requests replays:report unless you pass --read-only. A CLI authorized before the runner existed does not hold it; sign in again. Owners and admins can report a run; viewers cannot.
• Node.js 22 or later, and a handler module Node can import.

Declare the handler

The handler is one small module you write once per connection. It exports a function, as the default export or as handle, that takes a payload and says what your code did with it. Have it call the same function your webhook route or API client calls, so the replay exercises your real code.

// replay-handler.mjsimport { handleCandidateWebhook } from "./src/webhooks/candidate.js";
export default async function handle(payload) {  const candidate = await handleCandidateWebhook(payload);  return {    accepted: candidate !== null,    businessObjectType: "candidate",    output: candidate,  };}

The function may be synchronous or return a promise. It receives a fresh copy of the fixture input each time, and a second argument { fixtureId }. Keep it free of side effects you would not want repeated: a replay calls it once per fixture, and a fixture is not a real event.

If your handler is TypeScript, point --handler at the compiled module, or at a .ts file on a Node.js version that runs TypeScript directly.

Pull the dataset

seamward replays pull rds_example --connection int_example --workspace acme --out dataset.json

The file is a seamward.replay-dataset/1 document: the dataset's manifest hash, its fixture source, and for each fixture an input and the expected outcome. The command recomputes the manifest hash over the fixtures and writes the file only when it matches. Each download is recorded in the workspace audit log.

Fixture inputs are generated, never copied from your traffic. A shape dataset holds a placeholder for each field's type. A profile dataset holds a value in each field's recorded format and size, which is what lets a date parser or an email check run meaningfully. Replay explains the two sources.

Run it twice

git checkout mainseamward replays run --dataset dataset.json --handler ./replay-handler.mjs --label current --out current.json --deny-network
git checkout fix/provider-renameseamward replays run --dataset dataset.json --handler ./replay-handler.mjs --label candidate --out candidate.json --deny-network

run is local. It needs no sign-in and sends nothing anywhere. It runs your handler in a child process, compares each returned outcome with the fixture's expected outcome, and writes a seamward.replay-result/1 file.

• --label current is the code deployed now. On an incident's fixtures you expect it to fail, which shows the fixtures exercise the problem.
• --label candidate is your fix. You expect it to pass.
• --timeout-ms is the time allowed per fixture, from 100 to 120000 (default 5000).

The command exits 0 whenever the run completed, pass or fail. Read status and summary in its output:

{  "datasetId": "rds_example",  "label": "candidate",  "commitSha": "0b3f5a9c1d2e4f60718293a4b5c6d7e8f9012345",  "workingTreeDirty": false,  "networkDenied": true,  "status": "passed",  "summary": { "passed": 40, "failed": 0, "errors": 0 },  "file": "candidate.json"}

commitSha is the commit of the checkout the handler module is in, and workingTreeDirty is true when that checkout had uncommitted changes. Outside a git checkout commitSha is null.

What the result contains

For each fixture: its ID, an outcome of pass, fail or error, and for a failure up to ten mismatch paths with their kind, such as $.accepted and value. For an error: one of handler_threw, handler_timeout, or invalid_handler_output.

It contains nothing else about your data. The result format has no field for your handler's input, its output, its logs, or the text of an error it threw. Whatever your handler prints goes to your terminal and is not recorded.

Deny network access

A replay should not call a provider or write to a remote system. --deny-network runs the handler process with network access denied by the operating system:

Before any fixture runs, the handler process tries to connect to a local probe the runner listens on. If the connection succeeds, the command stops with replay_network_denial_failed and runs nothing. If the platform has no supported wrapper, or the wrapper cannot start (for example where unprivileged user namespaces are disabled), it stops with replay_network_denial_unavailable. It never falls back to running with network access.

Without the flag your handler has whatever access your shell has, and the result records networkDenied: false. Only network access is restricted: the handler can still read and write files, so point any database or queue it uses at a disposable target.

Submit the result

seamward replays submit --connection int_example --workspace acme --file current.jsonseamward replays submit --connection int_example --workspace acme --file candidate.json

Seamward checks that the result names this dataset and its manifest hash, that its digest matches, and that it reports every fixture exactly once. Submitting the same file again returns the first run.

In the incident's Repair tab, the runs appear under Your own handler, next to the sandbox validation, labelled Reported by your runner, with the commit and whether the runner denied network access.

What a reported run means

Seamward cannot rerun your handler, so it cannot confirm the outcomes. A reported run is evidence you supply for whoever reviews the repair:

• It is always shown as reported by your runner, never as verified by Seamward.
• It never changes a repair proposal's status and never approves one. Approval stays a decision your team records.
• It is bound to one dataset. A run on an older dataset is not shown against a newer proposal.
• A dataset holds at most 50 reported runs.

Troubleshooting

Next steps

• Replay: what the sandbox runs, the two fixture sources, and what a pass does not establish.
• CLI reference: every replay command and flag.
• Example application: a consumer with a declared replay handler.