Skip to content
VB-OSVB-OSVB-OS Docs

Replay Your First Evaluation in VB-OS

Replay is the ability to reproduce a past evaluation and get an identical result. This is not best-effort reproducibility. It is an architectural guarantee.

Why Replay Matters

Section titled “Why Replay Matters”

Traditional decision systems cannot reproduce past decisions reliably. The rules change. The state changes. The context is lost. When an auditor asks “why was this payment approved six months ago?”, the answer is often a reconstruction, not a reproduction.

VB-OS solves this by preserving everything needed to reproduce the evaluation:

  • The boundary version: the compiled version that was active at evaluation time
  • The evidence: the exact workload that was submitted
  • The binary version: the evaluation engine binary (integrity-verified)
  • The audit artifact: integrity hashes for all components

Because the evaluation engine is deterministic (integer-only, no side effects, no randomness), replaying the same evidence against the same boundary with the same binary always produces the same result.

Running a Replay

Section titled “Running a Replay”

Use the evaluation_id from any past evaluation to replay it:

from vbos import VBOSClient


client = VBOSClient(api_key="YOUR_API_KEY_HERE")


replay_result = client.evaluations.replay(
    project_id="PROJECT_ID",
    evaluation_id="EVALUATION_ID",
)


print(replay_result.decision)  # Same as the original

Or via cURL:

Terminal window
curl -X POST https://api.vb-os.org/v1/projects/{project_id}/evaluations/{evaluation_id}/replay \
  -H "Authorization: Bearer YOUR_API_KEY_HERE"

How Replay Works

Section titled “How Replay Works”

When you request a replay:

  1. The system retrieves the original evaluation record
  2. It loads the boundary snapshot from immutable storage, not the live database
  3. It verifies the boundary snapshot hash matches the recorded hash
  4. It loads the original evidence from immutable storage
  5. It identifies the binary version used in the original evaluation
  6. It verifies the engine version hash matches the recorded hash
  7. It re-executes the evaluation with the original inputs
  8. It returns the result: identical to the original

Immutable Storage, Not Live State

Section titled “Immutable Storage, Not Live State”

Replay uses stored artifacts (immutable storage), never hot database state. This is critical: the live boundary may have been updated to version 5, but replay uses the exact version 2 snapshot that was active at evaluation time. The live evidence table may contain updated records, but replay uses the exact evidence that was submitted.

Integrity Verification

Section titled “Integrity Verification”

Before replay executes, integrity hashes are verified:

  • Boundary snapshot hash: ensures the boundary has not been tampered with
  • Engine version hash: ensures the evaluation engine is the same version

If any hash does not match, replay fails with an integrity error rather than producing a potentially incorrect result. This is fail-closed behavior: the system will not produce an unverifiable result.

Replay Executions

Section titled “Replay Executions”

Each replay creates a ReplayExecution record that captures:

  • The original evaluation ID
  • The replay result (which must match the original)
  • The result_match field: a boolean indicating whether the replay verdict matched the original evaluation’s verdict
  • The binary version used
  • The integrity verification status
  • Who initiated the replay and when

Replay executions are append-only: they are themselves immutable records.

Conformance Re-evaluation

Section titled “Conformance Re-evaluation”

When the original binary version is unavailable (e.g., after a binary upgrade cycle), conformance re-evaluation occurs: the replay runs against the current binary version and the result_match field tracks whether the outcome is consistent with the original. A mismatch does not indicate an error in the original evaluation. It indicates a behavioral difference between binary versions that requires investigation.

Use Cases

Section titled “Use Cases”

Audit and Compliance

Section titled “Audit and Compliance”

Regulators and auditors can replay evaluations to independently verify that a past authorization decision was correct given the evidence at the time. The replay produces the same result regardless of how the boundary or system has changed since then.

Incident Investigation

Section titled “Incident Investigation”

When investigating an incident, replay lets you reproduce the exact evaluation that authorized (or deferred) the action in question. You see the same evidence, the same boundary, and the same result: no reconstruction needed.

Regression Testing

Section titled “Regression Testing”

After updating a boundary, you can replay historical evaluations against the new version to understand how the change would have affected past decisions.

Next Steps

Section titled “Next Steps”