Evidence and Release Qualification¶

The evidence system makes every claim about an rfgen dataset independently checkable after the fact, with no trust in whoever produced it. It ties a released dataset to the exact code, configuration, library versions, and validation results that produced it, using hashes and signatures. This page is the concept overview. Exact record fields and the validator contract are in Reference / Execution Evidence.

The problem: trusting a dataset¶

A generated RF dataset is opaque. In-phase and quadrature (IQ) samples and labels do not show, by inspection, whether the carrier was correct, whether a channel was real or mocked, whether a validation test was skipped, or whether a label was edited after generation. “The tests passed” is also not sufficient on its own, because tests can be skipped, run against a stub, or run on different code than the released artifact.

The evidence system removes those escape routes. To trust a dataset, rfgen traces it back to what produced it and whether that was validated: the implementation commit, the resolved configuration, the pinned library versions, and the validation results. That trace is recorded as hash-linked, signed records that a reviewer can recompute offline.

Cryptographic signing, in plain terms¶

Three primitives do the work, and this page uses them throughout.

Hash. A hash turns any file into a short, fixed fingerprint. A SHA-256 is 64 hex characters. Change one bit of the input and the fingerprint changes completely. rfgen stores every artifact by its hash, so a swapped or edited file is visible immediately.
Canonical bytes. Two JSON files can mean the same thing but differ as bytes, because of key order or whitespace. RFC 8785 defines one canonical byte form for any JSON value, so hashing it is reproducible by anyone, on any machine. rfgen hashes the canonical form.
Signature. A signature ties a fingerprint to an identity. The signer holds a private key, and anyone can check the signature with the signer’s public identity. A valid signature proves two things: who attested, and that the content was not changed after signing.

rfgen signs attestations with Sigstore. Sigstore replaces long-lived private keys with short-lived certificates bound to an identity, such as the CI job and commit that ran, and records them in a public, append-only log. A reviewer checks the log and the identity, with no prior shared key. See Reference / Attestation.

That is the whole toolkit. The rest of this page describes how rfgen combines hashes and signatures to make a dataset’s claims checkable.

The status ladder¶

In rfgen, a module is a named component of the framework, such as the Sigstore attestation component or the Zarr storage component. Every module climbs the same ladder. Each rung is a transition that a record must back.

planned. Named, with no code yet.
implemented. Code exists and produced acceptance artifacts.
engineering-validated. A real acceptance gate passed on the merged commit. “Real” means the validation ran against the actual library or process, not a stub or stand-in.
scientifically-validated. The scientific review passed. The review applies six lenses: construct validity, mathematical fidelity, empirical realism, literature grounding, experimental methodology, and robustness boundaries. Required only for modules whose evidence class is scientific.
release-qualified. The signed release was verified offline. Only the independent release verifier may record this rung.

Transitions are exact. No rung is skipped, reversed, or satisfied by a stub. Each module’s evidence class names the category of proof that backs it. The four classes are scientific, data-correctness, operational-reliability, and security-release. The class decides whether the science rung applies and what kinds of proof count.

The six records¶

Everything the system produces is one of six record kinds. Each is an immutable JSON object with a self-hash, defined in Reference / Execution Evidence.

Record	Proves	Written when
ModuleAcceptanceGateV1	A module’s tests passed on a pinned commit, on real, non-mock evidence	The acceptance suite runs and PASSes
ModuleStatusV1	A module’s current rung and its history	Each status transition
WorkstreamExecutionManifestV1	How independent work was partitioned into owners and scopes	When work is partitioned and merged
ChapterGateReportV1	A grouped set of required checks all passed	When those checks run
ScientificValidationHandoffV1	The frozen snapshot handed to scientific review	A scientific module enters review
ScientificValidationResultV1	The six-lens scientific outcome	The review completes

A module’s status records form a hash chain: each row points to its predecessor, so the chain is the module’s history. The records are hash-linked into one traversable graph rather than mutually cross-referenced. Each record references the workstream manifest and the underlying Git commit by hash, and the scientific handoff and result bind to the acceptance gate. That linkage is what lets a reviewer walk from a released dataset back to the evidence behind it.

Where records live and how they link¶

Evidence records are immutable JSON objects, content-addressed by their own self-hash: a record’s SHA-256 is its identity, and any change produces a different record. The run manifest, which binds a dataset to its implementation commit, resolved configuration, and pinned library versions, is persisted by the manifest repository at manifests/<sha>.json, written once with exclusive creation, and published by advancing a manifests/LATEST pointer under a lock. See Reference / Manifest.

At release time, a release-time component gathers every record into one signed release manifest, and the independent release verifier re-checks every hash and signature offline. Only that final verification can move a module to release-qualified.

Implementation status¶

The record format, the ExecutionEvidenceValidatorV1 semantic validator, and the rfgen evidence validate CLI are implemented and covered by contract tests. The core evidence machinery is landing incrementally. The signed-release step, including the release assembler and the independent verifier, is still in development, so no dataset has been release-qualified yet. Records are produced as components are validated; the signed-release step is still ahead.