rfgen.manifest

rfgen.manifest owns the versioned, canonical JSON contract for run provenance. It validates and canonicalizes a persisted manifest, but does not choose a storage path or publish a manifest transaction. Layer 13 owns that publication workflow.

All major-one payload models are frozen Pydantic v2 models with unknown fields forbidden. The outer envelope deliberately preserves forward-compatible JSON values. RFC 8785 canonical bytes define the config and manifest digests; SPDX validation uses the installed SPDX licensing vocabulary.

Public constant

MANIFEST_SCHEMA_MAJOR

MANIFEST_SCHEMA_MAJOR: Literal[1] = 1 is the only schema-major version this module parses. parse_manifest raises UnsupportedSchemaVersionError when an envelope declares a different major.

Lifecycle enums

ManifestStatus

STAGING, COMMITTED, and ABORTED are the only run-manifest lifecycle states.

ObjectStatus

EXPECTED, STAGED, COMMITTED, MISSING, and CORRUPT are the observed persistence states for one declared output object.

Envelope and payload models

SchemaIdentity

Carries name, major, and minor. Major-one manifests require the exact name rfgen.run-manifest, a major of at least one, and a non-negative minor.

ManifestObjectV1

One output object with non-empty key and media_type, non-negative size_bytes, lowercase SHA-256 sha256, ObjectStatus status, and optional provider version, generation, and ETag fields.

PackageProvenanceV1

One installed package or plugin identity: canonical Python distribution normalized_name, non-empty version, and optional entry_point.

ManifestAssetV1

One content-addressed external asset with URI, lowercase SHA-256 digest, validated SPDX 2.3 license_spdx expression, and absolute license_uri.

RunManifestPayloadV1

The strict major-one payload contains run and dataset UUIDs, UTC RFC 3339 creation and update timestamps, ManifestStatus, canonical resolved-config bytes and their SHA-256, non-negative root seed, seed schedule ID, sorted package and plugin provenance, source revision, assets, ordered output objects, optional annotation/audit/parent revisions, and manifest_sha256.

Package and plugin rows sort by (normalized_name, version, entry_point); objects sort by UTF-8 key bytes. updated_at cannot precede created_at. The resolved configuration must already be RFC 8785 UTF-8 JSON and its digest must match. manifest_sha256 is the SHA-256 digest of canonical payload bytes with only that field omitted; construction fills it when absent and rejects a mismatch when supplied.

RunManifestV1

The public repository and workflow type alias for RunManifestPayloadV1. It does not introduce a second schema or model; the alias makes the persisted run manifest contract explicit at publication boundaries.

ManifestEnvelopeV1

The forward-compatible outer envelope contains SchemaIdentity schema, JSON-object payload, and optional JSON-object extensions. Unknown top-level members and extension values are retained as JSON semantic trees. Values outside RFC 8785’s JSON domain are rejected.

ParsedManifestV1

Pairs a validated ManifestEnvelopeV1 with its validated RunManifestPayloadV1.

Verification models

VerificationMode

The closed request depth is METADATA, CHECKSUMS, or REPRODUCE. The orchestrator that executes those depths is Manifest Verification; these models define the persistable report only.

CheckStatus

PASS, FAIL, and SKIP are the only individual-check statuses. A report overall is only PASS or FAIL.

CheckCode

The closed machine-readable codes are SCHEMA_VALID, HASH_VALID, OBJECT_PRESENT, OBJECT_HASH_VALID, REPRODUCE_IQ, REPRODUCE_METADATA, CAPABILITY_NOT_REQUESTED, and CAPABILITY_UNAVAILABLE.

VerificationCheckV1

One deterministic verification result: non-empty scope, a possibly empty key, closed code and status, and optional expected, observed, and machine-readable diagnostic_code values.

VerificationReportV1

Carries schema version one, a lowercase manifest revision, requested VerificationMode, UTC start/end timestamps, PASS/FAIL overall, ordered rows, and exact pass/fail/skip counts. Rows sort by (scope, key, code); overall passes exactly when no row fails.

Local manifest repository

ManifestRevision

@dataclass(frozen=True, slots=True)
class ManifestRevision:
    revision: str
    dataset_id: UUID
    manifest: RunManifestV1

An immutable handle for one staged, content-addressed manifest. revision is the lowercase SHA-256 self-hash of manifest; dataset_id is the dataset to which that manifest and its publication pointer belong.

ManifestRepository

class ManifestRepository:
    def __init__(self, root: str | Path, *, dataset_id: UUID) -> None
    def stage(self, manifest: RunManifestV1) -> ManifestRevision
    def publish(
        self, revision: ManifestRevision, *, expected_latest: str | None
    ) -> ManifestRevision
    def latest(self, dataset_id: UUID) -> ManifestRevision | None
    def get(self, revision: str) -> RunManifestV1

ManifestRepository provides immutable, local persistence for exactly one dataset_id. root must be a local directory path or a file:// URL (an empty root, a non-local URL, and a non-local file://host/... URL are rejected). Provider/object-store publication is intentionally deferred to Layer 17 because this Layer 13 contract requires a local conditional update.

stage(manifest) rejects a manifest whose dataset_id differs from the repository’s dataset, canonicalizes its version-one envelope, and writes it once at manifests/<revision>.json. Re-staging identical bytes is idempotent; different pre-existing bytes at that path are a storage error. Staging never changes LATEST.

publish(revision, expected_latest=...) first confirms that revision still matches its staged manifest and repository dataset, then advances manifests/LATEST only if its observed revision is exactly expected_latest. Use None when no revision has yet been published. On a stale expectation it raises CommitConflictError and leaves the visible LATEST revision unchanged. Publication uses a local exclusive lock, a same-directory fsynced temporary file, and atomic replacement.

latest(dataset_id) returns None for a different dataset, for no published revision, or if the pointed manifest belongs to another dataset; otherwise it returns its ManifestRevision. get(revision) loads and validates a staged lowercase SHA-256 revision and rejects a missing object or filename/self-hash mismatch.

Parsing, canonicalization, and schemas

canonical_manifest_bytes()

def canonical_manifest_bytes(
    payload: RunManifestPayloadV1, *, omit_hash: bool = False
) -> bytes

Returns RFC 8785 canonical bytes for a typed payload. With omit_hash=True, it excludes only manifest_sha256, which is the self-hash input.

parse_manifest()

def parse_manifest(
    raw: bytes, *, preserve_unknown: bool = True
) -> ParsedManifestV1[RunManifestPayloadV1]

Parses UTF-8 JSON into a validated major-one manifest. It rejects invalid JSON, unsupported schema majors through UnsupportedSchemaVersionError, and all other envelope or payload contract failures through ManifestValidationError. Those failures include unknown major-one payload fields, invalid strict primitive types, ordering or digest failures, and unsupported RFC 8785 values. It retains unknown envelope members and extensions when preserve_unknown=True; with False, it rejects newer minor versions and any forward-compatible envelope data.

load_run_manifest_schema()

def load_run_manifest_schema() -> dict[str, JsonValue]

Loads the packaged Draft 2020-12 schema for the public run-manifest envelope.

load_verification_report_schema()

def load_verification_report_schema() -> dict[str, JsonValue]

Loads the packaged Draft 2020-12 schema for a verification report.

See Also