Quality ContractsΒΆ

Warning

Pre-release / design spec. These quality pages describe proposed contracts. Reproducibility guarantees, metric definitions, and test gates may change before implementation and contract tests land.

Audience. Test authors, release engineers, reviewers, and researchers reproducing dataset generation.

Before you start. Read Reference / Schemas for the contracts these quality checks enforce.

What this section is not. This section is not a tutorial for running one job. See How-to guides.

Reference contracts for reproducibility, statistical audits, metrics, and test expectations.

Page

Contract

Determinism

Seed flow, reproducibility guarantees, and nondeterminism boundaries.

Metrics

Dataset, label, text, channel, and release-gate metrics.

Test Contracts

Test pyramid, directory layout, and coverage gates.

Test Fixtures

Shared pytest fixtures and discovery helpers.

Contract Tests

Per-ABC tests every registered implementation must satisfy.

Integration Tests

End-to-end small generation and append paths.

Golden-Set Verification

Byte-exact, per-emitter, PAES, and cross-library release gates.

Property-Based Tests

Hypothesis strategies and invariants.

Test Execution

Marker policy and PR/full/nightly command sets.