Test Contracts¶

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, config field names, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

The test suite layout, what each contract test asserts, and the golden-set verification harness.

The test pyramid:

    ┌─────────────────────────────┐
    │  Golden-set verification    │  ← nightly; full coverage; gate releases
    │  (slow, comprehensive)      │
    └─────────────────────────────┘
    ┌─────────────────────────────┐
    │  Integration tests          │  ← PR-time small; nightly large
    │  (end-to-end, small)        │
    └─────────────────────────────┘
    ┌─────────────────────────────┐
    │  Contract tests             │  ← parametrized over plugins
    │  (per ABC, all impls)       │
    └─────────────────────────────┘
    ┌─────────────────────────────┐
    │  Unit tests                 │  ← per file
    │  (fast, isolated)           │
    └─────────────────────────────┘

Every PR runs unit + contract + small integration. Slow / GPU / Spark / HIL / annotator tests gate via pytest markers (see Reference / Build and CI § pyproject.toml).

Test directory layout¶

tests/
├── conftest.py                                # shared fixtures
├── unit/
│   ├── test_core_types.py
│   ├── test_core_rng.py
│   ├── test_core_registry.py
│   ├── emitters/
│   │   ├── test_torchsig_adapter.py
│   │   ├── test_radar_pulsed.py
│   │   ├── test_lora_css.py
│   │   ├── ...
│   ├── channels/
│   │   ├── test_awgn.py
│   │   ├── test_torchsig_impairments.py
│   │   ├── test_sionna_phy.py
│   │   ├── ...
│   ├── scene/
│   │   ├── test_default_composer.py
│   │   ├── test_freq_placement.py
│   │   ├── test_time_placement.py
│   │   ├── ...
│   ├── labels/
│   │   ├── test_bbox_derivation.py
│   │   ├── test_segmentation_rasterization.py
│   │   ├── test_taxonomy.py
│   │   ├── ...
│   ├── annotators/
│   │   ├── test_templater.py
│   │   ├── test_whitelist.py
│   │   ├── test_paes.py
│   │   ├── ...
│   └── storage/
│       ├── test_zarr_store.py
│       ├── test_webdataset_store.py
│       ├── test_hdf5_store.py
│       ├── test_sigmf_store.py
│
├── contract/                                  # parametrized over impls
│   ├── test_base_emitter_contract.py
│   ├── test_base_channel_contract.py
│   ├── test_base_scene_composer_contract.py
│   ├── test_base_labeler_contract.py
│   ├── test_base_annotator_contract.py
│   ├── test_base_store_contract.py
│   └── test_base_llm_client_contract.py
│
├── integration/
│   ├── test_quickstart.py                     # 100-sample end-to-end
│   ├── test_first_scene.py                    # 3-emitter end-to-end
│   ├── test_torchsig_roundtrip.py             # HDF5 byte-exact round trip
│   ├── test_zarr_phase2_append.py             # idempotent annotation
│   ├── test_dense_urban_md.py                 # full preset, 100 samples
│   └── test_reproducibility.py                # byte-exact re-run
│
├── golden/                                    # nightly only
│   ├── test_baseline_byte_exact.py            # CI smoke baseline
│   ├── test_per_emitter_golden.py             # per-family known IQ
│   ├── test_torchsig_compat_golden.py         # round-trip with TorchSig
│   └── test_paes_calibration.py               # tolerance bands stay tight
│
└── fixtures/
    ├── tiny_iq_samples.zarr                   # 10 known samples
    ├── reference_lora_payloads.json           # gr-lora_sdr cross-check
    ├── reference_adsb_frames.json             # pyModeS cross-check
    ├── mock_llm_responses.json
    └── golden_baseline_xs.zarr.tar.gz

Test family pages¶

Use this page for the test pyramid, directory layout, and coverage gates. Use the family pages for concrete fixtures and test files.

Test family page	Covers
Shared fixtures	Registry fixtures, RNG fixtures, and dataset temp paths.
Contract tests	Per-ABC contract files and required invariants.
Integration tests	Quickstart, TorchSig round-trip, reproducibility, and append paths.
Golden-set verification	Byte-exact, per-emitter, PAES, and cross-library gates.
Property-based tests	Hypothesis strategies and invariants.
Test execution	Marker policy and PR/full/nightly command sets.

Coverage targets¶

Layer	Min coverage
`core/`	95 %
`emitters/`	80 % (per family)
`channels/`	85 %
`scene/`	85 %
`labels/`	95 %
`annotators/`	80 %
`storage/`	90 %
`cli/`	60 % (CLI is mostly orchestration)
Overall	80 %

pytest --cov enforces this; CI fails on a drop below 80 % overall.