Determinism¶

Byte-exact reproducibility is a first-class invariant. A (global_seed, shard_id, sample_idx, emitter_idx, rx_idx, stage) tuple uniquely identifies an RNG draw; re-running with the same config and seeds produces byte-identical IQ and structurally identical metadata.

This page is normative. Implementations must follow this seed-derivation flow exactly.

Canonical attestation statements¶

Layer 14 extends the byte-identity boundary to signed provenance. Before SigstoreDSSEAdapter signs or accepts an in-toto Statement v1, it requires the supplied UTF-8 payload bytes to equal their RFC 8785 canonical JSON representation. Whitespace, key-order, or other semantically equivalent alternate encodings are rejected. The resulting signature therefore binds one statement byte representation, not only one parsed JSON value. See Attestation for the identity-policy and signature checks.

Why determinism matters¶

Need	Property
Distributed generation	Spark workers can re-run failed shards without coordination
Partial recomputation	Resume after interruption skips already-written shards
Bug investigation	Reproduce a single sample on a laptop from `(seed, shard_id, idx)`
Differential testing	Compare two implementation revisions byte-by-byte over the same dataset
CI smoke tests	Release gating on bit-identical reproduction of a baseline dataset

The only operations we accept as non-deterministic:

LLM completions (Phase 2). Prompts are deterministic; the LLM is not. We measure stability via PAES on the verifier subset.
GPU floating-point reductions. Sionna RT / PHY realizations may differ across devices due to backend reduction order; we treat small amplitude differences as below the relevant noise floor. For strict CI checks, use the deterministic controls provided by the active backend.

Everything else is byte-exact.

Public surface¶

The rfgen.rng module exports exactly three callables:

from rfgen.rng import derive_rng, seed_for, torch_to_numpy_rng

derive_rng(parent, *path) splits a parent torch.Generator into a child generator deterministically. seed_for(global_seed, shard_id, sample_idx, ...) collapses the canonical hierarchy into a 64-bit integer. torch_to_numpy_rng(rng) bridges a torch generator into a local NumPy generator without touching global state.

The shipped planner class used by orchestration is SeedSchedule, defined in rfgen.rng and re-exported from rfgen.orchestration for backward compatibility. Earlier draft helpers (derive_shard_seed, derive_sample_seed, split_for_layer, _hash_int64) are not part of the contract.

`derive_rng`¶

def derive_rng(parent: torch.Generator, *path: int | str) -> torch.Generator

Returns a fresh torch.Generator whose seed is a deterministic function of parent.initial_seed() and path. Identical (parent_seed, path) inputs always return generators that emit identical streams. The child runs on the same device as parent (CPU or CUDA).

Path components may be int or str and are mixed freely. Strings are canonicalized to a stable 64-bit prefix before entering the split (see How the split works).

`seed_for`¶

def seed_for(
    global_seed: int,
    shard_id: str,
    sample_idx: int,
    emitter_idx: int = 0,
    rx_idx: int = 0,
    stage: str | int = 0,
) -> int

Returns a deterministic non-negative 63-bit integer (suitable for any signed-int64 seed slot) derived from the canonical 5-component hierarchy (shard_id, sample_idx, emitter_idx, rx_idx, stage) rooted at global_seed. The 3-argument call shape seed_for(global_seed, shard_id, sample_idx) still works because emitter_idx, rx_idx, and stage default to 0.

stage accepts either a string (e.g., "tx", "channel", "post_sum") or an integer; both are canonicalized to a 64-bit prefix before the split.

`torch_to_numpy_rng`¶

def torch_to_numpy_rng(rng: torch.Generator) -> np.random.Generator

Returns a fresh local numpy.random.Generator deterministically derived from rng. The helper draws one seed from the inclusive range [0, 2**63 - 1] by combining two torch.randint(low=0, high=2**32, ...) 32-bit words and clearing the top bit. This preserves the framework-wide signed-int64 seed width while keeping NumPy and SciPy consumers on the same per-sample RNG path as torch-backed code.

The seed hierarchy¶

global_seed (int, from config.run.seed)
    ↓
shard_id (str)            ← content-addressed shard identifier
    ↓
sample_idx (int)          ← 0-based index within the shard
    ↓
emitter_idx (int)         ← 0-based emitter index within the sample
    ↓
rx_idx (int)              ← 0-based receiver index within the sample
    ↓
stage (str | int)         ← pipeline stage tag: "tx", "channel", "post_sum", ...
    ↓
seed_for(...) → int63     ← consumed by torch.Generator / numpy.Generator

Every layer that draws randomness gets its own independent stream. Two draws with different (emitter_idx, rx_idx, stage) paths cannot collide.

How the split works¶

The hierarchical split is delegated to numpy.random.SeedSequence.spawn: the canonical, tested, collision-free hierarchical split shipped by NumPy. The collision-free guarantee inherits from NumPy’s documented SeedSequence.spawn design (see the NumPy _seed_sequence.pyx source and its public docs).

import numpy as np
from rfgen.rng import seed_for

seed = seed_for(
    global_seed=1337,
    shard_id="cfg-abcdef0123456789-shard-000042",
    sample_idx=17,
    emitter_idx=2,
    rx_idx=0,
    stage="channel",
)
# seed is a non-negative int that fits in a signed 64-bit slot

hashlib.sha256 is used only to canonicalize string-valued inputs (e.g., shard_id, a string-valued stage) into a stable 64-bit integer prefix consumed by SeedSequence. It is never used to perform the split itself.

Two draft helpers from earlier revisions are gone: there is no BLAKE2b-based _hash_int64, and there are no derive_shard_seed / derive_sample_seed / split_for_layer functions. All of that responsibility now sits behind seed_for and SeedSequence.spawn.

Global-state isolation¶

The rfgen.rng module never reads from random or numpy.random global state, and never seeds them either. Every RNG it produces is a local object: a torch.Generator from derive_rng, or an integer seed from seed_for that the caller passes into a freshly constructed RNG.

A contract test snapshots both global states (Python stdlib random.getstate() and numpy.random.get_state()) before and after derive_rng / seed_for calls and asserts they are unchanged.

Collision contract¶

seed_for is collision-free across the documented hierarchy in the practical regime the framework operates in. The contract test draws N = 10**6 random paths from the hierarchy (shard_id, sample_idx, emitter_idx, rx_idx, stage), with each component sampled uniformly from a documented integer range, calls seed_for(...) for each, and asserts len(set(seeds)) == N (zero collisions in 1M draws).

The test is gated behind @pytest.mark.slow and runs in nightly CI rather than on every PR. The stronger >2^32 claim is not asserted by exhaustive enumeration; it cites NumPy’s SeedSequence documentation.

Round-trip contract¶

Running twice from seed_for(...) produces byte-identical 1024-sample float32 IQ tensors:

from rfgen.rng import seed_for
import torch

def draw_iq(seed: int) -> torch.Tensor:
    g = torch.Generator().manual_seed(seed)
    return torch.randn(1024, generator=g, dtype=torch.float32)

s = seed_for(global_seed=1337, shard_id="shard-0", sample_idx=0)
assert torch.equal(draw_iq(s), draw_iq(s))

This holds modulo the documented backend nondeterminism caveats for Sionna and GPU reductions (see GPU non-determinism caveat below).

Per-layer RNG flow¶

A complete sample lifecycle, showing every RNG creation:

from rfgen.rng import derive_rng, seed_for

def generate_sample(global_seed: int, shard_id: str, sample_idx: int,
                    config: GenerationConfig) -> Record:
    sample_seed = seed_for(global_seed, shard_id, sample_idx)
    sample_rng = torch.Generator().manual_seed(sample_seed)

    component_signals = []
    for slot_idx in range(num_emitters):
        slot_rng = derive_rng(sample_rng, "slot", slot_idx)

        # Per-slot emitter selection, device draw, waveform synthesis, and
        # placement all branch off the slot-local child stream.
        emitter_rng = derive_rng(slot_rng, "emit")
        tx_rng = derive_rng(slot_rng, "tx")
        freq_rng = derive_rng(slot_rng, "freq")
        time_rng = derive_rng(slot_rng, "time")
        signal = compose_one_slot(
            emitter_rng=emitter_rng,
            tx_rng=tx_rng,
            freq_rng=freq_rng,
            time_rng=time_rng,
        )

        for rx_idx in range(num_rx):
            channel_rng = derive_rng(slot_rng, "channel", rx_idx)
            signal = channel.apply(signal, rng=channel_rng, params=...)
        component_signals.append(signal)

    # Sum + per-receiver post-sum channel stages
    for rx_idx in range(num_rx):
        rx_rng = derive_rng(sample_rng, "rx", rx_idx)
        scene_prop_rng = derive_rng(rx_rng, "scene_prop")
        post_sum_rng = derive_rng(rx_rng, "post_sum")
        scene_signal = apply_rx_post_sum(
            component_signals,
            scene_prop_rng=scene_prop_rng,
            post_sum_rng=post_sum_rng,
        )

    # Labeling: pure, no RNG
    labels = labeler.label(scene_signal, config.scene)

    return Record(iq=scene_signal.iq, ..., labels=labels)

Key invariants:

Every child stream is named and positional: slot-local work hangs off derive_rng(sample_rng, "slot", slot_idx), then branches again by "emit", "tx", "freq", "time", and "channel", rx_idx.
Post-sum receiver work hangs off derive_rng(sample_rng, "rx", rx_idx), then branches into "scene_prop" and "post_sum".
Re-ordering emitters within a scene would break determinism because slot_idx is positional. The composer stabilizes that order after placement.
Labeler is pure: same (scene_signal, scene) produces same labels, no RNG.

RNG type per layer¶

Layer	RNG type	Why
Scene composer	`numpy.random.Generator` via path-keyed child derivation (`derive_rng(...)` -> `np.random.default_rng(child.initial_seed())`)	NumPy is faster for the scalar / small-array draws (slot count, class samples, placements), and the child-derivation path keeps the parent torch stream position unchanged
Emitter	`torch.Generator(...).manual_seed(seed)`	Most emitter waveforms run on torch (TorchSig); torch RNG matches
Channel	`torch.Generator` for stochastic; Sionna’s seeded API for RT/PHY	Sionna accepts an int seed; we pass it through
Labeler	none	Pure function

A layer that needs a NumPy Generator but must not advance the parent torch stream derives a child first, then seeds NumPy from that child’s initial_seed():

import numpy as np

scene_rng = derive_rng(sample_rng, "scene")
np_rng = np.random.default_rng(int(scene_rng.initial_seed()))

Use torch_to_numpy_rng(...) only for the consuming bridge case, where advancing an already-derived torch generator is part of the contract:

channel_rng = derive_rng(sample_rng, "channel", emitter_idx)
np_rng = torch_to_numpy_rng(channel_rng)

What breaks determinism¶

These pitfalls produce non-deterministic output despite correct seeding. The framework guards against each:

Pitfall	Guard
Python `set` iteration order	Use `sorted(set(...))` everywhere a set is iterated
`dict` insertion order across processes	Python 3.7+ guarantees insertion order; we still sort dict iteration in seed-sensitive paths
Multi-threaded RNG sharing	Each layer creates its own RNG; never share `Generator` instances across threads
GPU floating-point reductions	Set `torch.use_deterministic_algorithms(True)` in CI; document the amplitude tolerance otherwise
`os.urandom` / `time.time()` accidentally as a seed	Banned in `rfgen/`; CI greps for these patterns and fails on hits
`shuffle()` without explicit RNG	Banned; lint check rejects `random.shuffle(x)` and `np.random.shuffle(x)` without explicit RNG
Hash-based code with `PYTHONHASHSEED` ambient	Set `PYTHONHASHSEED=0` in the Spark image and locally
Reading from `random.` or `numpy.random.` global state	The `rfgen.rng` module is forbidden from touching either; a contract test snapshots both before and after every public call

Reproducibility contract¶

Run-manifest provenance¶

rfgen.manifest defines the versioned run-manifest contract used to bind a reproduction attempt to its inputs. It is separate from the per-asset CBOR manifest described in Storage Layout. A major one run manifest is an RFC 8785 canonical JSON envelope with this shape:

{
  "schema": {"name": "rfgen.run-manifest", "major": 1, "minor": 0},
  "payload": {"...": "strict major-one fields"},
  "extensions": {"...": "optional forward-compatible JSON values"}
}

RunManifestPayloadV1 fields root_seed, seed_schedule_id, resolved_config_canonical_json, and resolved_config_sha256 identify the deterministic input path. The resolved configuration is UTF-8 RFC 8785 JSON bytes, represented as a JSON string on the wire; its digest is SHA-256 of those exact bytes. The payload also records the run and dataset UUIDs, UTC creation and update timestamps, source revision, sorted package and plugin provenance, asset provenance, ordered output objects, optional annotation, audit, and parent revisions, and the manifest lifecycle status.

manifest_sha256 is SHA-256 of RFC 8785 canonical bytes for the payload with only manifest_sha256 omitted. Thus a reader recomputes the digest after validating all known fields, rather than hashing the received whitespace or object-member order. Package and plugin entries are ordered by (normalized_name, version, entry_point); output objects are ordered by the UTF-8 bytes of key. Invalid ordering, digest mismatch, noncanonical resolved configuration, invalid SPDX asset licensing, and invalid lifecycle values are validation failures.

Major-one payload fields are closed. parse_manifest rejects unknown payload fields and an unsupported major version, while preserving unknown envelope members and extensions as JSON semantic trees for a newer producer. It preserves value kind, array order, and object members, not insignificant JSON whitespace or member order. A newer minor version requires that preservation; callers that set preserve_unknown=False reject it and any forward-compatible envelope data. RFC 8785 values outside the supported JSON number domain also fail parsing.

VerificationReportV1 records the revision, requested verification mode, UTC start/end times, ordered VerificationCheckV1 rows, and exact pass/fail/skip counts. Rows sort by (scope, key, code) and use the closed CheckCode and CheckStatus vocabularies. The packaged JSON Schemas for the run manifest and verification report let non-Python consumers validate the wire structure.

ManifestVerifier is read-only. METADATA validates the revision and schema while marking object checks as not requested. CHECKSUMS streams every declared object in 8 MiB chunks and records every missing or digest-mismatched object. REPRODUCE includes those checks and asks an injected generation-runtime capability to regenerate the 32 sample IDs with the smallest sha256(run_id || sample_id) values. Each reproduction result must provide fresh-process evidence; integer IQ is exact, while floating IQ requires both maximum absolute and relative-RMS error at most 1e-6. Metadata, resolved config, asset and plugin hashes, and sample IDs compare exactly. A missing requested capability fails closed rather than being treated as a skip.

When callers configure a persisted DSSE bundle, the verifier only loads it and uses the Layer 14 SigstoreDSSEAdapter with the RUN_MANIFEST_V1 policy. It does not sign, write a bundle, or republish the manifest.

Layers 12 and 13 define canonical provenance and local publication. Layer 15 defines the typed verification service; it does not define a generation CLI. Text annotations (Phase 2) are not bit-identical because LLM sampling is non-deterministic; the prompts are.

GPU non-determinism caveat¶

Sionna RT and Sionna PHY use GPU operators that may produce non-deterministic results due to floating-point reduction order:

parallel reductions and matrix multiplies on GPU produce ULP-level differences across runs
cuDNN algorithm autotuning means kernel choice varies

For strict reproducibility:

import torch
torch.use_deterministic_algorithms(True)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

This forces deterministic kernels at a 10 to 30 percent runtime cost. The framework enables this in CI and in the Spark image’s smoke-test mode but leaves it off in bulk-generation mode (the amplitude tolerance is below the relevant noise floor).