Core Types¶
Module: rfgen.core.types
The data types every layer of the framework speaks. They are frozen dataclasses
(frozen=True, slots=True) to prevent accidental mutation across component
boundaries.
A published record is assembled by
SDSObservationAdapter
from the projections a run declares; the rfgen.records package that composed
them was retired with the legacy record path. It is distinct from the
worker-local LabeledScene that scene
and label stages pass around.
rfgen.core.metadata.decode_rfgen_metadata(value)¶
def decode_rfgen_metadata(value: object) -> object: ...
Recursively decodes RFGen’s tagged representation of permitted non-finite
floating-point metadata after reading a native record. Mappings become
dict[str, object]; tuples and lists become lists; other values are returned
unchanged. A mapping whose only key is "__rfgen_float__" decodes "nan",
"inf", or "-inf" to the corresponding Python float. Any other value for
that exact tagged mapping raises ValueError("invalid RFGen tagged float").
LabeledScene is the worker-local
composed scene and carries raw I/Q. StoredRecord is its redacted projection,
and the redaction is structural rather than conventional: StoredRecord has no
IQ field at all, so a consumer typed against it cannot reach a waveform. Scene
evidence may measure a worker-local LabeledScene, but only the resulting
bounded evidence JSON — never the scene or its I/Q — may cross to a provider or
manifest.
The module ships ten frozen dataclasses (Signal, SignalMetadata,
SceneMetadata, StoredRecord, BBox,
ComposerSampleContext,
GeometryPose,
GeometryAssetRef,
GeometryProvenance) plus
four TypeAlias declarations (IQ, Spectrogram, SampleRate, CenterHz).
All ten dataclasses use frozen=True, slots=True.
ComposerSampleContext¶
ComposerSampleContext(sample_ordinal) carries only the stable zero-based
ordinal of one composition sample. Local, distributed,
and Dataproc generation supply the same ordinal independently of retry attempt;
the context deliberately contains no storage, executor, shard, or complete
configuration state.
Type aliases¶
IQ, Spectrogram, SampleRate, CenterHz¶
from typing import TypeAlias
import torch
IQ: TypeAlias = torch.Tensor
"""Complex baseband. Single-RX shape `(2, N)` float32 (channel-first real/imag);
multi-RX shape `(num_rx, 2, N)` float32. Channel 0 is in-phase; channel 1 is
quadrature. Stored as float32 rather than complex64 for autograd
compatibility. See [Reference / IQ Layout Policy](../data/iq-layout-policy.md)
for the full in-memory vs storage contract."""
Spectrogram: TypeAlias = torch.Tensor
"""Shape `(F, T)` float32, log-magnitude unless otherwise specified."""
SampleRate: TypeAlias = float
"""Sample rate in Hz."""
CenterHz: TypeAlias = float
"""Carrier or center frequency in Hz, absolute, for HIL alignment."""
These are TypeAlias declarations (not NewType). For worker-local and native
IQ shape rules, see Reference / IQ Layout Policy.
class BBox¶
@dataclass(frozen=True, slots=True)
class BBox:
start_sample: int # inclusive start in the parent record's sample frame
duration_samples: int # number of samples this bbox spans
low_freq_hz: float # Hz, baseband-relative
high_freq_hz: float
class_id: int # integer class identifier
emitter_index: int # index into the scene's per-emitter metadata tuple
Time-frequency rectangle for one accepted, placed event. Used in detection
and segmentation labels. emitter_index is the event’s index in the parent
scene’s emitters tuple; the field retains its historical name, but it is not a
generator-slot index. start_sample and duration_samples are in the parent
receiver record’s sample frame, not necessarily the component metadata’s
scene/channel frame. The labeler maps the component’s half-open interval
outward to that receiver grid and rejects an out-of-capture result; the
component metadata remains unchanged. See Concepts / Coordinate
Systems
and Scene Capture Boundaries. The
low_freq_hz and high_freq_hz edges are baseband-relative; the parent
record’s per-RX rate sets the absolute axis.
BBox round-trips losslessly to and from a normalized YOLO-style 5-tuple
(t_norm, f_norm, dt_norm, df_norm, class_id) for inputs in valid ranges via
BBox.to_yolo and BBox.from_yolo. For the on-disk normalization (YOLO / COCO
conventions) see Label Schema.
class GeometryPose¶
@dataclass(frozen=True, slots=True)
class GeometryPose:
position_m: tuple[float, float, float]
orientation_rad: tuple[float, float, float]
velocity_mps: tuple[float, float, float] = (0.0, 0.0, 0.0)
frame: str = "scene"
Typed 3D pose in the Sionna/Mitsuba scene frame. orientation_rad is
Euler angles under the pinned
rotation convention, that is
(roll, pitch, yaw) in radians about (x, y, z);
frame must be "scene" for v1 (__post_init__ raises
ValueError otherwise). velocity_mps feeds SionnaRT’s Doppler wiring:
a node constructed from a pose with nonzero velocity produces genuinely
time-varying channel taps when RTSolverConfig.cir_num_time_steps > 1.
The Pydantic config-layer mirror is
GeometryPoseConfig.
class GeometryAssetRef¶
@dataclass(frozen=True, slots=True)
class GeometryAssetRef:
kind: GeometryAssetKind
uri: str
content_hash: str
version: str | None = None
entrypoint: str | None = None
media_type: str | None = None
content_addressed: bool = True
metadata: dict[str, object] = field(default_factory=dict)
Content-addressed reference to a scene-geometry-related asset.
kind is a GeometryAssetKind
closed enum (Sionna built-in scene, Mitsuba XML bundle, material database,
antenna pattern, and related kinds). uri must start with file://,
gs://, s3://, https://, or sionna://builtin/. content_hash must
be sha256: followed by 64 hex characters. For
GeometryAssetKind.SIONNA_BUILTIN_SCENE, the uri suffix after
sionna://builtin/ names the Sionna built-in scene (e.g. munich),
resolved via sionna.rt.scene.<name>. For Mitsuba XML bundles,
entrypoint is the worker-local filesystem path SionnaRT passes
directly to sionna.rt.load_scene. content_addressed is True when
content_hash is a true digest of the asset’s bytes (the normal case for a
directly-constructed typed ref) and False when it is only a URI-identity
stand-in that cannot detect an in-place edit at a fixed URI (the legacy-URI
fallback for remote or unreadable assets). A False ref bypasses
SionnaRT’s scene cache and is loaded fresh on every solve.
class GeometryProvenance¶
@dataclass(frozen=True, slots=True)
class GeometryProvenance:
asset_refs: tuple[GeometryAssetRef, ...]
tx_pose: GeometryPose
rx_pose: GeometryPose
tx_array_id: str | None
rx_array_id: str | None
material_db_hash: str | None
antenna_pattern_hash: str | None
solver_backend: str
sionna_version: str
mitsuba_version: str
drjit_version: str
Geometry provenance recorded on RT-generated component metadata.
SionnaRT.apply populates this on every real solve: asset_refs is the
resolved scene/material/antenna asset refs; tx_pose/rx_pose are the
typed poses actually used; solver_backend is "sionna-rt"; and
sionna_version/mitsuba_version/drjit_version are the installed
library versions (__post_init__ requires all three non-empty when
solver_backend == "sionna-rt"). Lives on
SignalMetadata.geometry for single-RX RT scenes and is threaded through
SceneMetadata.extras for joint multi-RX RT scenes.
class SignalMetadata¶
@dataclass(frozen=True, slots=True)
class SignalMetadata:
family: str
class_name: str
class_taxonomy: tuple[str, ...]
generator_name: str
device_id: str | None
sample_rate_hz: float
bandwidth_hz: float
realized_carrier_hz: float
start_sample: int
duration_samples: int
snr_db: float
tx_pose: GeometryPose | None = None
tx_power_dbm: float | None = None
tx_array_id: str | None = None
geometry: GeometryProvenance | None = None
extras: dict[str, object] = field(default_factory=dict)
schema_version: int = 1
Per-signal ground truth. Carried by every Signal: in a composed scene, there is one component metadata object for every accepted, placed event. This cardinality is event based, not generator based: one selected generator slot can yield zero events or several events. Populated by the producing layer; immutable after.
Fields¶
Field |
Type |
Set by |
Purpose |
|---|---|---|---|
|
str |
emitter |
Top-level emitter family, e.g. |
|
str |
emitter |
Specific class within the family, e.g. |
|
tuple[str, …] |
emitter |
Hierarchical path, e.g. |
|
str |
emitter |
Concrete class name of the producing emitter |
|
str | None |
scene composer |
Stable virtual-device identifier for fingerprinting |
|
float |
emitter |
Sample rate the IQ was synthesized at |
|
float |
emitter |
Occupied bandwidth |
|
float |
emitter for a direct output; scene composer for a placed component |
Absolute carrier frequency in Hz. During composed generation, the composer writes |
|
int |
scene composer |
Inclusive absolute start in the placed scene/channel buffer; not an emitter-native index. It remains component provenance when a per-RX label uses another rate. |
|
int |
emitter / scene composer |
Number of samples this signal occupies in its current frame; placed components use the scene/channel rate. Bboxes map the half-open component interval to the active receiver grid without rewriting this field. |
|
float |
channel |
|
|
GeometryPose | None |
scene |
Transmitter pose in the scene frame; |
|
float | None |
scene |
Realized transmit power; |
|
str | None |
scene |
Transmit antenna-array identifier within the antenna blob |
|
GeometryProvenance | None |
channel |
Solver backend and asset references, stamped by a ray-traced solve |
|
dict[str, object] |
any |
Backend-specific extras. The on-disk schema preserves these |
|
int |
constructor |
Storage schema version. Default |
Per-impairment values live in extras¶
Per-device fingerprint values (CFO, IQ imbalance, PA, phase noise) are not
inline fields. They are threaded through
SignalMetadata.extras["fingerprint_params"] as a dict whose keys are
documented in rfgen.core.protocols.FINGERPRINT_PARAM_KEYS:
("cfo_hz", "iq_imbalance_db", "iq_imbalance_rad",
"pa_p", "pa_a", "phase_noise_dbc_hz", "pa_model",
"alpha_a", "beta_a", "alpha_phi", "beta_phi")
The device-fingerprint module produces these values from FingerprintParams
(Pydantic v2). tx-impairments and rx-frontend transformations consume them
only via these documented keys, so a device-fingerprint field rename is
caught by a contract test rather than silently changing the shape
scene-composer threads through.
Notes¶
Field names mirror TorchSig where applicable. Hz-suffixed and sample-suffixed variants are used here for unambiguous units; both forms exist in the on-disk schema. See TorchSig Interop.
Carrier is absolute after placement. A direct emitter call may return native/provisional carrier metadata. In composed generation, the scene composer replaces it after placement with the absolute RF carrier (for example,
2.412e9for Wi-Fi channel 1). Stored scene metadata has no duplicate RF anchor; placed-component and per-RX frames meet at the BaseMixerStage step on the receiver’s capture plane (ReceiverStagePlane.CAPTURE).
class SceneMetadata¶
@dataclass(frozen=True, slots=True)
class SceneMetadata:
scene_id: str
duration_s: float
num_emitters: int
num_rx: int
rx_index: int | None
rx_pose: GeometryPose | None
rx_antenna_id: str | None
scene_geometry_hash: str | None
material_db_hash: str | None
antenna_pattern_hash: str | None
geometry_asset_refs: tuple[GeometryAssetRef, ...]
channel_realization_seed: int | None
realized_emitter_count: int
realized_snr_db_stats: dict[str, float]
realized_class_histogram: dict[str, int]
realized_cochannel_overlap_rate: float
realized_spectral_occupancy: float
extras: dict[str, object] = field(default_factory=dict)
schema_version: int = 1
Scene-level ground truth, with per-RX context. One per LabeledScene. SceneMetadata deliberately does not expose center_freq_hz or bandwidth_hz at the scene level (the no-shared-scene-RF-anchor decision); RF context lives on per-signal SignalMetadata, on the typed rx_pose, and on the resolved geometry-asset refs preserved for RT and Mitsuba-backed scenes.
Fields¶
Field |
Type |
Purpose |
|---|---|---|
|
str |
Stable identifier shared across records produced from one scene |
|
float |
Scene duration in seconds |
|
int |
Realized emitter count |
|
int |
Number of receivers in the parent scene |
|
int | None |
Index of this RX within the parent scene; |
|
|
Typed RX pose in scene-frame coordinates; populated for single-RX records whenever the resolved receiver carries a pose, |
|
str | None |
Antenna pattern identifier within the antenna-pattern blob |
|
str | None |
SHA-256 of the active scene-geometry asset, typically a Mitsuba XML bundle or a compatibility hash for a typed built-in scene ref; |
|
str | None |
Always |
|
str | None |
SHA-256 of the antenna pattern blob |
|
|
Resolved typed asset refs active for this scene. Preserved even for URI-only configs via compatibility refs synthesized by the composer. |
|
int | None |
Seed reproducing this exact channel realization, given the geometry and material hashes |
|
int |
Audit field: realized emitter count |
|
dict[str, float] |
Audit field: SNR statistics; required keys |
|
dict[str, int] |
Audit field: |
|
float |
Audit field: realized overlap rate in |
|
float |
Audit field: realized spectral occupancy in |
|
dict[str, object] |
Backend-specific extras |
|
int |
Storage schema version. Default |
The hash fields are content-addressed references into the dataset’s sibling
assets/ directory; see Concepts / Records, Receivers, and
Assets for the storage layout and
reconstruction contract.
extras["receivers"] receiver-catalog contract¶
When num_rx > 1, SceneMetadata.extras may carry a receiver catalog under
"receivers". The shipped scene composer writes one entry per configured
receiver, in the same order as the resolved receiver list:
extras["receivers"] = [
{
"rx_id": str,
"position_m": tuple[float, float, float],
"orientation": tuple[float, float, float, float],
"antenna_id": str | None,
},
...
]
Contract notes:
The list index is the stable receiver index. Entry
extras["receivers"][i]describes receiveri.Once a consumer resolves the scene to one receiver, recover that receiver’s configuration with
extras["receivers"][scene.rx_index].While the scene is unresolved,
scene.rx_index is None; the catalog is still present but no single entry is the “active” receiver.Single-RX records do not need this catalog; the current shipped composer omits it when
num_rx == 1.
The realized_* audit fields are read by name by validation-and-audit; any
rename or type change is a schema_version bump.
class Signal¶
@dataclass(frozen=True, slots=True)
class Signal:
iq: torch.Tensor
metadata: SignalMetadata | SceneMetadata
component_signals: tuple["Signal", ...] = ()
The universal currency of the channel pipeline. An IQ tensor plus its metadata, optionally with component_signals for nested multi-emitter scenes.
Invariants¶
iqhas dtypetorch.float32, shape(2, N)for one receiver or(num_rx, 2, N)for joint multi-RX records.A per-signal
Signalcarries SignalMetadata; a scene-levelSignalcarries SceneMetadata.Mutating any field after construction raises
dataclasses.FrozenInstanceError.component_signalsis atuple[Signal, ...](recursive); the type annotation matches the actual runtime type.
class StoredRecord¶
@dataclass(frozen=True, slots=True)
class StoredRecord:
scene: SceneMetadata
emitters: tuple[SignalMetadata, ...]
bboxes: tuple[BBox, ...]
seg_mask: torch.Tensor | None = None
text: dict[str, object] | None = None
schema_version: int = 1
@classmethod
def from_labeled_scene(cls, scene: LabeledScene) -> "StoredRecord": ...
A LabeledScene minus IQ. StoredRecord.from_labeled_scene(scene) is the core-type constructor that drops it. The annotator pipeline then applies whitelist_filter(record, sample_id=...) to build this same StoredRecord shape with prompt-safe metadata only, including the validated sample id in scene.extras["sample_id"].
StoredRecord lives in rfgen.core.types (rather than in annotators) so
annotation, validation, and audit consumers can import it without a dependency
cycle.
Provenance field roles (rfgen.core.provenance)¶
Every rfgen dataset’s provenance record is only trustworthy when each recorded
field’s relationship to the rendered signal is explicit. ProvenanceRole is
the shared four-value vocabulary (a StrEnum):
LABEL("label"): the class or target identity the record is labeled with.AXIS("axis"): a value drawn by a sampler AND proven to causally change the rendered output. EveryAXISfield must be covered by a causality test that renders twice, varying only that field, and asserts the output differs.DERIVED("derived"): a value read back from the realized render (an emitter or channel measurement, a realized parameter), never independently drawn.NOT_APPLICABLE("not_applicable"): explicitly recorded as inapplicable for this record (for example, Doppler on a channel model that cannot represent it).
A recorded value that fits none of these is a provenance defect by definition. The motivating case is a value copied from a draw the renderer never consumed, which reads as a faithful record of the render and is not one.
validate_manifest_covers(record, manifest)¶
Requires an exact two-way match between a nested provenance record’s leaf key
paths (dotted, for example drawn.snr_db) and a {key_path: ProvenanceRole}
manifest. Raises ValueError naming both the undeclared record fields (fields
the manifest does not know) and the unrecorded manifest fields (declared roles
no record field carries). Exactness in both directions is deliberate: a
manifest that merely “covers” the record cannot catch a field that silently
disappeared. An empty nested mapping counts as a leaf and must be declared like
any other field.
Datasets declare their manifest once (as data, embedded in their resolved configuration) rather than tagging every record row; the validator runs in tests and inspection tooling.
Schema versioning¶
SignalMetadata and SceneMetadata each carry an integer attribute
schema_version: int = 1 (literal default 1 at module ship time), and
StoredRecord carries the same attribute. LabeledScene deliberately does
not: it is an in-flight value, and a schema version is a property of a storage
format.
Adding a new optional field to any of these dataclasses keeps
schema_versionat its current value.Removing a field, changing a field’s type, or renaming a field requires bumping
schema_versionby+1and is a breaking change for storage round-trip tests.
Remote object identity¶
rfgen.core.object_identity holds two annotated string types, GcsUri and
Generation, used both by the annotation wire models and by
BaseImmutableObjectStore.
Generation excludes 0 and any leading zero. Zero is not a version: as a
write precondition it is the value meaning “this object must not exist”, so a
read pinned to generation zero asks for something that cannot exist. They live
here rather than beside the store because rfgen.annotation.contracts imports
nothing else from the framework, and importing the storage package there would
pull in every record backend to obtain two regular expressions.
Rotation convention¶
Module: rfgen.core.rotation
Every pose triple in rfgen is (roll, pitch, yaw) in radians about the
(x, y, z) axes of the scene frame, and the rotation it denotes is
R = Rz(yaw) Ry(pitch) Rx(roll). That is scipy’s extrinsic "xyz"
sequence, spelled in lowercase; uppercase in scipy would be intrinsic and a
different rotation. It is also Sionna’s (alpha, beta, gamma) about
(z, y, x) read in reverse order, which Sionna documents against 3GPP
TR 38.901 equation (7.1-4), the bearing / downtilt / slant convention the 3GPP
channel models are defined in.
The convention token is ROTATION_CONVENTION = "rpy_xyz_extrinsic_tr38901".
Read it as four claims: rpy for the stored order, xyz for the axes those
three name, extrinsic for the composition, and tr38901 for the standard
that fixes the signs. Fields carrying the triple in degrees, such as
CaptureAlignment.system_rotation_deg and RadarSystem.rotation_deg, carry
the same convention in degrees.
The sign, since it is the part that is easy to get backwards. A positive
pitch tilts a +x boresight toward -z, because Ry(beta) maps +x to
(cos beta, 0, -sin beta). Positive pitch is downtilt. This is asserted
live against a ray-traced solve, with the expected antenna gain computed in
closed form from the standard’s own table rather than from the solver
(tests/integration/sionna_rt/test_rotation_convention.py).
Name |
What it is |
|---|---|
|
The convention token, used by docstrings, provenance, and refusal messages |
|
|
|
The 3x3 float64 body-to-scene matrix; the single definition of what a triple means |
|
|
|
The two unit conversions, axis order untouched |
There is no class and no extension point here on purpose. A convention is a fact rather than a strategy: there is exactly one of it, and choosing a different one would reinterpret every pose field in every dataset already written.
Vendor boundaries convert or refuse. The Sionna RT adapter converts
through sionna_orientation. The RadarSimPy adapter refuses any nonzero
rotation_deg with radarsimpy_rotation_convention_unverified, because that
vendor’s own rotation parameter order is not verifiable from this repository
and guessing a vendor’s axis order produces a plausible cube with corrupted
angle labels rather than an error. The analytic radar engine and the
sionna_rt radar backend keep their unrotated-only refusals; those now state
that the convention exists and the rotated path is what is unqualified.
USD stage convention¶
Module: rfgen.core.usd_conventions
An rfgen USD stage is Z-up, in metres, with poses authored as a
double-precision xformOp:rotateXYZ. That sentence is the pin, and it is
carried by one token: USD_STAGE_CONVENTION = "usd_zup_meters_rotatexyz_double",
stamped into a stage’s customLayerData, named in refusal messages, and
greppable. Beside it sits USD_STAGE_CONVENTION_VERSION = 1, this
repository’s revision of the encoding the token names. They are two constants
because they move on different occasions: the token names the convention and
should survive a clarification, while the integer must move whenever the
encoding does, and a consumer written against 1 can refuse a version it does
not know rather than silently misread it.
Silence is not neutral, which is why the stage authors both frame facts
explicitly. Measured against usd-core 26.8, an unauthored stage reports
upAxis = "Y" and metersPerUnit = 0.01. Sionna is Z-up and
GeometryPose.position_m is metres, so a stage that said nothing would be
wrong by a factor of one hundred with two axes transposed, and would still
load, render, and solve. USD_UP_AXIS and USD_METERS_PER_UNIT are the two
values authored, and check_stage_conventions(up_axis, meters_per_unit) is
the rule the ingest side would apply: it refuses a foreign frame with
usd_stage_convention_mismatch, naming both the found and the required values,
rather than converting one. Nothing in src/ calls it today, and that is
honest rather than an oversight: rfgen writes USD and refuses to read it, so
the reader that would run this check does not exist yet. It is exported,
tested, and ready for the cycle that adds one. Converting is arithmetically trivial and that is
exactly the danger: a wrong conversion produces a scene that loads, solves, and
is wrong everywhere, and nothing downstream asserts otherwise.
The pose components go in unchanged, and that is a measurement rather than a
convenience. Authoring degrees(roll, pitch, yaw) into a double-precision
rotateXYZ op produces a local transformation whose upper-left block,
transposed, equals rotation_matrix_from_rpy(roll, pitch, yaw) to
2.220446049250313e-16, one ulp of a float64. There is no reordering and no
sign flip. The reason two different conventions need no rearranging is that
they differ in two ways that cancel: USD’s rotateXYZ applies X then Y
then Z to a row vector, which is Rx Ry Rz in row-vector layout, and
transposing that into column-vector layout both reverses the product and
transposes each factor, giving Rz Ry Rx, which is rfgen’s composition letter for
letter. A reader who applies one of the two corrections and not the other gets
the inverse rotation, which every zero-rotation configuration in this
repository passes under.
Name |
What it is |
|---|---|
|
The convention token, stamped into stage metadata and named in refusals |
|
The encoding revision, so a consumer can refuse a version it does not know |
|
|
|
|
|
The |
|
The refusal a USD reader would apply, |
UsdGeom.XformCommonAPI is not usable for this pin, and the reason is a
precision trap rather than a style preference. It is the obvious thing to reach for and it is wrong
here. Its SetRotate accepts only Gf.Vec3f, since passing Gf.Vec3d
raises ArgumentError because the double overload does not exist, so after
SetTranslate(Gf.Vec3d(...)) plus SetRotate(Gf.Vec3f(...)) the authored ops
report xformOp:translate double3 and xformOp:rotateXYZ float3, measured.
The API silently downgrades exactly the op whose precision matters, taking the
residual from 2.22e-16 to 2.18e-08, and it cannot be fixed by passing a
different type. The pinned spelling is the manual AddTranslateOp plus
AddRotateXYZOp pair, both at UsdGeom.XformOp.PrecisionDouble.
This module imports no pxr. check_stage_conventions takes the two frame
facts as arguments rather than reading them off a stage, so the conformance
rule and the rotation identity are both gated in an environment with no USD
runtime installed at all. There is no class and no extension point, for the
rotation convention’s reason restated: a convention is a fact rather than a
strategy.
See Also¶
API Reference: the ABCs that produce and consume these types.
IQ Layout Policy: in-memory and on-disk IQ shapes.
Label Schema: on-disk representation of
BBoxandseg_mask.TorchSig Interop: round-trip with TorchSig’s
SignalMetadataObject.