Architecture

RFGen separates scene planning, domain physics, record persistence, and text annotation. There is one generation path: a configuration mints one ScenePlan, one or more independent projections render it, and their fields are persisted together as exactly one Signal Dataset record per sample.

        flowchart LR
  C[Editable GenerationConfig] --> P[One ScenePlan]
  P --> O[Observation projections]
  O --> A[One aggregate Signal Dataset record]
  A --> AO[Optional annotation overlay]
    

One sample is one record, whatever the scene contains. A multi-receiver scene does not fan out into several records: each receiver arrives as its own named subtree inside the single record, and a consumer that wants one training example per receiver selects the subtree it needs when it reads. A scene carrying both communications and radar activity is likewise one record, with one projection subtree each.

The aggregate branch is described in Unified Scene Observations, including its shared clock, native-axis fields, plugin seam, identity envelope, and one-record memory boundary.

Annotation is published as an append-only overlay on a snapshot and never rewrites the records it describes. Use Generation Jobs / Annotate samples to select the supported source and execution path.

Use rfgen init to materialize a starting configuration that can be inspected and adapted. RFGen is a library of signal-generation, configuration, record, and storage primitives; dataset application policy belongs to its external owner.

dataproc_serverless is a bounded alternate execution edge: it stages a validated configuration, submits a PySpark batch whose tasks each write one Signal Dataset shard to a gs:// root, and publishes the snapshot root last once every expected shard has succeeded. It does not turn the local store API into a shared cloud database. See Dataproc Serverless.

See Emitters, Scenes, Channels, Labels, Storage, and Annotations for the responsibilities at each boundary.

Three boundaries the diagram above compresses have pages of their own. Unified Scene Observations is the aggregate record contract. Signal, Capture, Record says which container holds the samples at each stage, and in particular that the box labelled “Channels and receiver” is one shared receiver chain running over one ComplexCapture container for both domains. External Scene Seams covers the one place this pipeline touches an external 3D scene format: rfgen writes a scene plan as a USD stage and refuses to read one back.

Dataset Consumption

Read a published snapshot with the signal_dataset package; loading it into a training pipeline belongs to the consumer. rfgen ships no torch Dataset, sampler, or collator: it stops at the published snapshot, and nothing on the reading side alters stored IQ, labels, or annotations.

A record’s field inventory, and the contract that puts fields from different domains on one clock, are in Time coordinates.