Config Schema¶
Warning
Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, config field names, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.
The framework is driven entirely by configuration. Hydra owns composition and CLI overrides; Pydantic v2 owns validation and type safety. This page is the normative schema reference: every config the framework accepts is documented here.
Closed-set vs open-set fields¶
The schema follows one rule for every string-valued field, and the rule is machine-checked rather than described in prose:
Closed-set fields are typed as
StrEnummembers fromrfgen.enums. PlainLiteral[...]is forbidden; plainstris forbidden. YAML strings deserialize to enum members through Pydantic’s enum coercion, so YAML files keep reading naturally (backend: zarr_local).Open-set plugin selectors stay typed as
strbecause they resolve through the plugin registry at instantiation time. The registry, not the schema, decides which names are valid.
The closed-set fields and their backing enums are:
Field |
Enum |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The open-set plugin selectors are:
Field |
Why open |
|---|---|
|
Built-in strings coerce to StorageBackend, but custom store plugins resolve through |
|
Resolves through the |
|
Resolves through the |
|
Resolves through the |
|
Resolves through the storage backend’s own codec table (Zarr/Blosc, HDF5/gzip, etc.). |
|
Resolves through the |
|
Resolves through the |
LLMConfig.model is also an open str, but it is not a plugin selector and
does not resolve through a framework registry. It is a provider-local model
identifier passed to the selected inference client, so unsupported values fail
only when that provider or its API rejects them.
A field that today is open str may be promoted to a closed StrEnum later when its value set stabilizes. Such promotions are tracked as schema-version bumps; they are never silent.
Top-level structure¶
# configs/config.yaml: the default config tree
defaults:
- emitter_zoo: heterogeneous
- channel: default
- scene: dense
- placement: default
- label: joint
- annotator: caption_only
- storage: zarr_local
- executor: local
- _self_
run:
run_id: local-dev
num_samples: 10000
shard_size: 1000
seed: 42
fail_fast: false
storage:
backend: zarr_local
path: ./out/run-${now:%Y%m%d-%H%M%S}
The layered groups (emitter_zoo, channel, scene, placement, label, annotator, storage) plus executor are independently swappable. Any group can be overridden on the CLI:
rfgen generate channel=sionna_rt_urban scene=wideband annotator=full_suite
Root config: GenerationConfig¶
class RunConfig(BaseModel):
num_samples: int = Field(gt=0, default=10000)
shard_size: int = Field(gt=0, default=1000)
seed: int = 42
shard_failure_threshold: float = Field(gt=0.0, le=1.0, default=1.0)
fail_fast: StrictBool = False
@model_validator(mode="after")
def _shard_divides(self) -> "RunConfig":
if self.num_samples % self.shard_size != 0:
warnings.warn("num_samples not divisible by shard_size; last shard will be partial")
return self
class GenerationConfig(BaseModel):
"""Composes emitter, channel, scene, placement, label, annotator,
storage, executor, credentials, and run metadata."""
emitter_zoo: EmitterZooConfig = Field(default_factory=EmitterZooConfig)
channel: ChannelConfig = Field(default_factory=ChannelConfig)
scene: SceneConfig = Field(default_factory=SceneConfig)
placement: PlacementConfig = Field(default_factory=PlacementConfig) # grid-source selection
label: LabelConfig = Field(default_factory=LabelConfig)
annotator: AnnotatorConfig | None = None
storage: StorageConfig
executor: ExecutorConfig = Field(default_factory=ExecutorConfig)
credentials: CredentialsConfig | None = None # cloud auth (optional; defaults to provider chain)
run: RunConfig
fail_fast is a strict boolean knob. When true, the Layer 9 shard worker stops on the first
per-sample failure. Records yielded before that failure may already be stored, but the worker
aborts the shard commit before the backend publishes its shard-complete marker, allowing a retry
to repair the shard. The active shard-level failure ratio control remains
shard_failure_threshold.
PlacementConfig¶
class PlacementConfig(BaseModel):
time_strategy: TimePlacementStrategy = TimePlacementStrategy.IID_UNIFORM
freq_strategy: FrequencyPlacementStrategy = FrequencyPlacementStrategy.IID_UNIFORM
grid_source: str = "json_manifest" # open: rfgen.grid_sources entry-point name
channel_plan_source: str = "json_manifest" # compatibility alias for grid_source
Selects the canonical time and frequency placement strategies plus the
BaseGridSource plug-in used by the realistic_density frequency placement
strategy to look up per-band channel grids. time_strategy and
freq_strategy are closed enum fields; YAML uses their enum values.
grid_source is an open-set string resolved through the rfgen.grid_sources
entry-point group at strategy instantiation. The default "json_manifest"
value resolves to JsonManifestGridSource, which loads per-band JSON files
from rfgen/placement/data/grids/<band>.json. channel_plan_source remains
as a compatibility alias and must match grid_source when both are provided.
Third-party BaseGridSource plugins register under rfgen.grid_sources without
touching this schema or the strategy implementation.
Fields¶
Field |
Type |
Required |
Default |
Notes |
|---|---|---|---|---|
|
no |
|
Canonical time-domain placement strategy selector. |
|
|
no |
|
Canonical frequency-domain placement strategy selector. |
|
|
str |
no |
|
Open-set entry-point name for the grid source; resolved through |
|
str |
no |
|
Compatibility alias for |
EmitterZooConfig¶
The pool of emitters available to the scene composer, plus per-family parameter ranges.
class EmitterFamilyConfig(BaseModel):
family: EmitterFamily # closed: rfgen.enums.EmitterFamily
classes: list[str] # subset of the family's supported_classes
weight: float = 1.0 # relative sampling weight
params: dict[str, object] = {} # forwarded to BaseEmitter.generate
# Per-device fingerprint
fingerprint: FingerprintConfig | None = None
class FingerprintConfig(BaseModel):
enabled: bool = True
num_devices: int = Field(ge=1, default=10)
cfo_hz_range: tuple[float, float] = (-1000.0, 1000.0)
sfo_ppm_range: tuple[float, float] = (-20.0, 20.0)
iq_imbalance_db_range: tuple[float, float] = (-1.0, 1.0)
phase_noise_dbc_hz_range: tuple[float, float] = (-110.0, -90.0)
pa_model: PAModel = PAModel.RAPP # closed enum for the PA-nonlinearity model
class EmitterZooConfig(BaseModel):
families: list[EmitterFamilyConfig]
@field_validator("families")
@classmethod
def _at_least_one(cls, v):
if not v:
raise ValueError("emitter_zoo.families must be non-empty")
return v
pa_model is a closed PAModel enum. Unknown values fail during schema validation.
Example: configs/emitter_zoo/heterogeneous.yaml¶
families:
- family: comms
classes: [bpsk, qpsk, 16qam, 64qam, ofdm]
weight: 4.0
fingerprint:
enabled: true
num_devices: 20
- family: radar
classes: [pulse, fmcw, lfm]
weight: 1.0
- family: adsb
classes: [mode_s_short, mode_s_long]
weight: 0.5
ChannelConfig¶
class ChannelConfig(BaseModel):
name: str = "torchsig_impairments" # legacy compatibility field
chain: list[ChannelChainEntry] = []
params: dict[str, object] = {}
snr_db_range: tuple[float, float] = (-10.0, 30.0)
chain is the live ordered list of channel transformations. The schema rejects
more than one transformation in Group.CHANNEL, rejects any cross-group
backtracking (Group order must stay monotonic non-decreasing across the full
chain), and rejects adjacent transformations inside the same group when their
ordinals are not monotonic non-decreasing. The top-level name, params, and
snr_db_range fields remain only as compatibility placeholders; shipped
validation requires the default name, empty params, and the default
snr_db_range so config cannot silently set values runtime ignores.
Example: propagation-only chain¶
chain:
- transformation: propagation
params: {}
SceneConfig¶
class DensityConfig(BaseModel):
mode: DensityMode = DensityMode.RANGE # closed: rfgen.enums.DensityMode
min_emitters: int = Field(ge=0, default=1)
max_emitters: int = Field(ge=1, default=10)
poisson_rate: float | None = None # required when mode is DensityMode.POISSON
class OverlapConfig(BaseModel):
policy: SceneOverlapPolicy = SceneOverlapPolicy.ALLOW
class MultiRXConfig(BaseModel):
"""Multi-receiver layout. Either set `geometry` (preset array) OR `receivers` (explicit list)."""
geometry: ArrayGeometry | None = None # closed: rfgen.enums.ArrayGeometry
receivers: list[ReceiverConfig] = []
class SceneGeometryConfig(BaseModel):
backend: SceneGeometryBackend = SceneGeometryBackend.NONE # closed: rfgen.enums.SceneGeometryBackend
overlap_policy: SceneOverlapPolicy = SceneOverlapPolicy.ALLOW
rt_solver: RTSolverConfig | StatisticalSolverConfig | None = None
class SceneConfig(BaseModel):
sample_rate_hz: float = Field(gt=0, default=20_000_000.0)
duration_s: float = Field(gt=0, default=0.020)
bandwidth_hz: float = Field(gt=0, default=10_000_000.0)
center_hz: float = 0.0
density: DensityConfig = Field(default_factory=DensityConfig)
time_placement: TimePlacementStrategy = TimePlacementStrategy.IID_UNIFORM
time_placement_params: dict[str, object] = {} # per-strategy kwargs
frequency_placement: FrequencyPlacementStrategy = FrequencyPlacementStrategy.IID_UNIFORM
frequency_placement_params: dict[str, object] = {} # per-strategy kwargs
rx_array: RxArrayConfig = Field(default_factory=RxArrayConfig)
multi_rx: MultiRXConfig | None = None
channel_application: ChannelApplicationMode = ChannelApplicationMode.SCENE
geometry: SceneGeometryConfig = Field(default_factory=SceneGeometryConfig)
assets: SceneAssetsConfig = Field(default_factory=SceneAssetsConfig)
@model_validator(mode="after")
def _check_grid(self):
if self.sample_rate_hz < 2.0 * self.bandwidth_hz:
raise ValueError(
"scene.sample_rate_hz must be >= 2 * bandwidth_hz "
"(Nyquist requires at least 2x oversampling)."
)
return self
MultiRXConfig and SceneGeometryConfig cross-field rules¶
MultiRXConfig.geometryandMultiRXConfig.receiversare mutually exclusive: setting both is apydantic.ValidationError. A populatedgeometrybuilds the array from the named preset; a populatedreceiverslist places receivers explicitly.SceneGeometryConfig.backend == SceneGeometryBackend.SIONNA_RTrequires a scene-geometry asset on the parentSceneConfig: the preferred typedassets.scene_geometry_ref, or the legacyassets.scene_geometry_uri. The validator raises withloc=("assets", "scene_geometry_uri")when neither is present.SceneAssetsConfig.material_db_ref, when present, must useGeometryAssetKind.MATERIAL_DB; everySceneAssetsConfig.antenna_pattern_refs[*]entry must useGeometryAssetKind.ANTENNA_PATTERN.SceneGeometryBackendvalues are lowercase enum strings in new configs:sionna_rt,mitsuba, andnone. Pydantic still accepts the legacy spellings"SionnaRT"and"Mitsuba"for backward compatibility.
Example: configs/scene/dense.yaml¶
sample_rate_hz: 30.72e6
duration_s: 0.01
bandwidth_hz: 20e6
center_hz: 2.45e9
density:
mode: range
min_emitters: 4
max_emitters: 12
time_placement: iid_uniform
time_placement_params: {}
frequency_placement: stratified
frequency_placement_params:
min_spacing_hz: 1000
geometry:
backend: sionna_rt
overlap_policy: allow
assets:
scene_geometry_ref:
kind: mitsuba_xml_bundle
uri: file:///abs/path/assets/sionna/munich.xml
content_hash: sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
rx_array:
num_rx: 4
array: ula_4
spacing_lambda: 0.5
Selecting placement strategies¶
The two placement strategies are selected inside SceneConfig through the enum
fields time_placement and frequency_placement. Strategy-specific kwargs are
carried separately in time_placement_params and
frequency_placement_params.
scene:
time_placement: event_periodic_beacon # TimePlacementStrategy value
time_placement_params:
period_s: 0.1024 # Wi-Fi TBTT default
frequency_placement: realistic_density # FrequencyPlacementStrategy value
frequency_placement_params:
taxonomy: wifi-2.4ghz
min_spacing_hz: 5_000_000.0
time_placementis a TimePlacementStrategy member; the YAML value is also the entry-point key resolved through therfgen.time_placementplugin registry.frequency_placementis a FrequencyPlacementStrategy member; the YAML value is also the entry-point key resolved through therfgen.freq_placementplugin registry.*_placement_paramsis forwarded to the selected strategy’s constructor after Pydantic validation.
See the Placement Strategy Selection Guide for which strategy to pick per scene type, evidence tiers, and copy-paste config recipes.
LabelConfig¶
class LabelerSpec(BaseModel):
name: str # open: rfgen.labelers registry name
params: dict[str, object] = {}
class LabelConfig(BaseModel):
name: str = "joint" # open: rfgen.labelers registry name
params: dict[str, object] = {}
extra_labelers: list[str | LabelerSpec] = []
seg_n_fft: int = 1024
seg_hop: int = 256
segmentation_mode: SegmentationMode = SegmentationMode.SINGLE_LABEL
segmentation_tie_break: SegmentationTieBreak = SegmentationTieBreak.LOWER_EMITTER_INDEX
Segmentation capability is selected by name: bbox stays metadata-only,
while segmentation and joint emit segmentation by construction.
segmentation_mode and segmentation_tie_break are the public mask-shape and
overlap-rule controls.
AnnotatorConfig¶
class LLMConfig(BaseModel):
provider: str # open: rfgen.inference_clients registry name
model: str # open: per-provider model id
temperature: float = 0.2
max_tokens: int = 512
json_schema_mode: StrictBool = True
class AnnotatorConfig(BaseModel):
enabled: StrictBool = True
# closed: rfgen.enums.AnnotationType
types: list[AnnotationType] = Field(default_factory=lambda: [AnnotationType.CAPTION])
bulk_llm: LLMConfig # high-volume, low-cost
verifier_llm: LLMConfig | None = None # optional second-pass verifier
verifier_subset_pct: float = Field(ge=0.0, le=100.0, default=0.0)
enabled and json_schema_mode use Pydantic StrictBool: only YAML/JSON
booleans (true/false) are valid. Numeric and string-like booleans such as
0, 1, "false", and "true" are rejected at config-validation time.
provider is open str because inference providers are plugin-resolved through the rfgen.inference_clients entry-point group; the framework does not ship a closed enum of provider names. model is a provider-local model identifier and is validated only by the selected provider/client behavior. The class name LLMConfig and the field names bulk_llm / verifier_llm are retained for config-surface stability; the configured providers can be text-only LLMs, vision-language models, or audio-language models.
Example: configs/annotator/full_suite.yaml¶
enabled: true
types: [caption, qa, reasoning, scene_report, contrastive]
bulk_llm:
provider: gemini
model: gemini-3.1-flash-lite
temperature: 0.2
verifier_llm:
provider: anthropic
model: claude-sonnet-4-6
temperature: 0.0
verifier_subset_pct: 10.0
verifier_subset_pct accepts values from 0.0 through 100.0. 0.0
disables verifier scoring, 100.0 verifies the full annotation set, and
intermediate percentages require programmatic PAES membership selection before
records are handed to the annotator.
StorageConfig¶
class StorageConfig(BaseModel):
backend: StorageBackend | str # built-in enum or rfgen.stores plugin name
path: str # URI; SIGMF is local-only (`file://...`)
compression: str = "blosc" # open: backend-specific codec name
chunk_samples: int = Field(gt=0, default=128)
record_axis: RecordAxis = RecordAxis.PER_RX # closed: rfgen.enums.RecordAxis
assets_path: str | None = None
compression is open str because the valid codec set depends on the chosen backend. The shipped validators accept none/blosc for Zarr, none/gzip/lzf for HDF5, none/gzip/bz2/xz for WebDataset, and none for SigMF. Leaving the field at "blosc" rewrites to the effective backend default for HDF5 (gzip) and for WebDataset/SigMF (none).
chunk_samples is a shipped Zarr/HDF5 control. Explicitly setting it for WebDataset or SigMF raises pydantic.ValidationError. Custom store plugins receive the field unchanged and own any plugin-specific semantics.
record_axis selects how multi-RX scenes map to records: RecordAxis.PER_RX (default) emits one record per receiver, RecordAxis.JOINT emits one record per scene with a leading RX axis on the IQ tensor. See Concepts / Records, Receivers, and Assets and the full field surface on StorageConfig.
StorageConfig.backend == StorageBackend.SIGMF requires path to start with file://; the validator raises pydantic.ValidationError otherwise because the SigMF library surface used here is local-path based.
ExecutorConfig¶
Phase 1 execution selection. The name field is a free string registered via
Python entry points; new backends do not require updating this schema. PySpark
is the only shipped distributed substrate. Local execution is synchronous.
class ExecutorConfig(BaseModel):
name: str = "local" # open: rfgen.executors registry name
parallelism: int = Field(gt=0, default=1)
The framework looks up the registered executor by name. parallelism is the
requested PySpark partition count and is ignored by synchronous local
execution. The schema rejects booleans, non-positive values, blank names, and
unknown fields. Dask and Ray are not shipped executor backends.
Example: managed Spark serverless¶
# configs/executor/managed_spark_serverless.yaml
name: managed_spark_serverless
parallelism: 32
Example: Local¶
# configs/executor/local.yaml
name: local
parallelism: 1
AnnotationOrchestratorConfig¶
Phase 2 batch inference submission schema. It remains separate from the Phase 1
GenerationConfig tree.
class AnnotationOrchestratorConfig(BaseModel):
name: str = "local_loop" # open: rfgen.annotation_orchestrators
min_poll_interval_s: float = Field(ge=0.0, default=10.0)
resources: BatchResources | None = None
resources is a typed union of OpenAIBatchResources,
AnthropicBatchResources, VertexBatchResources,
SageMakerBatchResources, and AzureBatchResources. Each managed provider
name requires its matching resource type. local_loop rejects managed
resources. The schema rejects booleans for min_poll_interval_s, blank names,
negative poll intervals, and unknown fields.
Provider resource contracts are:
Resource model |
Required fields |
Defaults and validation |
|---|---|---|
|
|
|
|
|
|
|
|
Input and output use |
|
|
Input and output use |
|
|
Input and output use |
All resource strings must be non-empty. Resource models reject unknown fields and use strict booleans for lifecycle controls.
Managed fetch, persistence, and cleanup¶
Managed orchestrators provide at-least-once result delivery across fetch and
persistence retries. The stable idempotency key is
AnnotationJob.key = (sample_id, annotation_type, template_id, run_id).
Submitting a job whose key is already durable reuses the stored result instead
of submitting and billing it again.
A resumable batch handle checkpoints normalized results before store
persistence and records each key after its append-only annotation write
succeeds. If persistence fails partway through a batch, retrying fetch
skips the durable keys and resumes with the first unpersisted result. The
normalized provider output is not fetched again once it has been checkpointed.
Provider cleanup runs only after every result is durable. A cleanup failure is
observable and leaves the handle incomplete; retrying fetch repeats cleanup
without refetching provider output or rewriting durable annotations. On
success, retain_staging=false removes managed input and output staging
objects. retain_staging=true preserves them. OpenAI uses
delete_input_file_after_fetch for its staged input-file lifecycle.
Example: GCP Vertex Batch¶
# configs/annotation_orchestrator/vertex_batch.yaml
name: vertex_batch
min_poll_interval_s: 10.0
resources:
project: ${oc.env:GCP_PROJECT}
location: us-central1
model: gemini-3.1-flash-lite
input_uri: gs://rfgen-batch/input/
output_uri: gs://rfgen-batch/output/
Example: Anthropic Batch¶
# configs/annotation_orchestrator/anthropic_batch.yaml
name: anthropic_batch
min_poll_interval_s: 10.0
resources:
model: claude-3-5-haiku-latest
max_tokens: 1024
AuditConfig¶
Dataset-audit thresholds and bounded aggregation limits.
class AuditConfig(BaseModel):
snr_mean_error_db: float = Field(default=0.5, ge=0.0, le=100.0)
class_kl_divergence: float = Field(default=0.05, ge=0.0, le=1.0)
overlap_error: float = Field(default=0.02, ge=0.0, le=1.0)
occupancy_error: float = Field(default=0.05, ge=0.0, le=1.0)
max_records: int = Field(default=1_000_000, gt=0)
max_class_cardinality: int = Field(default=10_000, gt=0)
max_snr_bucket_cardinality: int = Field(default=10_000, gt=0)
max_emitter_count_cardinality: int = Field(default=10_000, gt=0)
Threshold fields reject booleans and enforce the shown inclusive ranges. Record and cardinality bounds reject booleans and must be positive integers. Unknown fields are rejected.
CredentialsConfig¶
class CredentialsConfig(BaseModel):
provider: str = "static" # open: rfgen.credentials registry name
# ("static", "gcp_adc", "aws_default",
# "azure_default", or a registered third-party name)
params: dict[str, object] = Field(default_factory=dict)
In most setups this is omitted entirely; each cloud SDK uses its own default credential chain. When present, GenerationConfig.build_credentials_provider() instantiates the provider and BaseAnnotator.from_generation_config(...) passes it into LLM client construction. Unknown provider names raise PluginNotFoundError during provider construction, before any inference call is made.
Static credentials use the existing per-scope provider schema:
credentials:
provider: static
params:
scopes:
llm:
api_key: ${oc.env:OPENAI_API_KEY}
CLI override patterns¶
Hydra’s CLI override syntax applies to every leaf field:
# Swap a whole group
rfgen generate channel=sionna_rt_urban
# Override a single field
rfgen generate scene.density.max_emitters=50
# Multi-run sweeps (Hydra --multirun)
rfgen generate -m channel=awgn,sionna_rt scene.density.max_emitters=10,50,100
Validation timing¶
Validation runs before any sample generation begins:
Hydra resolves the config tree.
GenerationConfig.model_validate(…) is called.
Every plugin’s
schema()validates its ownparamsblock.Any failure raises ConfigError with a path to the offending field.
A failed config never costs compute. A 12-hour Spark run that bombs on the first sample because of a typo is an explicit non-goal.
Contract Tests¶
The implementation must carry tests for these invariants:
Test |
Expected behavior |
|---|---|
Minimal local config |
A complete local config validates and materializes the emitter, channel, scene, placement, label, annotator, and storage blocks plus executor settings. |
Unknown closed-enum value |
A closed-set field such as |
Open plugin name |
|
Executor controls |
|
Annotation resources |
Every managed annotation provider requires its matching typed resource model; |
Audit bounds |
Audit thresholds enforce their documented ranges; record and cardinality limits require positive integers. |
Multi-RX storage |
|
Multi-RX exclusivity |
|
Scene geometry asset |
|
SigMF URI |
|
Round-trip |
|
See Also¶
Reference / API / Config - proposed Pydantic class surface.
Reference / API / Enums - closed value sets used by this schema.
Reference / Plugin Metadata - open plugin naming and compatibility contracts.
Background / Open Questions - unresolved schema questions.