Config Schema¶
The framework is driven by YAML configuration composed by Hydra and validated
by Pydantic. The executable configuration contract is the materialized
config.yaml plus rfgen validate --config-dir DIRECTORY; this page retains
schema detail that must be checked against the public API before use.
Closed-set vs open-set fields¶
The schema follows one rule for every string-valued field, and the rule is machine-checked rather than described in prose:
Closed-set fields are typed as
StrEnummembers fromrfgen.core.enums. PlainLiteral[...]is forbidden; plainstris forbidden. YAML strings deserialize to enum members through Pydantic’s enum coercion, so YAML files keep reading naturally (mode: poisson). The one intentional exception isEventDurationConfig.mode: Literal["fixed"]: it is a fixed-only discriminant reserved for future duration-policy variants, rather than a general closed choice that belongs inrfgen.core.enums.Open-set plugin selectors stay typed as
strbecause they resolve through the plugin registry at instantiation time. The registry, not the schema, decides which names are valid.
The closed-set fields and their backing enums are:
Field |
Enum |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The open-set plugin selectors are:
Field |
Why open |
|---|---|
|
Built-in strings coerce to StorageBackend, but custom store plugins resolve through |
|
Resolves through the |
|
Resolves through the |
|
Selects the retained static local credential provider. |
|
Resolves through the |
|
Resolves through the |
|
Built-in values coerce to TimePlacementStrategy; another non-empty name selects an installed |
|
Resolves through |
|
Resolves built-ins or the |
LLMConfig.model is also an open str, but it is not a plugin selector and
does not resolve through a framework registry. It is a provider-local model
identifier passed to the selected inference client, so unsupported values fail
only when that provider or its API rejects them.
A field that today is open str may be promoted to a closed StrEnum later
when its value set stabilizes. Such promotions are tracked as schema-version
bumps; they are never silent.
Top-level structure¶
# configs/config.yaml: the default config tree
defaults:
- emitter_zoo: heterogeneous
- channel: default
- scene: dense
- placement: default
- label: joint
- annotator: caption_only
- storage: signal_dataset
- executor: local
- _self_
run:
run_id: local-dev
num_samples: 10000
shard_size: 1000
seed: 42
storage:
backend: signal_dataset
path: ./out/run-${now:%Y%m%d-%H%M%S}
projections:
- projection_id: receiver
selector: rfgen.communications.receiver
A generation config names one ScenePlan template and a nonempty ordered projection list:
plan:
clock:
time_origin_s: 0.0
duration_s: 0.001
systems:
- system_id: comm-1
domain: communications
pose:
position_m: [0.0, 0.0, 0.0]
orientation_rad: [0.0, 0.0, 0.0]
velocity_mps: [0.0, 0.0, 0.0]
tx_element_locations_m: [[0.0, 0.0, 0.0]]
rx_element_locations_m: [[0.0, 0.0, 0.0]]
events:
- planned_event_id: comm-event
system_id: comm-1
start_offset_s: 0.0
stop_offset_s: 0.001
comms:
device_id: device-1
transmitter_role: uplink_ue
link_id: link-1
link_direction: uplink
emitter_key: torchsig_comms
class_label: qpsk
bandwidth_hz: 200000.0
frequency_offset_hz: 0.0
tx_power_dbm: 10.0
duration: {kind: fixed, seed: 7, value_s: 0.001}
projections:
- projection_id: communications_rx
selector: rfgen.communications.receiver
contract_version: 2 # must equal the installed plugin's; both shipped
# projections are at 2
scene:
duration_s: 0.001 # must equal plan.clock.duration_s; they describe one scene
observation:
max_tensor_bytes: 1073741824
projections is how a run declares what it observes. See the runnable unified
observation guide.
The layered groups (emitter_zoo, channel, scene, placement, label,
annotator, storage) plus executor are independently swappable. Any group
can be overridden on the CLI:
rfgen init narrowband-baseline ./my-config
rfgen validate --config-dir ./my-config
Root config: GenerationConfig¶
class RunConfig(BaseModel):
num_samples: int = Field(gt=0, default=10000)
shard_size: int = Field(gt=0, default=1000)
seed: int = 42
shard_failure_threshold: float = Field(gt=0.0, le=1.0, default=1.0)
@model_validator(mode="after")
def _shard_divides(self) -> "RunConfig":
if self.num_samples % self.shard_size != 0:
warnings.warn("num_samples not divisible by shard_size; last shard will be partial")
return self
class GenerationConfig(BaseModel):
"""Composes emitter, channel, scene, placement, label, annotator,
storage, executor, credentials, and run metadata."""
emitter_zoo: EmitterZooConfig = Field(default_factory=EmitterZooConfig)
channel: ChannelConfig = Field(default_factory=ChannelConfig)
scene: SceneConfig = Field(default_factory=SceneConfig)
placement: PlacementConfig = Field(default_factory=PlacementConfig) # grid-source selection
label: LabelConfig = Field(default_factory=LabelConfig)
annotator: AnnotatorConfig | None = None
augmentation: AugmentationConfig | None = None
storage: StorageConfig | None = None
executor: ExecutorConfig = Field(default_factory=ExecutorConfig)
credentials: CredentialsConfig | None = None # optional static local credentials
run: RunConfig | None = None
projections: tuple[ObservationProjectionConfig, ...] | None = Field(
default=None, min_length=1
)
observation: ObservationConfig = Field(default_factory=ObservationConfig)
plan: ScenePlanConfig | None = None
run and storage are required only for executor-backed generation. A Python
caller that uses generate_record(config, seed=...) may omit both because it
returns one in-memory LabeledScene and does not open a store. rfgen generate and
generate_local_signal_dataset(config) reject an omitted block before they resolve components or
open storage; when provided, run.run_id and storage.path remain required.
ObservationProjectionConfig strictly validates projection_id, a portable
namespaced selector, a positive integer contract_version, and params.
Projection IDs must be unique. ObservationConfig.max_tensor_bytes is a
strictly positive integer and defaults to 1 GiB. It bounds all primary and
generated-coordinate tensors in one aggregate Signal Dataset record.
shard_failure_threshold is the shard-level failure control, read by the
remote executor when it decides whether a shard failed. Local generation has no
per-sample failure tolerance: an exception ends the run.
There is no fail_fast. It named a shard worker that no longer exists, and
nothing read it after that worker was removed.
Optional augmentation¶
augmentation:
selector: torchsig_classification
params:
profile: torchsig_rx_classification_v2_1_1
torchsig_version: 2.1.1
label_contract: classification_only
augmentation is optional. When present, selector is a required non-empty
open rfgen.augmentations entry-point name. It is resolved by
GenerationConfig.build_augmentation(), not during Pydantic validation. The
shown selector requires the optional TorchSig integration and accepts only its
pinned classification-only profile. It is not a channel-chain entry.
PlacementConfig¶
class PlacementConfig(BaseModel):
time_strategy: TimePlacementStrategy = TimePlacementStrategy.IID_UNIFORM
freq_strategy: FrequencyPlacementStrategy = FrequencyPlacementStrategy.IID_UNIFORM
grid_source: str = "json_manifest" # open: rfgen.grid_sources entry-point name
channel_plan_source: str = "json_manifest" # compatibility alias for grid_source
Selects the canonical time and frequency placement strategies plus the
BaseGridSource plug-in used by the realistic_density frequency placement
strategy to look up per-band channel grids. time_strategy and
freq_strategy are closed enum fields; YAML uses their enum values.
grid_source is an open-set string resolved through the rfgen.grid_sources
entry-point group at strategy instantiation. The default "json_manifest"
value resolves to JsonManifestGridSource, which loads per-band JSON files
from rfgen/placement/data/grids/<band>.json. channel_plan_source remains
as a compatibility alias and must match grid_source when both are provided.
Third-party BaseGridSource plugins register under rfgen.grid_sources without
touching this schema or the strategy implementation.
Fields¶
Field |
Type |
Required |
Default |
Notes |
|---|---|---|---|---|
|
no |
|
Canonical time-domain placement strategy selector. |
|
|
no |
|
Canonical frequency-domain placement strategy selector. |
|
|
str |
no |
|
Open-set entry-point name for the grid source; resolved through |
|
str |
no |
|
Compatibility alias for |
EmitterZooConfig¶
The pool of emitters available to the scene composer, plus per-family parameter ranges.
class EmitterFamilyConfig(BaseModel):
family: EmitterFamily # closed: rfgen.core.enums.EmitterFamily
classes: list[str] # subset of the family's supported_classes
selector: str | None = None # required when building an emitter pool
weight: float = 1.0 # relative sampling weight
params: dict[str, object] = {} # forwarded to BaseEmitter.generate
# Per-device fingerprint
fingerprint: FingerprintConfig | None = None
class FingerprintConfig(BaseModel):
enabled: bool = True
num_devices: int = Field(ge=1, default=10)
cfo_hz_range: tuple[float, float] = (-1000.0, 1000.0)
sfo_ppm_range: tuple[float, float] = (-20.0, 20.0)
iq_imbalance_db_range: tuple[float, float] = (-1.0, 1.0)
phase_noise_dbc_hz_range: tuple[float, float] = (-110.0, -90.0)
pa_model: PAModel = PAModel.RAPP # closed enum for the PA-nonlinearity model
class EmitterZooConfig(BaseModel):
families: list[EmitterFamilyConfig] = Field(default_factory=list)
selector is optional in the Pydantic object so partial composition remains
possible, but it is mandatory at emitter-pool construction. The factory rejects
both an empty pool and a family without a selector; it never chooses by
registry-discovery order. For example: selector: torchsig_comms.
pa_model is a closed PAModel enum. Unknown
values fail during schema validation.
Example: configs/emitter_zoo/heterogeneous.yaml¶
families:
- family: comms
selector: torchsig_comms
classes: [bpsk, qpsk, 16qam, 64qam]
weight: 4.0
fingerprint:
enabled: true
num_devices: 20
- family: radar
selector: chirp_radar
classes: [lfm_chirp]
weight: 1.0
- family: adsb
selector: adsb
classes: [adsb_extended_squitter]
weight: 0.5
ChannelConfig¶
class ChannelConfig(BaseModel):
name: str = "physical_pipeline" # legacy compatibility field
chain: list[ChannelChainEntry] = []
params: dict[str, object] = {}
snr_db_range: tuple[float, float] = (-10.0, 30.0)
chain is the live ordered list of channel transformations. Each entry pins a
closed transformation slot and may name an installed third-party implementation
with selector; the selected class must declare that same slot. The schema rejects
more than one transformation in Group.CHANNEL, rejects any cross-group
backtracking (Group order must stay monotonic non-decreasing across the full
chain), and rejects adjacent transformations inside the same group when their
ordinals are not monotonic non-decreasing. The top-level name, params, and
snr_db_range fields remain only as compatibility placeholders; shipped
validation requires the default name, empty params, and the default
snr_db_range so config cannot silently set values runtime ignores.
Example: propagation-only chain¶
Transformation is the one closed set that is an IntEnum, not a StrEnum:
a chain entry is written as the slot’s integer, and the member name is rejected.
chain:
- transformation: 21 # Transformation.PROPAGATION
params: {}
# An installed third-party implementation can disambiguate a shared slot:
# - transformation: 15 # Transformation.CFO
# selector: my_channel_plugin
SceneConfig¶
class DensityConfig(BaseModel):
mode: DensityMode = DensityMode.RANGE # closed: rfgen.core.enums.DensityMode
min_emitters: int = Field(ge=0, default=1)
max_emitters: int = Field(ge=0, default=10)
poisson_rate: float | None = None # required when mode is DensityMode.POISSON
class EventDurationConfig(BaseModel):
mode: Literal["fixed"] = "fixed"
duration_s: float # finite, positive
class MultiRXConfig(BaseModel):
"""Multi-receiver layout. Either set `geometry` (preset array) OR `receivers` (explicit list)."""
geometry: ArrayGeometry | None = None # closed: rfgen.core.enums.ArrayGeometry
receivers: list[ReceiverConfig] = []
class SceneGeometryConfig(BaseModel):
backend: SceneGeometryBackend = SceneGeometryBackend.NONE # closed: rfgen.core.enums.SceneGeometryBackend
overlap_policy: SceneOverlapPolicy = SceneOverlapPolicy.ALLOW
rt_solver: RTSolverConfig | StatisticalSolverConfig | None = None
geometry_ingest: str | None = None # open: rfgen.geometry_ingests
class SceneConfig(BaseModel):
sample_rate_hz: float = Field(gt=0, default=20_000_000.0)
duration_s: float = Field(gt=0, default=0.020)
event_duration: EventDurationConfig | None = None
bandwidth_hz: float = Field(gt=0, default=10_000_000.0)
center_hz: float = 0.0 # rejects YAML/JSON bool, NaN, and +/-Inf on construction/assignment; zero remains baseband
density: DensityConfig = Field(default_factory=DensityConfig)
time_placement: TimePlacementStrategy | str = TimePlacementStrategy.IID_UNIFORM
# Per-strategy kwargs. The registry construction path reserves and
# injects scene_duration_samples, and conditionally sample_rate_hz.
time_placement_params: dict[str, object] = {}
frequency_placement: FrequencyPlacementStrategy = FrequencyPlacementStrategy.IID_UNIFORM
frequency_placement_params: dict[str, object] = {} # per-strategy kwargs
rx_array: RxArrayConfig = Field(default_factory=RxArrayConfig)
multi_rx: MultiRXConfig | None = None
channel_application: ChannelApplicationMode = ChannelApplicationMode.SCENE
geometry: SceneGeometryConfig = Field(default_factory=SceneGeometryConfig)
assets: SceneAssetsConfig = Field(default_factory=SceneAssetsConfig)
@model_validator(mode="after")
def _check_grid(self):
if self.sample_rate_hz < self.bandwidth_hz:
raise ValueError(
"scene.sample_rate_hz must be >= bandwidth_hz "
"for complex-baseband Nyquist sampling."
)
return self
RTSolverConfig¶
SceneGeometryConfig.rt_solver selects this typed contract when the geometry
backend is sionna_rt. SionnaRT forwards only the four effect flags below;
it does not expose Sionna diffraction, edge-diffraction, or
diffraction-lit-region controls.
Field |
Type and default |
Construction-time constraint and runtime meaning |
|---|---|---|
|
|
Must be |
|
|
Enables/disables line-of-sight paths. |
|
|
Enables/disables specular reflection. |
|
|
Enables optional Sionna diffuse scattering through its |
|
|
Enables/disables refraction. |
|
|
Forwarded to the Sionna PathSolver. |
|
|
An explicit value must be finite and |
|
|
Typed Sionna RT planar-array definitions. |
|
|
Forwarded to |
|
|
With the default, zero valid paths raise |
|
|
Must be |
|
|
Sionna merges every shape sharing a radio material into one scene object, which is what lets a large scene solve at all — and which leaves a shape an author needs to name with no name. Shapes whose scene-file id matches this expression stay separate objects under their own ids. Omitted by default, and omitted from the parameter dump when unset. |
|
|
Scene objects the initial-ray launch is biased toward. See |
|
|
Ray budget for the biased launch. See |
|
|
Refused. The asset was validated, hashed into the scene cache key, and written into record provenance as |
RTSolverConfig also inherits the shared discrete-time CIR window fields. A
finite-impulse-response (FIR) filter’s tap indices select the discrete-time
impulse-response window:
Field |
Type and default |
Constraint and meaning |
|---|---|---|
|
|
When set, must be |
|
|
The last FIR tap index. When authored it must be |
|
|
When set, must be |
|
|
When set, must be |
ArraySpec is the typed value used by both array fields:
Field |
Type and default |
Validation and meaning |
|---|---|---|
|
|
Each must be |
|
literal, |
One of |
|
literal, |
One of |
|
|
Optional stable array identifier; otherwise RFGen derives a deterministic identifier from the grid, pattern, and polarization. |
Target-biased sampling¶
Sionna launches initial rays uniformly over the whole sphere, so the fraction
reaching a small target falls as A_proj / (4 pi R^2). Past a range set by the
target’s size and the ray budget the returned power stops falling as R^-4 and
walks toward R^-2. That error is deterministic and grows with range, so it
does not average away.
Declaring a target adds a narrow cone of rays aimed at it, on top of the uniform budget the scene already had. The correction is folded into the probability Sionna’s field calculator already divides by, so the answer does not change — only what it costs. The environment is unaffected: cone rays are additive and the uniform component returns what it would have returned anyway.
This is an estimator, not a physics model. Nothing here declares what a target is: no cross-section, no material, no scattering behaviour.
TargetSpec¶
Field |
Type and default |
Constraint and meaning |
|---|---|---|
|
|
Must be non-empty, and must name an object the loaded geometry contains — a scene without it raises |
|
|
The estimator’s view of the target. Kept under its own key so a future declaration about the target lands beside it rather than inside it. |
TargetSamplingGeometry:
Field |
Type and default |
Constraint and meaning |
|---|---|---|
|
|
Must be |
|
|
|
bounding_radius_m and range_m are declared because rfgen validate binds
nodes without loading a scene or resolving another node’s pose, and the rules
below need both before a ray is cast. Neither is trusted. At solve time the
runtime resolves the object’s real bounds and the real transmitter range,
refuses the record if either differs from the declaration by more than 25% in
either direction, and re-applies the cone-angle and world-coordinate floors to
the geometry that actually resolved. Over-declaring is the direction that
matters: where the cone is narrower than the sphere, radius and range cancel out
of the ray-budget rule, so declaring a target larger than it is cannot move that
rule and would otherwise pass unnoticed while pushing the real geometry below a
floor it was never checked against.
TargetSamplingSpec¶
Field |
Type and default |
Constraint and meaning |
|---|---|---|
|
|
Enabled by default: there is no configuration in which the uniform launch is preferable, since it is both slower and biased. The switch exists to reproduce a corpus generated before this sampler did. |
|
|
Must be |
|
|
Cone half-angle as a multiple of the target’s angular radius. Must be |
What rfgen validate refuses¶
Every input these rules need is known before a ray is cast, so a configuration whose returns would be structurally wrong fails to generate rather than generating something a model would learn the artefact from. Each message names the knob, gives the number that would fix it, and — where the sampler is off — offers enabling it first.
With target_sampling.enabled: true:
Rule |
Refused when |
Why |
|---|---|---|
Cone margin |
|
The cone is aimed by the target’s largest half-extent and a box’s corners reach |
Hit floor |
fewer than 200 hits per target at the far end of |
Below that the returned power is not within 0.05 dB of its converged value. At the defaults this is |
Cone-angle floor |
half-angle |
Below it Sionna’s geometry stops resolving the cone; the returned power reads low by up to 1.4 dB. Not fixable by spending rays. |
World-coordinate floor |
|
Below it Mitsuba’s float32 world coordinates stop intersecting the mesh accurately; the power reads low by up to 0.7 dB and the loss depends on the mesh rather than the ray budget. |
Far field |
|
Warning, not a refusal. The target is not in the far field of its own bounding sphere at the near end of its range, so the level carries a near-field constant. The range law is unaffected. |
With target_sampling.enabled: false, the maximum-range rule applies instead:
a target is refused when range_m[1] exceeds
sqrt(scene_rays * A_proj / (4 pi * 20)), the range beyond which the uniform
launch stops delivering the 20 hits the R^-4 falloff needs.
Refused in either case: declaring the same object_id more than once, since two
cones on one object is one cone assembled from two halves of the budget with
each half validated as though it were the whole; declaring target_sampling
with no targets, since the block would be dropped from the parameter dump and
the setting silently lost; and declaring targets with synthetic_array: false,
since a cone is aimed from one source position while that option launches from
each antenna element separately.
MultiRXConfig and SceneGeometryConfig cross-field rules¶
MultiRXConfig.geometryandMultiRXConfig.receiversare mutually exclusive: setting both is apydantic.ValidationError. A populatedgeometrybuilds the array from the named preset; a populatedreceiverslist places receivers explicitly.SceneGeometryConfig.backend == SceneGeometryBackend.SIONNA_RTrequires a scene-geometry asset on the parentSceneConfig: the preferred typedassets.scene_geometry_ref, or the legacyassets.scene_geometry_uri. The validator raises withloc=("assets", "scene_geometry_uri")when neither is present.SceneAssetsConfig.material_db_ref, when present, must useGeometryAssetKind.MATERIAL_DB; everySceneAssetsConfig.antenna_pattern_refs[*]entry must useGeometryAssetKind.ANTENNA_PATTERN.SceneGeometryBackendvalues are lowercase enum strings in new configs:sionna_rt,mitsuba, andnone. Pydantic still accepts the legacy spellings"SionnaRT"and"Mitsuba"for backward compatibility.
Event duration policy¶
event_duration is optional. When omitted, every emitter event is generated
for the full scene.duration_s, preserving full-scene event generation.
The only currently supported policy is {mode: fixed, duration_s: ...}; it is
scene-wide, so all emitter slots in that scene use the same requested duration.
duration_s must be finite and positive, must span at least one exact sample
(sample_rate_hz * duration_s >= 1), and must not exceed scene.duration_s.
The resolved nested
object serializes with SceneConfig as ordinary record provenance.
The policy controls generic RF event extent only. It does not claim protocol fidelity for Wi-Fi, Bluetooth, drone links, packet timing, hopping, or calibrated captures. For both omitted and fixed policies, time placement is drawn after resampling and each returned start must retain the whole generated event in the scene; plugins that return an out-of-range start are rejected rather than cropped.
This event-extent disclaimer is distinct from the fidelity claims of the
EmitterFamily.DRONE emitters (droneid, fhss_rc_link, analog_fpv_video,
remote_id, ocusync_surrogate). Those emitters do reproduce, and validate,
the measurable PHY signatures of the signals they name: subcarrier spacing,
occupied bandwidth, Zadoff-Chu sync, hop cadence, deviation, and the like
(see use_cases/signal-atlas/cuas-v1/docs/validation/), but carry synthetic
payloads and, for
ocusync_surrogate, are explicitly non-protocol-faithful. Neither this generic
policy nor those emitters claim bit-decodable protocol conformance.
Example: configs/scene/dense.yaml¶
sample_rate_hz: 30.72e6
duration_s: 0.01
bandwidth_hz: 20e6
center_hz: 2.45e9
density:
mode: range
min_emitters: 4
max_emitters: 12
time_placement: iid_uniform
time_placement_params: {}
frequency_placement: stratified
frequency_placement_params:
min_spacing_hz: 1000
geometry:
backend: sionna_rt
overlap_policy: allow
assets:
scene_geometry_ref:
kind: mitsuba_xml_bundle
uri: file:///abs/path/assets/sionna/munich.xml
content_hash: sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
rx_array:
num_rx: 4
array: ula_4
spacing_lambda: 0.5
Selecting placement strategies¶
The two placement strategies are selected inside SceneConfig. Built-in names
are enum values; time_placement also accepts a non-empty installed
third-party entry-point name. Strategy-specific kwargs are carried separately
in time_placement_params and frequency_placement_params. For a
registry-selected time strategy, these are not a blind forwarding dictionary:
the composer reserves and supplies scene_duration_samples, plus
sample_rate_hz when the strategy constructor declares it. A conflicting
supplied value fails configuration/strategy construction rather than changing
the capture bound. This injection does not apply to
DefaultSceneComposer(time_planner_factory=...); that zero-argument factory
constructs its planner with its own context, after which the composer still
checks every returned start.
scene:
time_placement: event_periodic_beacon # TimePlacementStrategy value
time_placement_params:
period_seconds: 0.1024 # Wi-Fi TBTT default
frequency_placement: realistic_density # FrequencyPlacementStrategy value
frequency_placement_params:
taxonomy: wifi-2.4ghz
min_spacing_hz: 5_000_000.0
time_placementaccepts a TimePlacementStrategy member or a non-empty installedrfgen.time_placemententry-point name. Built-in enum values retain their enum type in Python.frequency_placementis a FrequencyPlacementStrategy member; the YAML value is also the entry-point key resolved through therfgen.freq_placementplugin registry.*_placement_paramsis forwarded to the selected strategy’s constructor after Pydantic validation.
See the Placement Strategy Selection Guide for which strategy to pick per scene type, evidence tiers, and copy-paste config recipes.
LabelConfig¶
class LabelerSpec(BaseModel):
name: str # open: rfgen.labelers registry name
params: dict[str, object] = {}
class LabelConfig(BaseModel):
name: str = "joint" # open: rfgen.labelers registry name
params: dict[str, object] = {}
extra_labelers: list[str | LabelerSpec] = []
seg_n_fft: int = 1024
seg_hop: int = 256
segmentation_mode: SegmentationMode = SegmentationMode.SINGLE_LABEL
segmentation_tie_break: SegmentationTieBreak = SegmentationTieBreak.LOWER_EMITTER_INDEX
Segmentation capability is selected by name: bbox stays metadata-only,
while segmentation and joint emit segmentation by construction.
segmentation_mode and segmentation_tie_break are the public mask-shape and
overlap-rule controls.
AnnotatorConfig¶
class LLMConfig(BaseModel):
provider: str # open: rfgen.inference_clients registry name
model: str # open: per-provider model id
base_url: str | None = None # required by some providers, e.g. openai_compatible
temperature: float = 0.2
max_tokens: int = 512
timeout_s: float = 60.0 # per-request transport timeout
json_schema_mode: StrictBool = True
class AnnotatorConfig(BaseModel):
enabled: StrictBool = True
# closed: rfgen.core.enums.AnnotationType
types: list[AnnotationType] = Field(default_factory=lambda: [AnnotationType.CAPTION])
bulk_llm: LLMConfig # high-volume, low-cost
verifier_llm: LLMConfig | None = None # optional second-pass verifier
verifier_subset_pct: float = Field(ge=0.0, le=100.0, default=0.0)
enabled and json_schema_mode use Pydantic StrictBool: only YAML/JSON
booleans (true/false) are valid. Numeric and string-like booleans such as
0, 1, "false", and "true" are rejected at config-validation time.
provider is open str because inference providers are plugin-resolved
through the rfgen.inference_clients entry-point group; the framework does not
ship a closed enum of provider names. model is a provider-local model
identifier and is validated only by the selected provider/client behavior.
base_url is optional and provider-interpreted: the shipped gemini provider
ignores it, while openai_compatible requires it and raises at construction
when it is absent.
openai_compatible targets any server that speaks the
OpenAI chat-completions wire format, including a local model server such as
Ollama, vLLM, or LM Studio, and a hosted OpenAI-compatible API such as
Together, Groq, or Fireworks; see rfgen.annotators’s inference-client
extension boundary.
The class name LLMConfig and the field names bulk_llm / verifier_llm are
retained for config-surface stability; the configured providers can be
text-only LLMs, vision-language models, or audio-language models.
scene_report_evidence_v2_max_tokens was removed along with the
measured-evidence annotation path. A config that still sets it fails validation
with a message naming the field; delete the line.
Example¶
enabled: true
types: [caption]
bulk_llm:
provider: gemini
model: gemini-3.1-flash-lite
temperature: 0.2
verifier_subset_pct: 0.0
types has one permitted value. Naming qa, reasoning, scene_report or
contrastive fails validation with a message saying the value was retired.
verifier_subset_pct accepts values from 0.0 through 100.0. 0.0
disables verifier scoring, 100.0 verifies the full annotation set, and
intermediate percentages require programmatic PAES membership selection before
records are handed to the annotator.
StorageConfig¶
class StorageConfig(BaseModel):
backend: StorageBackend | str = StorageBackend.SIGNAL_DATASET
path: str
assets_path: str | None = None
Three fields, because a store needs to be selected and told where to write.
path names the local or gs:// dataset root.
compression, chunk_samples, dataset_filename, and record_axis described
the HDF5 and WebDataset on-disk layouts. Neither store ships, so the fields are
gone from the model rather than validated and ignored: a configuration that
sets one is refused by name. record_axis in particular no longer has anything
to select — one sample publishes one record, and a multi-receiver scene carries
its receivers as named subtrees inside it. See Concepts / Records, Receivers, and Assets.
backend stays an open selector. A string that is not a known member is passed
through to the rfgen.dataset_stores entry-point group, so an installed
third-party store is reachable without a schema change, and owns its own path
validation.
Projection parameters¶
renderer_selector is a projection parameter — params.renderer_selector —
not a storage field. It resolves
through rfgen.scene_renderers and validates against the renderer class’s exact
ParamsModel. A projection names the renderer that produces what it observes;
both shipped projections default it to their own renderer.
validation_study is not a GenerationConfig field either. It lives on the
radar renderer’s parameters, beside the backend it gates, because the
composition root cannot see which backend a projection selected. It defaults to
false; selecting the vendor radarsimpy engine is rejected unless
validation_study: true declares the run an explicit validation study. Normal
generation uses the default backend.
ExecutorConfig¶
local is synchronous. dataproc_serverless is the shipped asynchronous
Dataproc Serverless selector.
class ExecutorConfig(BaseModel):
name: str = "local"
parallelism: int = Field(default=1, gt=0)
dataproc: DataprocServerlessConfig | None = None
parallelism is a positive requested partition count for Dataproc and is
ignored by local execution. executor.dataproc is required only when
name: dataproc_serverless; it has project (default rf-foundation-models),
region, required staging_uri (gs://), required service_account, optional
image_uri, labels, network or subnet (not both), spark_properties, and
extra_packages. Dataproc also requires a native dataset-store backend
(normally signal_dataset) with a gs:// path.
Field |
Type / default |
Nullable |
Contract |
|---|---|---|---|
|
|
no |
Open registry selector; |
|
|
no |
Requested PySpark partitions; ignored by local execution. |
|
|
yes |
Required exactly when |
|
|
no |
Trimmed non-empty GCP project. |
|
|
no |
Trimmed non-empty Dataproc Serverless region. |
|
|
no |
Trimmed |
|
|
no |
Trimmed worker service-account email. |
|
|
yes |
Optional trimmed runtime image. Defaults to the maintained image pinned by digest in |
|
|
no |
Empty is allowed. Each pair is passed verbatim as |
|
|
yes |
At most one may be set; both are trimmed when present. |
|
|
no |
Empty is allowed. Each pair is passed verbatim as |
|
|
no |
Local |
The validator rejects blank string settings, a non-GCS staging URI,
simultaneous network and subnet, a missing Dataproc block for the Dataproc
selector, a Dataproc configuration whose storage is a legacy fixed-IQ backend
or whose path is not gs://, and an extra_packages entry that is blank,
names a remote reference, does not end in .whl or .zip, or collides with
another entry’s staged filename. Whether a local entry actually exists, is a
valid zip archive, and declares an rfgen.* entry-point group is deliberately
not a schema rule: rfgen validate may run before the artifact is even
built, so that check runs only immediately before rfgen generate stages the
submission, against the local filesystem in hand at that moment. See
Shipping plugin packages.
Example: Local¶
# configs/executor/local.yaml
name: local
parallelism: 1
See Dataproc Serverless for the remote example and operational prerequisites.
AnnotationConfigV1¶
rfgen annotate submit composes a separate, versioned AnnotationConfigV1;
it is not a field in GenerationConfig. The closed source/execution matrix is
validated before Spark or a provider is created:
class AnnotationConfigV1(BaseModel):
source_kind: Literal["signal_dataset"] = "signal_dataset"
execution_mode: Literal["local", "dataproc_serverless"]
annotation_type: Annotated[str, Field(pattern=r"^[a-z][a-z0-9_-]{0,62}$")] # the pattern permits underscores
template_id: TemplateId
run_id: Slug
model: str
backend: str = "local_concurrent" # an open registry selector
inference: AnnotationInferenceV1 = AnnotationInferenceV1()
signal_dataset: SignalDatasetSourceV1 | None
dataproc: DataprocSparkV1 | None
declared_evidence: DeclaredEvidenceConfigV1 # defaulted, not optional
signal_dataset names a published Signal Dataset snapshot, which is what
rfgen generate writes. It reads that snapshot by path alone, so the same
reader serves a local path and a gs:// bucket. It requires a
signal_dataset block. Use local_concurrent (one provider call per record)
for a smoke test or gemini_batch (many records per provider job, roughly half
the price) for a dataset.
execution_mode: dataproc_serverless submits a batch that runs the same loop
execution_mode: local runs in process, so the two modes differ only in where
that loop executes. It requires dataproc.region, a project
dataproc.project, and a gs:// dataset_uri, because a cluster cannot read the submitting
machine’s disk. Naming Dataproc without those is refused rather than accepted:
that combination used to run the whole job locally while its operator waited
for a cluster job nobody had submitted.
Availability limit. Local execution against this source runs end to end.
backend: local_concurrent (one provider call per record) with
execution_mode: local is qualified against a real bucket as well as a local
path. See Signal Dataset annotation over
gs://.
gemini_batch (many records per provider job) is qualified over a local path
and a gs:// root alike, and as a Dataproc Serverless batch. See Signal
Dataset annotation with backend: gemini_batch.
execution_mode: dataproc_serverless is accepted, and submits that same
loop as a Dataproc batch; how far that path is qualified is stated in
rfgen.annotation.
A backend must implement BaseAnnotationExecutor; the check is by class rather
than by name, so a third-party backend is held to the same contract.
source_kind defaults to signal_dataset: the source rfgen generate
writes by default is the one annotation reads by default, so generating and then
annotating needs no source selection in between.
See Annotate an existing dataset
for a copyable configuration and the lifecycle commands, and
rfgen.annotation’s annotation-backend extension
boundary for the
backend selector’s registry contract.
local_hdf5, remote_webdataset, vertex, spark, and local are gone. A
configuration naming one is refused by name rather than migrated: it describes a
source or a runtime this build cannot reach, and silently ignoring the key would
run the job against something other than what was written.
backend and inference: pluggable annotation execution¶
backend is an open selector, resolved through the rfgen.annotation_backends
entry-point group, mirroring ExecutorConfig.name: a new backend does not
require a schema change here. inference configures the inference clients the
backend resolves.
Field |
Type, default |
Effect |
|---|---|---|
|
open str; |
Selects the registered |
|
open str; |
Selects the registered |
|
Secret Manager resource name; optional |
Where the driver reads the provider’s API key. A resource name, never a key: |
|
list of |
One round-robin endpoint per entry, each with its own |
|
int > 0; |
Requested |
|
dict[str, float]; |
Optional per-model USD-per-million-token price hints for cost reporting; an absent entry reports as unknown/$0. Local and self-hosted models have no per-token provider cost. |
|
int > 0; |
Bounds one wave’s in-flight driver memory; not a fixed hardware or API limit. See |
|
float > 0; |
Per-request transport timeout applied to every endpoint in the group, and to the single client built when |
Why a secret name and not a key¶
inference.api_key_secret holds a resource name. Any other shape, including
something that looks like a key, is refused at validation rather than as a 404
inside a batch that has already been billed for.
It is effectively mandatory for execution_mode: dataproc_serverless whenever
the provider needs a key, and omitting it is not refused at validation, so it
fails every row of a batch you have already paid for. A keyless
openai_compatible endpoint does not need it.
Two reasons the key itself never travels:
A batch cannot read the submitting machine’s environment.
Passing a key through a Spark property would write it into the batch resource, where
gcloud dataproc batches describeechoes it to anyone holdingdataproc.batches.get.
The driver resolves the secret under its own identity, which needs
roles/secretmanager.secretAccessor on the secret rather than on the
project. Needs the gcs extra.
InferenceEndpointV1 has two required fields, base_url (non-empty string)
and model (non-empty string), plus optional timeout_s (float > 0). A
per-endpoint timeout_s overrides inference.timeout_s; leaving it unset
inherits the group value. Endpoints in the same round-robin group may run
different model builds behind different servers, and may be sized for very
different response times.
declared_evidence: DeclaredEvidenceConfigV1¶
Optional. It defaults to the full registered field set, so annotating needs
no declared_evidence stanza at all; set it only to narrow what the evidence
surfaces. caption.declared.v1 is the one shipped template, so there is no
other template to forbid it for. It configures the declared-metadata evidence
pipeline
(see declared-metadata annotation
evidence). All values are
strict; unknown fields are rejected.
Field |
Type and default |
Effect |
|---|---|---|
|
int in |
Maximum components included per scene. The evidence-model absolute ceiling is 64; the cap is a product bound below it. |
|
|
|
|
list of registered quantity names; |
Selects the per-component numeric claims. Every name must have a registered extractor; unknown names fail validation. |
|
list of registered label names; |
Selects the per-component declared labels. Absent metadata fields are omitted, never fabricated; identifier-shaped values are rejected by the canonical-label sanitizer. |
|
bool; |
Emits exact pairwise time/frequency overlap claims for every component pair. |
|
int in |
Provider generation capacity for the narrative-only response. |
Source, local, Vertex, and Dataproc nested models¶
The following fields are the remaining nested AnnotationConfigV1 models.
Required means required whenever the model is present; the capability matrix
below determines when that model is present.
Model and field |
Type and default |
Conditional presence and effect |
|---|---|---|
|
non-empty string; required |
Present only for |
|
non-empty list of annotation requests; optional |
Present only with |
|
non-empty selector; |
Present with |
|
non-empty string; optional |
The project the batch is submitted to. Falls back to |
|
GCS URI (bucket and an object prefix); required |
Present for |
|
non-empty trimmed string, not |
Present for |
|
integer 1-1000; defaults to 20 |
Read under |
|
non-empty string; optional |
Present for |
signal_dataset requires a signal_dataset block and accepts both execution
modes, because a Dataproc batch runs the same loop the source runs locally.
execution_mode: local forbids dataproc. dataproc_serverless requires a
dataproc block naming region and project, and requires
signal_dataset.dataset_uri to be a gs:// URI, since a cluster cannot read
the submitting machine’s disk. Naming Dataproc without those is refused rather
than accepted: that combination used to run the whole job locally while its
operator waited for a cluster job nobody had submitted.
Validation does not constrain backend; AnnotationRunner.run refuses one
whose resolved class does not implement BaseAnnotationExecutor, before any
inference.
CredentialsConfig¶
class CredentialsConfig(BaseModel):
provider: str = "static"
params: dict[str, object] = Field(default_factory=dict)
In most direct-Gemini setups this is omitted entirely: the command reads
GEMINI_API_KEY. When present, this block configures retained static local
credentials only.
Static credentials use the existing per-scope provider schema:
credentials:
provider: static
params:
scopes:
llm:
api_key: ${oc.env:GEMINI_API_KEY}
Configuration changes¶
Edit a materialized config.yaml, then validate the directory before
generation. Historical Hydra config-group and multi-run examples are not part
of the supported tutorial contract.
rfgen validate --config-dir ./my-config
rfgen generate --config-dir ./my-config --output ./new-output
Validation timing¶
Validation runs before any sample generation begins:
Hydra resolves the config tree.
GenerationConfig.model_validate(…) is called.
Every plugin’s
schema()validates its ownparamsblock.Any failure raises ConfigError with a path to the offending field.
A failed config is rejected before local generation or a Gemini request.
Contract Tests¶
The implementation must carry tests for these invariants:
Test |
Expected behavior |
|---|---|
Minimal local config |
A complete local config validates and materializes the emitter, channel, scene, placement, label, annotator, and storage blocks plus executor settings. |
Unknown closed-enum value |
A closed-set field such as |
Open plugin name |
|
Plugin params |
Each plugin validates its own |
Retired storage layout option |
|
Multi-RX exclusivity |
|
Scene geometry asset |
|
Round-trip |
|
See Also¶
Reference / API / Config - proposed Pydantic class surface.
Reference / API / Enums - closed value sets used by this schema.
Reference / Plugin Metadata - open plugin naming and compatibility contracts.
Background / Open Questions - unresolved schema questions.