Project Layout

Note

Layer 3 shipped, Pass-1. The Layer 3 implementation (emitters, device-fingerprint, tx-impairments, propagation, rx-frontend) landed on branch rfgen-impl-2026-06-25-105955 (PR #94). The class names, Pydantic schemas, and Transformation enum members referenced below match the shipped surface; Pass-1 stubs (GNU Radio OOT emitters, cellular emitters, Sionna propagation backends) construct cleanly and raise an EmitterError or ChannelError tagged with stage="pass1_stub" until backend wiring lands. See Reference / rfgen.emitters and Reference / rfgen.channels for the shipped class roster.

This page lists the shipped repository structure first, then labels future targets explicitly. Every path should be reviewed against API Reference or Config Schema. Paths marked planned do not exist yet.

Top-level tree

rf-data-generation/
├── README.md
├── pyproject.toml                  # build + dependency declaration (PEP 621)
├── .github/
│   └── workflows/
│       ├── docs.yml                # publish docs site to GitHub Pages
│       ├── test.yml                # (planned) pytest matrix
│       └── lint.yml                # (planned) ruff + mypy
├── docs/                           # Sphinx documentation sources (this site)
│   ├── conf.py
│   ├── requirements.txt            # docs-only deps
│   ├── _static/
│   ├── scripts/                    # docs-build helper scripts
│   └── (content directories: getting-started, concepts, how-to,
│       reference, signal-catalog, scenario-presets, background)
├── src/
│   └── rfgen/                      # the framework package
├── tests/                          # pytest suite, mirrors src/rfgen/
├── configs/                        # planned Hydra config groups (not present yet)
├── scripts/                        # one-off ops scripts (Spark submit wrappers, etc.)
└── examples/                       # planned runnable examples (not present yet)

src/rfgen/ package tree

The tree below is the shipped filesystem on this branch. A path ending in .py is a module, while a path ending in / is a package. Do not create a package directory with the same import name as an existing module. In particular, rfgen.storage and rfgen.orchestration currently resolve to storage.py and orchestration.py.

src/rfgen/
├── __init__.py                     # version plus the small root public facade
├── cli.py                          # Typer console entry point (`rfgen` script)
├── public.py                       # root `open_store` and doctor diagnostics
├── execution_evidence.py           # Layer 11 immutable evidence records + validator
├── core_types.py                   # Record, Signal, metadata, asset refs
├── enums.py                        # framework-owned closed choices
├── errors.py                       # framework exception hierarchy
├── registry.py                     # entry-point and manifest registries
├── rng.py                          # deterministic seed schedule
├── credentials.py                 # credential-provider ABC and shipped providers
├── log_sinks.py                    # logging sink ABC and shipped sinks
├── channel_pipeline.py             # channel ABCs and ordered pipeline
├── propagation.py                  # propagation backends
├── tx_impairments.py               # transmitter impairments
├── rx_frontend.py                  # receiver frontend
├── record_rf_context.py            # record RF-context helpers
├── scene.py                        # placement-facing scene contracts
├── scene_composer.py               # shipped scene composer
├── labels.py                       # labelers and label contracts
├── storage.py                      # Layer 7 store ABCs and all shipped stores
├── orchestration.py                # Layer 9 shard worker and result contracts
├── executors.py                    # generation command and executor implementations
├── device_fingerprint.py           # device-fingerprint channel
├── emitters/
│   ├── __init__.py
│   ├── base.py                     # BaseEmitter and EmitterLike
│   ├── adsb.py
│   ├── cellular.py
│   ├── gnuradio_oot.py
│   ├── lora.py
│   ├── radar.py
│   └── torchsig_*.py               # shipped TorchSig emitter families
├── channels/
│   ├── __init__.py
│   └── protocols.py                # structural channel protocols
├── placement/
│   ├── __init__.py
│   ├── base.py
│   ├── channel_plans.py
│   ├── config.py
│   ├── frequency.py
│   └── time.py
├── config/                         # shipped Pydantic v2 models
│   ├── __init__.py
│   ├── _root.py                    # GenerationConfig composition root
│   ├── annotation.py
│   ├── channel.py
│   ├── credentials.py
│   ├── emitter.py
│   ├── executor.py
│   ├── label.py
│   ├── placement.py
│   ├── reference_host.py
│   ├── run.py
│   ├── scene.py
│   └── storage.py
├── dataset/                        # consumer-side dataset adapters
│   ├── __init__.py
│   ├── collators.py
│   ├── samplers.py
│   └── torch_adapter.py
├── inference/
│   ├── __init__.py
│   ├── clients.py
│   ├── protocols.py
│   └── testing.py
├── annotators/__init__.py          # shipped annotation surface
├── schemas/
│   └── execution-evidence-v1.schema.json  # packaged Draft 2020-12 evidence schema
├── cli_data/                       # packaged Hydra configs and scenario presets
└── data/channel_plans/             # packaged channel-plan JSON assets

Planned package splits

No src/rfgen/storage/, src/rfgen/orchestration/, src/rfgen/cli/, or src/rfgen/presets/ directory exists today. The shipped console command lives in cli.py, and the Layer 11 evidence schema is package data under schemas/; use importlib.resources.files("rfgen") / "schemas" to read that installed schema rather than a repository-relative path. A future split of a same-named .py module must move it atomically and preserve its public imports; adding a directory beside the module is not a valid intermediate state.

configs/ (planned Hydra config groups)

The repository does not yet contain a top-level configs/ directory. The tree below is the Layer 10 target: Hydra will discover each directory as a config group after those files land.

configs/
├── config.yaml                     # top-level defaults list
├── emitter_zoo/
│   ├── comms_only.yaml
│   ├── heterogeneous.yaml
│   ├── radar_only.yaml
│   └── drone_focus.yaml
├── channel/
│   ├── awgn.yaml
│   ├── torchsig_impairments.yaml
│   ├── sionna_rt_urban.yaml
│   ├── sionna_phy_tdl_a.yaml
│   └── sionna_phy_cdl_c.yaml
├── scene/
│   ├── narrowband.yaml
│   ├── wideband.yaml
│   ├── dense.yaml
│   ├── sparse.yaml
│   └── multi_rx.yaml
├── label/
│   ├── bbox_only.yaml
│   ├── seg_only.yaml
│   └── joint.yaml                  # default: bbox + seg + metadata
├── annotator/
│   ├── none.yaml
│   ├── caption_only.yaml
│   ├── full_suite.yaml             # all 5 annotation types
│   └── verifier_subset.yaml
├── storage/
│   ├── zarr_local.yaml
│   ├── zarr_gcs.yaml
│   ├── zarr_s3.yaml
│   ├── zarr_azure.yaml
│   ├── webdataset_shards.yaml
│   └── hdf5_torchsig.yaml
├── executor/                       # Phase 1 distributed execution
│   ├── local.yaml
│   ├── spark_byo.yaml              # bring-your-own Spark cluster
│   ├── managed_spark_serverless.yaml   # GCP
│   ├── emr.yaml                    # AWS
│   ├── synapse.yaml                # Azure
│   ├── databricks.yaml             # multi-cloud
│   ├── dask.yaml
│   └── ray.yaml
├── annotation_orchestrator/        # Phase 2 batch inference submission
│   ├── local_loop.yaml
│   ├── vertex_batch.yaml           # GCP
│   ├── sagemaker_batch.yaml        # AWS
│   ├── azure_batch.yaml            # Azure
│   ├── anthropic_batch.yaml        # cloud-agnostic
│   └── openai_batch.yaml           # cloud-agnostic
└── credentials/
    ├── static.yaml
    ├── gcp_adc.yaml
    ├── aws_default.yaml
    └── azure_default.yaml

The sionna_phy_* filenames remain compatibility-oriented preset names. The shipped schema keeps ChannelConfig.name pinned to the legacy placeholder torchsig_impairments, and live channel behavior is selected through ChannelConfig.chain.

tests/

The shipped tests use one unit-test module per current source area. Integration and validation directories exist, but the proposed reusable ABC contract suites have not landed.

tests/
├── conftest.py                     # shared fixtures
├── unit/
│   ├── test_storage.py             # shipped stores and StoreHandle contract
│   ├── test_orchestration.py       # Layer 9 shard-worker contract
│   └── test_<source_area>.py       # remaining module-focused tests
├── integration/
│   └── test_live_gemini_annotation.py
├── validation/__init__.py
├── conf/                           # test-only configuration fixtures
└── fixtures/                       # deterministic expected-output fixtures

examples/ (planned)

No top-level examples/ directory exists today. The target is one self-contained file or notebook per example:

examples/
├── 01_single_emitter_bpsk.py
├── 02_multi_emitter_scene.py
├── 03_swap_channel_to_sionna_rt.py
├── 04_custom_emitter_plugin.py
├── 05_annotate_existing_zarr.py
├── 06_train_loop_webdataset.ipynb
└── 07_validate_against_capture.py

Module boundaries: the rules

  1. Layers depend downward only. Scene composition may import core types, emitters, and channels; those lower modules do not import scene composition.

  2. Flat foundation modules are dependency roots. core_types.py, enums.py, errors.py, rng.py, and registry.py do not import higher orchestration behavior.

  3. config/ owns schemas, not runtime execution. Plugin discovery lives in registry.py; store construction lives at the storage boundary.

  4. Top-level services compose lower layers. orchestration.py builds shard workers; cli.py owns argument parsing and imports command owners lazily; execution_evidence.py imports only its typed records, trusted plan parser, and Git adapter. These services are top-level entry points, not dependencies of lower layers.

  5. Backends are optional extras. Sionna, srsRAN, RadarSimPy, GNU Radio adapters are gated behind pip install rfgen[sionna]-style extras and import lazily inside their adapter modules.

  6. No layer imports a concrete backend at the package level. from rfgen.channels import sionna_rt is fine; from rfgen.channels import SionnaRT at __init__.py would force-import the dependency.

What lives where: quick lookup

You want to…

Look in

Add a new emitter family today

Add a module under src/rfgen/emitters/; test it in tests/unit/test_emitters.py

Add a new channel model today

Use the relevant shipped flat module (channel_pipeline.py, propagation.py, tx_impairments.py, or rx_frontend.py); add its unit tests beside the matching current test module

Change scene composition policy today

src/rfgen/scene_composer.py; tests in tests/unit/test_scene_composer.py

Adjust label format today

src/rfgen/labels.py + Reference / Label Schema

Add a new annotation type today

Extend the shipped src/rfgen/annotators/ surface; Layer 10 template subpackages remain planned

Add a built-in storage backend today

src/rfgen/storage.py; tests in tests/unit/test_storage.py. Do not create src/rfgen/storage/ while the module exists

Add a third-party storage backend today

Implement the rfgen.storage ABCs in the external package and register an rfgen.stores entry point; see Add a custom store

Change the Layer 9 shard worker today

src/rfgen/orchestration.py; tests in tests/unit/test_orchestration.py. Do not create src/rfgen/orchestration/ while the module exists

Change the installed CLI or doctor/evidence commands today

src/rfgen/cli.py; keep command ownership in its service module (executors.py, annotation_orchestrators.py, or audit.py) and test CLI/evidence contracts in tests/unit/test_layer11_public_evidence.py

Change Layer 11 evidence records or semantic validation today

src/rfgen/execution_evidence.py, with packaged schema src/rfgen/schemas/execution-evidence-v1.schema.json; test in tests/unit/test_layer11_public_evidence.py

Add a credentials provider today

src/rfgen/credentials.py; cloud-specific implementations may live in an optional external package

Add a log sink today

src/rfgen/log_sinks.py; cloud-specific implementations may live in an optional external package

Add a scenario preset

Planned package/config work; no src/rfgen/presets/ or top-level configs/ directory exists yet

For the cloud-extension story (which subclasses ship in which extra, how to add a new cloud), see Reference / Cloud Backends.

Open layout questions

These are unresolved; reviewers, please weigh in.

  1. Should validation/ live inside rfgen or as a sibling package rfgen-validate? Pulling USRP/HackRF drivers in as optional extras keeps it together; splitting it keeps the core import surface lean.

  2. presets/ as YAML files inside the package vs. shipped under configs/presets/? Inside-the-package wins on rfgen list-presets discoverability; shipping under configs/ wins on Hydra-native overrides.

  3. emitters/cellular/ vs. emitters/lte/ + emitters/nr/? Cellular groups them by domain, but LTE (srsRAN/ZMQ) and NR PUSCH (Sionna PHY) share almost no code.

  4. Where do hardware-fingerprint parameters live, per-emitter or in channels/? Resolved: the per-device parameter store (DEVICE_REGISTRY, keyed by device_id) lives in emitters/fingerprint.py because the parameters belong to the device; the impairment operator (DeviceFingerprint, a BaseChannel) lives in channels/, so emitters keep producing clean baseband (snr_db = +inf) and the channel layer owns all impairment. channels imports the read-only parameter store from emitters (a downward dependency the layering rules permit).