Generate a small local dataset

Warning

Pre-implementation. Commands describe the target CLI surface.

Generate a small dataset on your workstation before using Spark, GPUs, or cloud storage. This page is a proposed command contract: it shows the CLI shape the implementation should support, not commands that work in the current pre-implementation repository.

When to use this

Use this for smoke tests, config iteration, emitter debugging, and quick inspection. A local run writes a small dataset under storage.path so you can check records, in-phase/quadrature (IQ) shape, and labels before moving to a distributed executor. Do not use it for production-scale datasets or expensive ray-traced scenes.

Prerequisites

pip install "rfgen[torchsig]"

The torchsig extra installs TorchSig-backed communication emitters used by the baseline presets. See Emitter Library Landscape for the backend policy.

Pick a local output directory that can be deleted and regenerated.

Minimal command path

Select a scenario preset with Hydra’s +preset=... override syntax. Hydra is the config-composition library behind the proposed CLI; the leading + adds a preset config group. The _xs suffix names the small local variant used for this smoke path. Exact preset fields can be inspected with rfgen show-preset; the catalog starts at Scenario Presets.

rfgen generate +preset=narrowband_classifier_baseline_xs \
    run.num_samples=128 \
    run.seed=1234 \
    orchestration=local \
    storage.path=./out/local-smoke

The overrides mean:

Override

Meaning

run.num_samples=128

Generate 128 records for a small smoke test.

run.seed=1234

Use a fixed random seed so the run can be repeated.

orchestration=local

Use the local executor config group instead of Spark or cloud execution.

storage.path=./out/local-smoke

Write the dataset under this local directory.

For more override examples, see Modify a scenario preset and Reference / CLI.

For a slightly richer wideband detection smoke test:

rfgen generate +preset=wideband_detection_baseline_xs \
    run.num_samples=256 \
    run.seed=1234 \
    orchestration=local \
    storage.path=./out/wideband-smoke

The narrowband classifier baseline predicts one class for each record. The wideband detection baseline predicts time-frequency boxes for emitters inside a receiver capture. See Labels for the label modalities.

Verify

rfgen inspect ./out/local-smoke summary
rfgen inspect ./out/local-smoke distribution --bins 10
rfgen inspect ./out/local-smoke sample --random --n 3

Expected checks:

Command

Should report

summary

Sample count, storage schema version, shard count, total bytes, and canonical IQ shape.

distribution --bins 10

Class counts and any preset-defined numeric distributions, such as signal-to-noise ratio bins.

sample --random --n 3

Three records with IQ shape, label types, class metadata, and component metadata.

Check that sample count, class distribution, IQ shape, and label types match the resolved preset. Run rfgen show-preset narrowband_classifier_baseline_xs when you need the expected values.

Troubleshoot

Symptom

Fix

Output path already exists

Re-run with the same command to resume, or change storage.path.

Validation fails before generation

Run the validation command below with the same +preset, run.*, orchestration, and storage.path overrides, then fix the reported config field.

Dataset is too slow locally

Reduce run.num_samples, use a smaller preset, or switch to Run on Spark.

 
rfgen validate +preset=narrowband_classifier_baseline_xs \
    run.num_samples=128 \
    run.seed=1234 \
    orchestration=local \
    storage.path=./out/local-smoke

Next steps