Export or convert a dataset

Warning

Pre-implementation. Commands describe the target CLI surface.

Export the canonical Zarr dataset into a format expected by a downstream training or analysis workflow.

When to use this

Use Zarr as the source of truth. Export when a consumer needs WebDataset shards, TorchSig-compatible HDF5, SigMF, or a smaller local subset.

Prerequisites

Start from a generated dataset:

rfgen inspect ./out/local-smoke summary

Minimal command path

Export WebDataset shards:

rfgen export ./out/local-smoke \
    --format webdataset \
    --output ./out/local-smoke-wds

Export TorchSig-compatible HDF5:

rfgen export ./out/local-smoke \
    --format hdf5 \
    --output ./out/local-smoke.h5

Export a SigMF subset:

rfgen export ./out/local-smoke \
    --format sigmf \
    --sample-filter 'family == "radar"' \
    --limit 100 \
    --output ./out/radar-sigmf

Verify

rfgen inspect ./out/local-smoke-wds summary
rfgen inspect ./out/local-smoke.h5 summary

For a lossy or subset export, compare expected and exported counts.

Troubleshoot

Symptom

Fix

HDF5 export drops fields

Check TorchSig interop docs; extra rfgen metadata may move to extras.

WebDataset shards are too large

Lower shard size or sample count per tar.

SigMF export duplicates geometry

Prefer content-addressed assets unless shipping self-contained examples.

Next steps