Annotation Templates

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, config field names, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

The Phase-2 annotation pipeline is normative: every prompt structure, output schema, closed-vocabulary list, and validation rule is specified here. Implementations must produce annotations that conform.

Note

Adopted from RF-GPT (template + LLM-rewrite pattern), WavCaps (3-stage filter+reformat+caption), ShareGPT4V (verifier-tier), Qwen-Audio (hierarchical-tag conditioning), RF-Analyzer (PAES). See Background / Literature Review.

Prompt structure

Every prompt is a triple (system, user, output_schema) produced by a pure deterministic templater:

templater = Templater(closed_vocab=ClosedVocab(...))
prompt = templater.render(record, annotation_type, template_id)

The templater consumes only the StoredRecord returned by whitelist_filter(record, sample_id=...). It never reads iq.zarr, never sees the raw signal. The ALLOWED VOCABULARY block is rendered from the ClosedVocab owned by the Templater instance, not passed as a separate render-time argument.

Whitelist

Only fields below enter any prompt. This is the metadata-only grounding principle made operational.

Source

Permitted fields

scene

bandwidth_hz, center_freq_hz, sample_rate_hz, duration_s, num_emitters, signal_density, noise_floor_dbm, realized_snr_distribution, realized_class_counts, realized_taxonomy_top_counts, multi_rx, num_rx, scene_preset_name

per_emitter[i]

class_name, taxonomy_path, source_dataset, source, emitter_offset_hz, bandwidth_hz, start_sample, duration_samples, low_freq_hz, high_freq_hz, snr_db, power_dbm, noise_floor_dbm, channel_profile, channel_params.{backend, profile, delay_spread_ns, k_factor_db, max_doppler_hz, num_taps}, aoa_deg, path_loss_db, delay_spread_rms_s, doppler_hz, rx_snr_db

per_emitter[i].extras

psdu_length, data_rate_mbps, phy, channel_index, pdu_length, intrapulse_modulation, pulse_width_us, pri_us, num_pulses, sf, bw, cr (LoRa), df (Mode-S), and per-emitter-family allowlists in rfgen.annotators

bbox[i]

class_name, taxonomy_path, abs.{start_sample, stop_sample, low_freq_hz, high_freq_hz}

Excluded by policy:

  • device_fingerprint_id: opaque hash; would just be hallucination bait.

  • extras.fingerprint_params: per-device CFO/IQ/PA params; we do not want the LLM to invent fingerprint signatures.

  • extras.channel_taps: full per-tap delays/powers; the compound channel_params summary is enough.

  • realization_seed, config_hash, scene_seed: provenance, not useful in prose.

  • runtime_executor_pool, runtime_seconds: operational metadata.

The whitelist is enforced in one place: rfgen.annotators.whitelist_filter(record, sample_id=...) -> StoredRecord. Templates consume only the filter’s output.

System prompt: the forbid clause

Every system prompt carries this fixed clause (versioned with template_id):

You must describe only the attributes given in the FACTS section. You must not invent, infer, or speculate about any attribute that is not stated. You must use only terms from the ALLOWED VOCABULARY list. If a fact is missing, omit any mention of it; do not guess. Do not describe how the signal “looks,” “sounds,” or “appears”; you have not seen it.

Closed vocabulary

Auto-generated from:

  1. The full hierarchical taxonomy (Reference / Label Schema § Hierarchical taxonomy), covering every legal class_name and taxonomy_path node.

  2. The channel-profile list (tdl-a, tdl-b, tdl-c-300, cdl-a-100, rt:munich/..., perfect, …).

  3. Physical-attribute units (Hz, kHz, MHz, GHz, dB, dBm, µs, ms, samples, MSps).

  4. Generic prose descriptors (bursty, continuous, narrowband, wideband, coexisting, weak, strong, dominant).

Validator rule: if the rewrite contains a token unknown to the closed vocabulary and claims a physical attribute, fail the record. Generic prose tokens that do not make physical claims are not constrained.

Versioning

Each template carries a template_id like "caption.dense.v3". Bumping the version writes to a parallel column rather than overwriting:

samples/NNNNNN/text/caption/
├── caption.dense.v2/<run_id>/
└── caption.dense.v3/<run_id>/

Both runs are queryable; manifest.text_runs_current records which is canonical.

The five annotation types

1. Dense caption: annotation_type="caption"

Purpose. Single fluent paragraph (60–120 tokens) describing the scene. Trains the CLIP-for-RF contrastive head.

Output schema:

class CaptionOutput(BaseModel):
    caption: str = Field(min_length=20, max_length=600)

Token budget. ~120 in + ~120 out ≈ 240 tokens. max_output_tokens=200.

Prompt template caption.dense.v3:

SYSTEM
You are an RF metadata-to-caption rewriter. {forbid clause from above}.
Output must be valid JSON matching the OUTPUT SCHEMA.

ALLOWED VOCABULARY
classes: {full taxonomy injected}
channels: tdl-a, tdl-b, tdl-c, tdl-c-300, cdl-a-100, perfect, rt:munich/..., ...
units: Hz, kHz, MHz, GHz, dB, dBm, µs, ms, samples, MSps
descriptors: bursty, continuous, narrowband, wideband, coexisting, weak,
  strong, dominant

OUTPUT SCHEMA
{
  "caption": {"type": "string", "maxLength": 600,
              "description": "A single paragraph, 60–120 tokens, factual, no speculation."}
}

USER
FACTS
scene:
  bandwidth_hz: {scene.bandwidth_hz}
  center_freq_hz: {scene.center_freq_hz}
  duration_s: {scene.duration_s}
  num_emitters: {scene.num_emitters}
  signal_density: {scene.signal_density} emitters/MHz
  noise_floor_dbm: {scene.noise_floor_dbm}
emitters:
{for each emitter:}
  - class_name: {emitter.class_name}
    emitter_offset_hz: {emitter.emitter_offset_hz}
    bandwidth_hz: {emitter.bandwidth_hz}
    start_sample: {emitter.start_sample}
    duration_samples: {emitter.duration_samples}
    snr_db: {emitter.snr_db}
    channel_profile: {emitter.channel_profile}
    extras: {whitelisted_extras}

TASK
Produce one caption (60–120 tokens) describing the scene. Mention each emitter
once. Quote SNR and bandwidth in the units given. Mention coexistence if
emitters overlap in time or frequency. Do not describe the channel beyond its
profile name.

Example output:

{
  "caption": "A 100-MHz wideband recording at 2.45 GHz contains three coexisting emitters across 100 ms. A bursty Wi-Fi 11g OFDM-20MHz transmission at 2.452 GHz dominates with 22.1 dB SNR for 1.2 ms at the data rate of 54 Mbps. A narrow BLE GFSK 1M advertising packet on channel 37 (2.402 GHz) appears for 380 µs at 14.7 dB SNR. A Barker-13 phase-coded pulsed radar at 2.485 GHz spans the full duration with 100 pulses at 1 ms PRI and 8.2 dB SNR. All three traverse a TDL-C 300 ns channel."
}

2. Attribute Q&A: annotation_type="qa"

Purpose. 5–10 Q&A pairs grounded in metadata. Mix of classification, value-extraction, counting, comparison, free-form. For RF-LLM instruction tuning.

Output schema:

class QAPair(BaseModel):
    q: str
    a: str
    qa_type: Literal["classification", "value_extraction", "counting",
                     "comparison", "freeform"]

class QAOutput(BaseModel):
    qa: list[QAPair] = Field(min_length=5, max_length=10)

Token budget. ~150 in + ~250 out ≈ 400 tokens. max_output_tokens=400.

Task instruction:

Generate 5–10 question-answer pairs about the scene. Each question must be answerable from the FACTS alone. Mix question types: classification (1–2), value-extraction (2–3), counting (1), comparison (1), free-form description (1–2). Answers must be terse and factual.

Example output:

{
  "qa": [
    {"q": "How many emitters are in the scene?", "a": "Three.", "qa_type": "counting"},
    {"q": "What is the modulation of the emitter at 2.452 GHz?",
     "a": "Wi-Fi 802.11g OFDM, 20 MHz channel, 54 Mbps.",
     "qa_type": "classification"},
    {"q": "What is the SNR of the BLE packet?", "a": "14.7 dB.",
     "qa_type": "value_extraction"},
    {"q": "What is the radar's pulse repetition interval?", "a": "1 ms.",
     "qa_type": "value_extraction"},
    {"q": "Which emitter has the highest SNR?",
     "a": "The Wi-Fi 11g transmission at 22.1 dB.",
     "qa_type": "comparison"},
    {"q": "Describe the radar's temporal pattern.",
     "a": "100 Barker-13 pulses with 13 µs pulse width at 1 ms PRI, spanning the full 100 ms scene.",
     "qa_type": "freeform"},
    {"q": "What channel profile applies?", "a": "TDL-C with 300 ns delay spread.",
     "qa_type": "classification"}
  ]
}

3. Reasoning chain: annotation_type="reasoning"

Purpose. Chain-of-thought walkthrough explaining why each emitter has its label, anchored on physical attributes that justify the classification.

Output schema:

class ReasoningStep(BaseModel):
    emitter_idx: int
    class_name: str
    reasoning: str = Field(min_length=80, max_length=400)

class ReasoningOutput(BaseModel):
    steps: list[ReasoningStep]

Token budget. ~200 in + ~400 out ≈ 600 tokens. max_output_tokens=600.

Task instruction:

For each emitter, produce a 3–5 sentence justification of the class label using only physical attributes. State the bandwidth, the modulation family inferred from taxonomy_path, and any distinguishing features (PRI, hop count, advertising channel index, SF/BW for LoRa). Do not speculate beyond the FACTS.

Example output (one step):

{
  "steps": [
    {
      "emitter_idx": 0,
      "class_name": "comms.wifi.11g.ofdm-20mhz.r-54mbps",
      "reasoning": "The emitter occupies 20 MHz centered at 2.452 GHz, consistent with an 802.11g channel placement in the 2.4 GHz ISM band. Its bandwidth and the OFDM family in the taxonomy path indicate a 64-subcarrier 802.11a/g waveform. The 54 Mbps data rate identifies the 64-QAM rate-3/4 mode. The 1.2 ms duration and 1024-byte PSDU are consistent with a single MAC frame at 54 Mbps."
    }
  ]
}

4. Scene report: annotation_type="scene_report"

Purpose. Structured machine-readable summary. The single artifact RF agents consume directly.

Output schema:

class SceneReport(BaseModel):
    counts_by_top_level: dict[str, int]      # {"comms": 1, "iot": 1, "radar": 1}
    dominant_emitter: str                    # class_name of highest-SNR
    dominant_snr_db: float
    occupancy_fraction: float                # ∈ [0, 1]
    model_flags: dict[str, object] = {}      # optional boolean or nested audit flags
    notable_events: list[str]                # bounded vocabulary
    summary: str = Field(max_length=200)     # single sentence

Token budget. ~150 in + ~150 out ≈ 300 tokens. max_output_tokens=200.

notable_events closed enum (rfgen/annotators/scene_report_events.py):

  • "multi-family coexistence: comms + iot + radar" and other family-set permutations

  • "phase-coded pulsed radar across full duration"

  • "high-SNR Wi-Fi burst above 20 dB"

  • "all emitters below 5 dB SNR"

  • "frequency-overlapping cochannel pair"

  • "time-overlapping bursts"

  • "single dominant emitter"

  • "sparse: density below 0.5 emitters/MHz"

  • "dense: density above 5 emitters/MHz"

  • "multi-RX scene" (when num_rx > 1)

Example output:

{
  "counts_by_top_level": {"comms": 1, "iot": 1, "radar": 1},
  "dominant_emitter": "comms.wifi.11g.ofdm-20mhz.r-54mbps",
  "dominant_snr_db": 22.1,
  "occupancy_fraction": 0.042,
  "model_flags": {"line_of_sight": true},
  "notable_events": [
    "multi-family coexistence: comms + iot + radar",
    "phase-coded pulsed radar across full duration",
    "high-SNR Wi-Fi burst above 20 dB"
  ],
  "summary": "Three coexisting emitters across the 2.4 GHz ISM band: Wi-Fi 11g, BLE advertising, and a Barker-13 phase-coded pulsed radar."
}

5. Contrastive pair: annotation_type="contrastive"

Purpose. Match caption + hard-negative caption (same caption with exactly one physical attribute swapped to a plausible-but-incorrect value). For CLIP-for-RF contrastive training.

Output schema:

class ContrastivePair(BaseModel):
    match: str = Field(min_length=20, max_length=600)
    hard_negative: str = Field(min_length=20, max_length=600)
    swapped_attribute: str                   # e.g. "radar.code: barker_13 -> frank_64"
    swapped_emitter_idx: int

Token budget. ~150 in + ~250 out ≈ 400 tokens. max_output_tokens=400.

Task instruction:

Produce one match caption and one hard-negative caption. The match is the standard caption per template caption.dense.v3. The hard negative is the same caption with exactly one physical attribute changed to a plausible but incorrect value drawn from the closed vocabulary (a different modulation in the same family, a different channel profile, an SNR ±10 dB different, a different radar code, a different LoRa SF). State which attribute was swapped using the canonical key format.

Allowed swap categories:

Category

Example swap

Modulation (same family)

comms.psk.qpsk → comms.psk.8psk

Channel profile

tdl-c-300 → tdl-a-30

SNR (±10 dB)

22.1 dB → 12.1 dB

Radar intrapulse mod

barker_13 → frank_64

LoRa parameter

sf7_bw125 → sf12_bw125

Wi-Fi data rate

54 Mbps → 12 Mbps

Cellular bandwidth

lte_10mhz → lte_20mhz

Example output:

{
  "match": "A 100-MHz wideband recording at 2.45 GHz contains three coexisting emitters ... A Barker-13 phase-coded pulsed radar at 2.485 GHz spans the full duration with 100 pulses at 1 ms PRI ...",
  "hard_negative": "A 100-MHz wideband recording at 2.45 GHz contains three coexisting emitters ... A Frank-64 polyphase pulsed radar at 2.485 GHz spans the full duration with 100 pulses at 1 ms PRI ...",
  "swapped_attribute": "radar.intrapulse_modulation: barker_13 -> frank_64",
  "swapped_emitter_idx": 2
}

Hallucination control: six concrete techniques

#

Technique

When

1

Whitelist-only injection

Prompt-time. Only whitelist_filter(record, sample_id=...) output enters prompts.

2

Closed-vocabulary instruction

Prompt-time. System prompt + ALLOWED VOCABULARY block.

3

JSON-constrained decoding

Generation-time. Provider-side structured output.

4

Token budget per type

Generation-time. Caps max_output_tokens per annotation type.

5

Second-pass inference consistency check (PAES)

Validation-time. See Reference / Metrics.

6

Per-template hallucination audit

Release-time. Per-template PAES gates dataset promotion.

JSON-constrained decoding per provider

Provider

Mechanism

Gemini gemini-3.1-flash-lite

response_mime_type="application/json" + response_json_schema=<JSON Schema> (google-genai JSON Schema field)

OpenAI GPT-4o / 4o-mini

response_format={"type": "json_schema", "json_schema": {...}}

Anthropic Claude

tools=[{"name": "emit", "input_schema": {...}}] + tool-choice forcing

vLLM-served local

Outlines / lm-format-enforcer / xgrammar grammars

The validator uses JSON Schema plus Pydantic model validation. A response that doesn’t parse triggers one retry with temperature × 0.5; a second failure flags the record for the verification audit path.

Vertex AI Batch Prediction integration

Input JSONL

One line per (sample_id, annotation_type):

{
  "request": {
    "contents": [
      {"role": "user", "parts": [{"text": "<full prompt body>"}]}
    ],
    "system_instruction": {"parts": [{"text": "<system prompt>"}]},
    "generation_config": {
      "temperature": 0.7,
      "max_output_tokens": 200,
      "response_mime_type": "application/json",
      "response_json_schema": {"type": "object", "properties": {...}}
    }
  },
  "metadata": {
    "sample_id": "f9a3b7c2-...",
    "annotation_type": "caption",
    "template_id": "caption.dense.v3"
  }
}

The metadata block is opaque to Vertex; it round-trips with the response so the coordinator can join results back without parsing the prompt.

Output JSONL

{
  "request": {...},
  "response": {
    "candidates": [
      {"content": {"parts": [{"text": "<JSON-formatted response>"}]},
       "finish_reason": "STOP"}
    ],
    "usage_metadata": {"prompt_token_count": 154, "candidates_token_count": 118}
  },
  "metadata": {"sample_id": "...", "annotation_type": "caption", "template_id": "..."}
}

Coordinator flow

  1. Build per-(annotation_type, sample_id) prompts. Write JSONL to gs://<bucket>/phase2/jobs/<run_id>/<annotation_type>.jsonl.

  2. Submit BatchPredictionJob(model="gemini-3.1-flash-lite", input=..., output=...).

  3. Poll until JOB_STATE_SUCCEEDED.

  4. Read output JSONL; validate each response against the Pydantic schema; fold validation failures into a retry JSONL for one re-submission; append validated responses through StoreHandle.append_annotation_row keyed by sample_id.

  5. For records selected for verification, repeat with model="gemini-3.1-flash-lite" and the extractor / re-generation prompts.

Idempotency

The coordinator skips records whose target Zarr column already exists. Re-running a job is safe; only missing annotations are filled.

Bulk + verifier strategy

Tier

Default model

Coverage

Purpose

Bulk

Gemini gemini-3.1-flash-lite

100 %

High-volume, low-cost annotation

Verifier

Gemini gemini-3.1-flash-lite (or Claude Sonnet 4.6)

0 %, 100 %, or preselected partial subset

PAES + hallucination measurement

The provider alias gemini and explicit gemini-3.1-flash-lite model selection follow the shipped compute_paes registry contract.

AnnotatorConfig.verifier_subset_pct accepts 0.0 to disable verification or 100.0 to verify every record handed to annotate(...). Partial verification is selected out of band with select_paes_verifier_subset(...), stratified by taxonomy_path[0] and signal_density quartile, then passed to the annotator as preselected membership. Subset sample IDs are recorded in manifest.annotation_audit.verifier_subset_ids.

See Also