Add a custom annotation template

Warning

Pre-implementation. APIs describe the target plugin surface.

Goal

Add a YAML annotation template that produces a new text-annotation variant grounded in existing metadata and labels. An annotation template is Phase 2 config consumed by the Templater; it turns Phase 1 record metadata and labels into a named text run stored under the dataset’s append-only text annotation area.

When to use this

Add an annotation template when the target is natural language derived from already-labeled records. The closed AnnotationType values are CAPTION, QA, REASONING, SCENE_REPORT, and CONTRASTIVE; contrastive annotations pair a factual caption with one hard-negative caption for retrieval or similarity training. If the target is structured (bbox, mask, per-emitter row), see Add a custom labeler instead.

Prerequisites

Read Concepts / Annotations and Reference / Annotation Templates. Templates specialize an existing AnnotationType value; adding a new top-level annotation kind requires a framework enum change, not just a YAML file.

Minimal command path

Author the template as a named task string for the shipped Templater. The current AnnotatorConfig schema does not expose template registration keys; runtime configuration selects enabled annotation types and LLM endpoints.

from rfgen.annotators import MetadataAnnotator, Templater
from rfgen.enums import AnnotationType

templater = Templater(
    templates={
        "caption.rf_scene.v2": (
            "Produce one caption from the whitelisted FACTS. Mention class, "
            "carrier, bandwidth, and SNR only when those facts are present."
        )
    }
)
annotator = MetadataAnnotator.from_components(
    client=client,
    templater=templater,
    allowed_annotation_types=[AnnotationType.CAPTION],
)

Run only the affected annotation type:

rfgen annotate ./out/local-smoke \
    annotator.types=[caption] \
    annotator.bulk_llm.provider=openai \
    annotator.bulk_llm.model=gpt-4o-mini

annotator.types=[caption] is Hydra list override syntax; it limits this run to the caption annotation type. The shipped config surface does not choose template ids; custom templates are passed through the Python construction path above.

Verify

rfgen inspect ./out/local-smoke sample --first 3
rfgen inspect ./out/local-smoke distribution --field text

Confirm:

  • The output JSON validates against output_schema.

  • Closed-vocabulary fields (e.g., modulation names) match the canonical set.

  • The verifier subset reports Physical Attribute Extraction Score (PAES) within the accepted metric range for the chosen annotation type.

  • Re-annotation appends a new (template_id, run_id) row through StoreHandle.append_annotation_row rather than overwriting prior text; run_id identifies one Phase 2 annotation execution.

Troubleshoot

Symptom

Fix

Annotation contains unsupported facts

Update the task string to mention only whitelisted FACTS, or tighten the closed vocabulary.

Re-annotation overwrites old text

Bump template_id or run_id; Phase 2 should be append-only.

PAES below threshold

Check whether the task asks for facts absent from the prompt, or whether the output schema is over-constrained.

Next steps

See Also