rfgen.annotation

rfgen.annotation turns a generated record’s declared facts into an annotation and publishes it as an append-only set, without modifying the record.

An RF domain is annotated by writing one class and one Markdown prompt against rfgen.annotation.contract. Everything else is mechanics the domain gets for free and must not rebuild: resolution, request assembly, both backends, batching, retries, provenance, and immutable publication.

Two independent channels

A published row carries two things that are produced separately and never joined.

The prose is the provider’s, and it is unverified. Nothing in this package checks whether a caption is true of the capture, because nothing mechanically can: a language model may restate a value in a different unit, name a different quantity, invert a relation, or invent a property, and no check over the text sees any of that. A row’s provenance says so explicitly and always, in prose_verification.

The claims are built here, from evidence this process already holds, by Annotator.build_structured_output, and they are computed before the provider is called. No value on that side of a row was ever authored by a model.

A caption may therefore contradict the claims in its own row. That is accepted and recorded rather than gated. What the split buys is that the machine-readable half is exact for every row however badly the prose reads, so a defective caption cannot contaminate it.

Verification of prose, where it is wanted, is ad hoc analysis with a stronger model, outside this pipeline.

The contract

rfgen.annotation.contract is the whole public surface an author needs.

Annotator is the ABC. A subclass declares identity, template_id and prompt, then implements build_evidence, build_prompt and build_structured_output. AnnotatorContext is everything an annotator is told: its product, its options, and the projection the run declared. PreparedAnnotation is one record’s evidence, request and claims, built once by the final prepare. RecordNotAnnotatable distinguishes a record with nothing truthful to say from a failure.

AnnotationPrompt is one rendered request: system text, user payload, response schema and token budget. PromptResource addresses a Markdown prompt as a package resource, so it loads from an installed wheel and not only from a source tree; it carries a declared prompt_id and revision that must agree with the front matter of the file, and PromptResourceError is raised when they do not.

TEMPLATE_ID_PATTERN, require_template_id, require_annotation_type and annotation_type_of define how a product is named — <annotation_type>.<variant>.v<N> — and what core is entitled to check about it. Core holds no closed list of annotation types: a sensing domain producing an occupancy_report names its own output.

Selecting an annotator

A run names its annotator by import path, package.module:ClassName. There is no registry, no entry point, and no reinstall before a new annotator resolves.

resolve_annotator imports and role-checks the named class, returning a ResolvedAnnotator that records the identity, the product, the origin and the installed distribution, all of which reach a published row’s provenance. ANNOTATOR_SPEC_PATTERN is the accepted shape.

Role checking is nominal rather than structural: a runtime_checkable protocol would admit any object that happens to carry the right method names. Each way of naming an annotator wrongly raises its own error, all under AnnotatorResolutionError: AnnotatorSpecError for a malformed path, AnnotatorImportError when the module cannot be imported, AnnotatorNotFoundError when the module defines no such name, and AnnotatorRoleError when the object is not a concrete Annotator.

Configuring and running

rfgen.annotation.run owns one annotation run.

AnnotationRunConfig names a dataset, a template_id, an annotator path, a model, an inference block, an optional annotator_options block opaque to core, and the projection. coerce_declaration_policy parses a configured policy name into its member.

preflight resolves and constructs the annotator before a dataset is opened or a client is built, returning a PreflightResult; a packaging mistake, a bad option or an unreadable prompt is a startup error rather than a failure discovered after a provider has been billed.

prepare_records then builds every record’s evidence, request and claims exactly once, returning one PreparationOutcome per ordinal. A record that cannot be prepared becomes its own outcome rather than an exception out of the run, so the published set stays dense and the rest of the corpus is still annotated. A record whose annotator raises RecordNotAnnotatable is published SKIPPED rather than FAILED — a quiet epoch is a fact about the corpus, not an outage — and is never sent to the provider.

run_annotation runs the whole sequence and publishes.

decode_reply decodes one provider reply and checks it against the schema the annotator itself declared. Core consults no table of output models keyed by product, so the only authority on a reply’s shape is the annotator that asked for it.

Backends

AnnotationBackend is the ABC both compute legs implement. LocalConcurrentBackend calls a provider directly from a bounded thread pool; RemoteBatchBackend stages the run as one batch and pairs replies back by request id, refusing a batch whose replies do not cover its requests exactly. build_backend constructs whichever the configuration asks for.

Neither backend builds a request. assemble_request is the only function that turns a PreparedAnnotation into provider-shaped input, returning an AssembledRequest, and both legs call it — so “the two backends send the same thing” is structural rather than asserted. A backend returns one ReplyOutcome per record and builds no rows.

Provenance

AnnotationProvenanceV1 is the complete provenance one row carries, assembled by annotation_provenance and nowhere else; an annotator that wrote its own could misreport its own identity. It nests AnnotatorProvenanceV1 (which implementation, and which installed copy), PromptProvenanceV1 (the declared prompt id and revision, never a digest of the body), ProjectionProvenanceV1 (what the run was willing to send) and InferenceProvenanceV1 (backend, model and usage). failure_provenance is the deliberately thin record a failed row carries, rather than filling absent fields with nulls that would read as measurements.

Publication

annotation_record builds one source-bound row without reading waveform fields. annotation_records validates externally produced outcomes into one dense snapshot-aligned sequence. publish_annotation_set publishes them atomically under a publication id.

The record projection

rfgen.annotation.projection decides what a corpus is willing to say to a model core does not run. DeclarationPolicy selects between the historical closed core vocabulary, an author-declared allow-list, and the local-inspection policy that the transmit path refuses; permits_transmission and require_transmittable enforce that, and WITHHELD_MEMBERS is a floor beneath every policy.

rfgen.annotation.record_view reads a record under a policy. metadata_view returns SdsMetadataView/SdsEmitterView for an unframed record; a framed record returns SdsFramedMetadataView with its checked catalog and SdsFramedEmitterView entries keyed by (emitter_group_id, emitter_index). SdsSceneView and SdsBoxView carry scene scalars and detection boxes. emitter_snr_by_entity_receiver exposes the composite SNR map, while the bare emitter_snr_by_receiver refuses a framed record rather than inferring group zero. entity_view_declarations and emitter_source_declarations project declarations under a policy; EMITTER_GROUP is the default entity collection, TYPED_EMITTER_MEMBERS the typed members a view carries, and CARRIER_DECLARED and CARRIER_DERIVED distinguish a declared absolute carrier from a derived one.

The reference annotator

rfgen.annotation.reference is the shipped example, and it is labelled reference deliberately: core shipping a communications annotator would be core making a domain decision. Everything in that package is communications vocabulary, and a second domain replaces all of it by pointing a configuration at its own class.

SceneReportAnnotator produces one technical characterization of a captured scene under SCENE_REPORT_PROMPT. Its epistemic voice is inferential — declared facts are phrased as observations, because the deployed reader of these captions sees only a waveform — and that is a domain choice carried in its own prompt, not a shared rule.

DeclaredEvidenceConfigV1 is its options block. build_declared_scene_evidence derives DeclaredSceneEvidenceV1 from a projection, raising DeclaredEvidenceOverflowError when a scene exceeds the configured component cap; supported_component_quantity_fields and supported_component_label_fields name what it can read.

The evidence models are SceneEvidenceNamespacesV1, ComponentEvidenceV1, InteractionEvidenceV1, NumericClaimV1, LabelClaimV1, QuantityV1, LabelValueV1, BookkeepingV1 and OverflowMarkerV1, bounded by MAX_DECLARED_COMPONENTS_ABSOLUTE. canonical_declared_evidence_json serializes evidence deterministically; transmittable_evidence_json is what is actually sent, dropping the GENERATOR_SIDE_BOOKKEEPING keys that name this generator’s own types and functions, with strip_generator_bookkeeping doing the walk. DECLARED_EVIDENCE_PROMPT_PREFIX fronts the payload; declared_evidence_path_value and declared_evidence_path_equals resolve a pointer into evidence. DECLARED_EVIDENCE_TEMPLATE_ID and is_declared_evidence_template name the product.

render_declared_prompt assembles the request and narrative_response_schema is its reply contract: an array of paragraphs, and nothing else.

build_scene_claims projects the same evidence into SceneClaimsV1 — the machine-readable half — holding SignalClaimsV1, InteractionClaimsV1 and MeasuredValueV1. It deliberately drops the bookkeeping that explains how this generator knew a fact, and omits an unresolvable quantity entirely rather than publishing it as null.

Declared label safety

canonical_ground_truth_string bounds what a declared label may say when it leaves the machine, truncating at MAX_GROUND_TRUTH_STRING_LENGTH. It is a projection-safety rule rather than a formatting convenience, which is why it sits beside the evidence that uses it.

Inference

BaseInferenceClient is the provider contract every client implements, returning an InferenceResponse. RemoteInferenceExecutor is the batch-side equivalent, consuming PreparedInferenceRequest values that the driver renders locally.

BaseCredentialsProvider resolves a provider’s credentials into ResolvedCredentials at the point of use, so a credential is never carried on a configuration model. Schema projection and the built-in providers live alongside them under rfgen.annotation.inference.