Signal Dataset annotation over gs:// (2026-08-13)

Note

Read for the route, not for the configuration. This run used gemini-2.5-flash-lite, which was withdrawn on this record’s own date, and template scene_report.declared.v1, since renamed caption.declared.v1. Take the current model and template from Annotation; the Gemini 2.5 family is deprecated and must not be used.

A real bucket round trip of the Signal Dataset annotation source and sink, run to settle one claim the design rests on: that a snapshot is annotated the same way whether its root is a local path or a bucket, because the reader takes a root and never lists a directory.

The two transports are not literally the same code, which is why this was run rather than reasoned about. object_store_for branches on the scheme, generation_of branches on it again, and absence itself has two spellings: a local store raises FileNotFoundError where GCS raises google.api_core.exceptions.NotFound, which is not an OSError. A probe written against disk fails on the first shard of the first bucket-backed run.

What was run

Six records were generated to gs://rfgen-e2e-409806686028-8a95942a34de/annotation-sds-validation/dataset.sds in project rf-foundation-models, bucket region us-central1, and annotated with rfgen annotate submit at branch feat/annotation-signal-dataset, commit bb2a15ea. Provider gemini, model gemini-2.5-flash-lite, backend: local_concurrent, execution_mode: local, run_id: caption-v1. The signal_dataset block differs from a local one in exactly one character sequence:

signal_dataset:
  dataset_uri: gs://rfgen-e2e-409806686028-8a95942a34de/annotation-sds-validation/dataset.sds

Run

Provider calls

Result

First submission

6

6 succeeded, 0 failed; one dense annotation set published

Second submission, same configuration

0

the same set, adopted rather than re-paid

The second run is the load-bearing one. The count is the run’s own terminal_counts: the second submission reported succeeded: 0 with all six records under skipped_existing, and the fake-free path means a provider call would have appeared as a success or an error, not as a skip.

Two guards can produce that zero and it is worth naming which one fired. refuse_if_already_published runs first, and it did not stop the run: the set’s publication_id matched this configuration’s, which is the idempotent replay rather than a conflict. The zero therefore came from the per-window sink.staged() probe (one object read against a derived pointer path, before any request is prepared), and the final publish was the idempotent replay of identical bytes.

Boundaries

  • One account, one bucket, one region, six records, one model. This is evidence that the path works, not a throughput or a cost measurement.

  • backend: local_concurrent only. gemini_batch over a gs:// root was not run here; it was qualified separately on 2026-08-14. See signal-dataset-gemini-batch-20260814.md.

  • Do not copy the model. gemini-2.5-flash-lite was withdrawn from the direct Gemini API later on 2026-08-13 and returns 404 NOT_FOUND for every call. See Signal Dataset annotation on Dataproc Serverless.

  • execution_mode: dataproc_serverless was not exercised. Submission, staging, and observation for that mode are covered by unit tests against a faked gcloud and an object store as strict about URIs as the real one. No real Serverless batch had been run at the time of this run, because the then-pinned worker image predated the route. That gap is now closed: see Signal Dataset annotation on Dataproc Serverless.

  • The run predates part of what this branch now publishes. rfgen.row_counts and the Dataproc submission route were both added after bb2a15ea, and the tally is written into the set root’s metadata, so the set root’s bytes differ from what the branch publishes today. What this run evidences is the source reader, the sink’s stage-then-publish sequence, and the per-window resume probe over gs://. It does not evidence the current set-root metadata.

  • The artifacts were verified and then deleted, so the URIs above no longer resolve. They are recorded to name what was run, not to be re-read; the layout below is the observed listing, kept because the objects are not.

The published layout, as observed

Abridged: identifiers are shown as their templates rather than the observed digests, repeated objects are shown as one representative entry (or, for the signal shards, not at all), and no file’s contents are reproduced: including the set’s root.json, whose metadata predates the row tally (see the boundary above). caption-v1 below is the run’s run_id, which is what names an annotation set.

dataset.sds/
  root.json
  shards/...                                  (not enumerated; untouched here)
  annotations/
    catalog.json
    catalogs/<sha256>.json                    (one per revision; one shown)
    sets/
      caption-v1/
        <publication_id>/
          root.json
          objects/<xx>/<32-hex>.arrayrecord
          shards/<xx>/<sha256(logical_shard_id)>.json

The part shown matched the documented layout, including the placement of the set’s root.json inside the publication directory rather than beside it.


Addendum: local model server, 2026-08-13

The same source_kind: signal_dataset route annotated by a local model server rather than a hosted API, over a fresh local snapshot, not the gs:// one above, because what is under test here is the provider rather than the transport. It is the third of the three modes this route claims.

Ollama on http://localhost:11434/v1 serving gemma4, through inference.provider: openai_compatible, over a six-record local snapshot: 6 succeeded, 0 failed. Verbatim:

A single emitter is present, occupying the entire frequency and time domain. This emitter exhibits a strong SNR and maintains a continuous occupied interval. The signal is consistent with qpsk modulation.

Note

“Occupying the entire frequency and time domain” is a fact-projection artifact. The caption facts normalize frequency to the union of the record’s bounding boxes, so a single-emitter scene reads as full-band regardless of its real occupancy. This run evidences that the route completed, not that the sentence is true.

The configuration names no source_kind, so this run also exercises the default: omitting the field annotates the Signal Dataset snapshot.

Two settings were load-bearing and are worth copying. inference.timeout_s defaults to 60 seconds, which a 9.6 GB local model exceeds on its first call while the weights load. Every row failed with provider_error until it was raised to 600. concurrency: 1 keeps a single-GPU workstation from serving six requests at once. Neither is a defect; both are what a local server needs and neither is implied by the defaults, which are sized for a hosted API.