GCS loader qualification

Retained M1 result

The corrected M1 run m1-gcs-20260828-03 passed the registered gcs_remote@1 thresholds. Its complete artifact, exact-generation artifact receipt, and successful post-exit terminal receipt are retained as .agent-state/deliveries/m1-gcs-sds021-loader-{qualification,artifact-receipt,terminal-receipt}.json. The artifact truthfully retains generation source head 2a9f64bb2b6949e7092b17f1fff791cdefe2ebcb. The checked-in registry entry remains an unbound production template; the artifact’s selected, environment-bound profile and its terminal receipt are the measured authority.

This gate creates an approximately 8.4 GB, 4,096-record radar-shaped SDS fixture plus about 430 MB of selected annotations. GCS storage, operations, container CPU/memory, and reads are cost-bearing. It does not delete the fixture; a failed or partial run is abandoned and retried with a new run ID.

Build the dedicated image from an exact clean source head, then resolve the registry tag to its immutable digest:

HEAD=$(git rev-parse HEAD)
IMAGE=REGION-docker.pkg.dev/PROJECT/REPOSITORY/loader-qualification
python ci/testing/build_loader_qualification_image.py \
  --approved-source-head "$HEAD" \
  --image-uri "$IMAGE" \
  --image-tag "$HEAD" \
  --context-output /tmp/rfgen-loader-build-$HEAD
# Inspect the JSON argv, then repeat with --submit.
DIGEST=$(gcloud artifacts docker images describe "$IMAGE:$HEAD" \
  --format='value(image_summary.digest)')

The dry plan leaves its sanitized context at the explicit path so the printed argv remains reusable; remove that context after inspection. --submit creates and removes its own fresh context instead of trusting the planned copy.

The builder rejects a dirty checkout or a head mismatch, clones the approved commit into a temporary credential-free context, and includes only that clean checkout plus minimal Git object/index/HEAD authority. The Docker build derives and verifies the revision from Git before removing .git; a caller-provided label cannot make a different tree pass. The destination must be an untagged regional Artifact Registry path, and its Docker tag must exactly equal the approved 40-character source head; unsafe or ambiguous reference syntax is rejected before the build context is prepared.

This dedicated storage image installs pinned torch==2.13.0 from PyTorch’s official CPU wheel index before resolving the complete rfgen[gcs] dependency closure under the same constraint. The build fails if CUDA is enabled, if an NVIDIA/CUDA/Triton distribution is installed, if SDS is not exactly 0.2.1, or if Torch’s base version differs (only the authentic +cpu local suffix is accepted), or if the non-root qualifier smoke test fails. Cloud Build records the local image size and rejects anything above 3 GiB. That ceiling leaves room for the CPU Torch wheel and rfgen/SDS dependencies while preventing an accidental multi-gigabyte GPU runtime from being published.

The deploy helper requires the operator account, project, region, job, runtime service account, digest URI, bucket, owned dataset prefix, unique run ID, and a separate evidence object URI. Without --apply it only prints the deploy command. With --apply it creates or updates the single-container Cloud Run Job; it does not execute it.

The generated replacement document follows Cloud Run’s v1 Job YAML hierarchy: Job metadata, execution-template metadata/spec, then a task template containing only spec. The runtime reads the provider’s v2 Job and Execution resources, whose task-template locations differ, and attests both independently.

ci/testing/deploy_gcs_sds021_loader_job.sh \
  --operator-account OPERATOR --project PROJECT --region REGION --job JOB \
  --runtime-service-account RUNTIME_SERVICE_ACCOUNT \
  --image-uri IMAGE@sha256:DIGEST --source-head HEAD --bucket BUCKET \
  --prefix DATASET_PREFIX --run-id RUN_ID \
  --evidence-uri gs://BUCKET/qualification-evidence/RUN_ID.json

First pass --qualification-dry-run --apply to the helper and execute the read-only preflight. After inspecting its bounded plan, redeploy with --apply but without --qualification-dry-run, then explicitly execute:

EXECUTION=$(gcloud run jobs execute JOB --project PROJECT --region REGION --wait \
  --format='value(metadata.name)')
gcloud run jobs describe JOB --project PROJECT --region REGION --format=json \
  > job-revision.json
gcloud run jobs executions describe "$EXECUTION" --project PROJECT --region REGION \
  --format=json > execution-revision.json

The registered resource envelope is one task, parallelism one, zero retries, 2 CPU, 4 GiB memory, a 7,200-second timeout, second-generation execution, and native GCS access without mounted volumes. The 4 GiB limit is deliberately above the profile’s 2 GiB measured-memory ceiling. The long timeout and 2 CPU raise possible Cloud Run compute cost; the Job and current Execution must attest these exact settings before the fixture is created.

Command authority and threat boundary

Arguments visible inside the container are claims, not authorization. The operator-reviewed structured manifest is deployed through Cloud Run, and the runner independently reads the provider-returned Job and current Execution. Both resources must contain one container with the exact canonical python command, ordered argv, labels and annotations; they must also equal each other. The annotations retain profile identity, exact dataset run URI, and evidence URI, while labels retain reviewed source head, run ID, and qualification mode. Missing/default commands, duplicate or reordered flags, an unexpected dry-run mode, or a one-sided Job/Execution edit fails before dataset creation.

QualificationCommandAuthority is retained inside the provider-attested runtime authority in the owner marker and is cross-checked against the final artifact request and receipt. This does not use a container-provided argument as its own proof, and it does not invent a manifest digest: the provider’s Job and Execution resources are the deployed authority. Operators should retain the reviewed manifest and Cloud Run resource revision/etag alongside the generation-qualified artifact receipt when recording the external gate.

At startup the runner reads both resources twice and requires stable native name, UUID, etag, generation, and observed generation. Normal Cloud Run startup convergence is polled under the versioned bounded policy (eight observations, 0.25-second exponential backoff capped at two seconds), but two identical active-phase reads are still mandatory. In that provider phase the Job is non-reconciling and current, while the Execution remains reconciling:true with successful Started, reconciling Completed, one running task, no terminal counters/time, successful ContainerReady and ResourcesAvailable, and optionally the qualified Retry. Each core type appears exactly once; terminal collection requires the same supporting successes and successful Completed. Endless churn or a stale latest Execution times out; configuration/resource/semantic drift, a wrong Job relationship, or a terminal/failure state fails immediately. Immediately before sealing the dataset inventory, a second stable checkpoint requires the unchanged Job deployment and the same active Execution identity, generation, conditions, command, and resources. Only provider status etag and update-time evolution is allowed.

Cloud Run v2 status-only completion does not necessarily advance resource updateTime. Terminal validation therefore keeps createTime <= updateTime as resource chronology, while independently requiring start, completion, successful Completed transition, and collector observation in that order.

The benchmark recollects its runtime environment at measurement time. The remote service supplies the same immutable image digest obtained from the provider-attested Job/Execution authority to that collection, then validates the recollected identity against the bound profile. A missing digest fails; caller environment variables are not trusted as a replacement.

Cloud Run v2 emits the relationship fields as short names even though each resource’s primary name is fully qualified. The qualifier binds those short names to the exact expected full Job and Execution segments; path-like or cross-resource names fail. Proto-default omission/null is accepted only for a false reconciling field and omitted nonnegative counters, without truthy or integer coercion; present counters must be within the Cloud Run int32 range. All provider create/update/start/completion/transition timestamps use one strict timezone-qualified RFC3339 decoder. Only an omitted/null optional completion time means “not completed”; empty strings and other falsey values fail. The authority preserves and compares all nine provider fractional-second digits as integer epoch nanoseconds, including offset-equivalent and pre-epoch instants.

Cloud Run’s v2 CONDITION_FAILED state is terminal regardless of condition type. Startup therefore rejects failed Started, ResourcesAvailable, or any other condition immediately, as well as error severity and fatal values in the official condition reason union; pending/reconciling polling reasons remain eligible only for the bounded convergence loop. Unknown state/severity/reason enums, enum values placed in the wrong reason field, malformed values, and multiple reason-union members fail closed. The official common reason UNKNOWN is a failure, not a retry signal. Unknown Condition document fields and malformed, naive, or non-string transition timestamps also fail closed. Live Cloud Run Jobs evidence includes a synthetic successful retry-scheduling condition: Retry / CONDITION_SUCCEEDED / WAITING_FOR_OPERATION / INFO. The qualifier admits only that exact exception and keeps it nonterminal; changed type, reason field/value, or severity is rejected.

Cloud Run cannot report an Execution complete while that same container is still producing the artifact. The artifact therefore carries an explicitly in-process final checkpoint, not a terminal-success claim. After the Job exits, collect the separate terminal authority against the exact artifact generation:

python ci/testing/collect_gcs_sds021_loader_terminal.py \
  --artifact-receipt artifact-receipt.json \
  --output terminal-receipt.json

The collector re-reads the exact Job and Execution twice, requires the Job revision unchanged, and accepts only one completed task with zero running, failed, or cancelled tasks. Exactly one provider Completed condition must be CONDITION_SUCCEEDED, with status chronology ordered from start through completion and its transition to collector observation. Resource updateTime is not a status clock. Exactly one successful Started, ContainerReady, and ResourcesAvailable must also be present; only the qualified optional Retry may accompany those four core rows. The successful condition accepts only absent, unspecified, or informational severity. Its Cloud Run reason union (reason, revisionReason, executionReason, and instanceReason) must be empty, so a nominal success cannot carry a non-zero-exit, cancellation, deletion, or container failure reason. Cloud Run v2 Condition has no observed-generation field. Generation authority comes from the Execution resource itself: its canonical positive REST int64 generation and observedGeneration strings must decode equally and remain stable across reads. Its create-only local QualificationTerminalReceipt@1 contains the complete terminal revisions and nested artifact receipt. The deploy helper prints all follow-up operations as JSON argv arrays; the exact Execution name comes from the provider’s execute response and is also bound in the artifact receipt.

The container freezes complete immutable Job and Execution authorities: native identity/generation/create/start fields, image, runtime service account, command/argv, labels/annotations, resources, environment, volumes, and the canonical complete provider template. Final and terminal checks require exact field-for-field equality. Only the separately typed provider state may evolve: etag, update time, reconciliation/status counters, completion time, and conditions. The transported terminal authority independently validates its configured task count against zero running/failed/cancelled tasks and the exact success count. The terminal receipt also binds the Execution name, UID, and terminal etag to the nested artifact receipt; collector-time checks alone are insufficient. Each typed command, argv, image, service account, label, annotation, identity, generation, and provider time is reconstructed from and compared with its canonical immutable provider document. Job and Execution task templates must match, and both native taskCount values must equal the terminal configured task count. Retries, timeout, execution environment, volumes, CPU, memory, and parallelism also match the retained closed resource envelope. Required counts reject null and Python-compatible type tricks such as true == 1; string-valued resource fields require exact strings. Captured provider documents omit volumes, and the native-GCS contract therefore accepts omission only—not explicit null, empty collections, or other falsy values.

The container verifies its Job/execution image and runtime service account against Cloud Run control-plane state. It seals the dataset run-prefix inventory, publishes the validated artifact create-only to the disjoint evidence URI, and prints a receipt containing that URI and native GCS generation. Retrieve only that generation:

gcloud storage cat 'gs://BUCKET/qualification-evidence/RUN_ID.json#GENERATION' > qualification.json

The program emits exactly one compact structured rfgen.loader_qualification.artifact_receipt_event@1 stdout event. Do not copy it manually. Use the deploy plan’s generated receipt-collector argv with the exact Execution name and UUID returned by Cloud Run, a maximum 24-hour RFC3339 window, reviewed run/source/evidence values, and a new output path:

python ci/testing/collect_gcs_sds021_loader_artifact_receipt.py \
  --project PROJECT --location REGION --job JOB \
  --execution EXACT_EXECUTION_NAME --execution-uid EXACT_EXECUTION_UID \
  --run-id RUN_ID --source-head HEAD --evidence-uri gs://BUCKET/EVIDENCE.json \
  --start-time START_RFC3339 --end-time END_RFC3339 \
  --output artifact-receipt.json

The collector uses Cloud Logging’s cloud_run_job monitored resource, exact project/location/job labels, and the run.googleapis.com/execution_name log label. It accepts exactly one matching event (query limit two), binds the Execution UUID, run, source, and URI, then reads and revalidates the artifact at its native GCS generation. The local receipt is create-only. Zero, duplicate, malformed, stale-generation, or cross-Execution entries fail. Any Logging continuation token also fails closed: the exact filtered cardinality cannot be proven from a partial page. Every query value uses the centralized Cloud Logging string-literal encoder; GCS prefix and evidence segments use the closed alphanumeric/dot/underscore/dash operational character set.

The artifact contains the sealed dataset inventory. Its own generation cannot be embedded before creation; QualificationArtifactReceipt@1 is the authority pairing. A collision is terminal and requires a new run ID and evidence URI.