Skip to content

v0.8.2.post3 — Image distribution & reproducible push

Sandbox-required datasets (pr_runtime, mutation_bugs, code_instruct, cve_patches, equivalence_tests, refactor_synthesis, commit_runtime) are now reproducible end-to-end by any consumer who pulls them from HF Hub.

What changed

Before this release, every _runtime dataset emitted an environment/Dockerfile with FROM local/r2e-bootstrap/... — un-pullable from any other machine. The Hub dataset looked correct on paper but failed at harbor run time on a fresh machine. This release closes that loop.

repo2rlenv push now:

  1. Discovers registry credentials in ~/.docker/config.json (GHCR / ECR Private+Public / ACR / GCP AR / Docker Hub / local / others).
  2. Verifies them via the OCI Distribution Spec L1-L4 probe — reachability, auth resolution, read access, write access — without pushing anything to your registry.
  3. Pushes the bootstrap image to the first verified registry, rewriting each task's environment/Dockerfile + task.toml in-place to point at the registry-qualified digest (@sha256:...).
  4. Falls back to inline-Dockerfile mode when no verified registry is available — bakes the bootstrap's reconstructed Dockerfile into each task and records fallback_reason so consumers know what they're getting. Pass --require-registry to hard-fail instead.

Plus: GHCR package visibility is auto-flipped to public via the GitHub API when the dataset is public; ECR / GCP AR repositories are auto-created on first push.

New CLI surface

# Default: auto-detect + verify + push (or fall back to inline with warning)
repo2rlenv push ./datasets/mydataset owner/mydataset

# Force a specific registry
repo2rlenv push ./datasets/mydataset owner/mydataset \
  --image-registry ghcr.io/myorg

# Inline-recipe mode — no registry needed
repo2rlenv push ./datasets/mydataset owner/mydataset --inline-dockerfile

# CI / launch: refuse to fall back silently
repo2rlenv push ./datasets/mydataset owner/mydataset --require-registry

# Diagnose credentials without doing anything
repo2rlenv push --check-auth
repo2rlenv push --check-auth --json     # for CI assertions
repo2rlenv push --check-auth --fast     # stop at L2, skip wire-level probes

See docs/reference/REGISTRY_AUTH.md for the per-registry login instructions.

Spec changes

[metadata.repo2env.spec_version] is bumped from "0.1.0" to "0.2.0". The bump is additive — v0.1 readers ignore the new subtable.

New: [metadata.repo2env.reproducibility] on every sandbox-required task:

[metadata.repo2env.reproducibility]
mode = "registry"                                    # registry | inline_dockerfile | local_only
image_ref = "ghcr.io/huggingface/r2e-bootstrap-pallets-click@sha256:..."
image_tag = "ghcr.io/huggingface/r2e-bootstrap-pallets-click:a1b2c3d4e5f6-7d8e9f01"
image_visibility = "public"
pushed_at = "2026-05-19T11:30:00Z"
pushed_by = "huggingface"

For inline-mode datasets the same subtable carries inline_recipe_sha256, inline_recipe_lines, inline_recipe_source (user_dockerfile for bit-exact recipes, agent_replay for best-effort), and fallback_reason when inline was an auto-fallback rather than an explicit choice.

The legacy [metadata.repo2env.<pipeline>].bootstrap_image field is still written for back-compat but the canonical location is now [metadata.repo2env.reproducibility].image_ref.

Bug fixes

  • cmd_push no longer omits pipeline / repo_source from the dataset card (C1). Previously, calling repo2rlenv push standalone (not via --out hf://) produced a README with empty pipeline + source-repo fields. Now read from the first task's [metadata.repo2env] automatically.

Implementation

New src/repo2rlenv/registry/ subpackage with focused modules:

  • auth.py — file-level discovery from ~/.docker/config.json
  • probe.py — OCI L1-L4 verification (stdlib urllib, no network deps)
  • push.py — docker push subprocess wrapper + digest extraction
  • visibility.py — GHCR PATCH visibility flip via gh CLI
  • ecr.py, gar.py — repo pre-creation for registries that need it
  • naming.py — slug + tag construction reusing cache._options_hash
  • integration.py — the discover → verify → select → push → rewrite orchestrator called from hub.push_to_hub

127 new unit tests, plus optional live-registry integration tests gated on R2E_LIVE_PROBE_LOCAL / R2E_LIVE_PROBE_GHCR env vars. Total: 614/614 tests green.

Migration

Datasets pushed with v0.8.2.post2 or earlier have un-pullable FROM references and no [metadata.repo2env.reproducibility] subtable. They won't break the CLI (validate still passes), but harbor run will fail on a fresh machine. Re-push to upgrade:

repo2rlenv pull <old-dataset>
repo2rlenv push ./datasets/<old-dataset> <hf-target>

The pulled dataset re-emerges with mode = "local_only" plus a still-local FROM ref; re-pushing rewrites everything to registry / inline mode.

What's NOT in this release

Deferred to future work (no ETA): multi-arch images, image squashing / size reduction, cosign signing, HF Hub as a native OCI registry, re-pushing old datasets in bulk via a --rewrite-only flag.

Full design rationale: plans/v0.8.2.post3_image_distribution.md.