Reward Schema Reference

Every task emitted by Repo2RLEnv writes its reward to /logs/verifier/ inside the container after the agent's patch is applied and the verifier runs. Harbor reads reward.txt as the primary training signal. Native pipelines with a graded verifier also write reward-details.json, the full breakdown for analysis, filtering, and debugging.

An ungraded pr_diff run (evaluation_status = "ungraded": the grader could not produce a verdict for reasons outside the agent's control, such as an unavailable judge or a verifier crash) writes reward-details.json with a null reward but no reward file. Harbor raises RewardFileNotFoundError, classes the trial as VERIFIER_FAILED with reward_valid = false and does not retry it; the trial's result.json shows verifier_result: null and the exception under exception_info (the failure cause and reward_valid are computed from it, not stored). RL consumers drop the rollout instead of training on a 0, and Harbor's job mean and pass@k still count it as 0. Failures the agent can cause (a capture failure from a broken Git object store, a judge answer with no valid score, a refused or filtered judge reply, a predicted patch too large to score) are graded 0, not ungraded. Native runtime pipelines also omit ungraded rewards, as described below. Research recipes and Tasksmith retain their pipeline-specific reward contracts.

Files written per taskCopy Anchor Link

Ungraded runs (native runtime pipelines). pr_runtime, commit_runtime, cve_patches, code_instruct and equivalence_tests grade in Harbor's separate verifier. When grading fails for a reason the submission can't cause (no system python3 in the image, a grader step that can't start, a crashed verifier), the run writes reward-details.json with "reward": null and "evaluation_status": "ungraded" and no reward file. Harbor then records the trial as VERIFIER_FAILED (reward_valid false; its result.json shows verifier_result: null and a RewardFileNotFoundError), RL consumers drop the rollout instead of training on a 0, and harbor run's job mean and pass@k still count it as 0. A failure the submission can cause is a graded 0 instead (evaluation_status: "graded", reward file written): see the table under pr_runtime.

pr_diffCopy Anchor Link

Reward kind: diff_similarity

reward.txt: weighted combination of 6 components, clamped to [0, 1].

reward-details.json:

{
  "final_reward": 0.74,
  "components": {
    "format_valid":   1.0,
    "size_sanity":    0.85,
    "file_targeting": 0.67,
    "region_overlap": 0.60,
    "similarity":     0.55,
    "llm_judge":      0.80
  },
  "weights": {
    "format_valid":   0.00,
    "size_sanity":    0.08,
    "file_targeting": 0.12,
    "region_overlap": 0.20,
    "similarity":     0.10,
    "llm_judge":      0.50
  },
  "judge_model":  "claude-haiku-4-5-20251001",
  "judge_endpoint": null,
  "judge_status": "ok"
}

Field reference:

Weight override env vars (set inside the verifier container via --ve): R2E_W_FORMAT, R2E_W_SIZE, R2E_W_FILE, R2E_W_REGION, R2E_W_SIM, R2E_W_JUDGE

Judge routing env vars (also via --ve): R2E_JUDGE_MODEL, R2E_JUDGE_ENDPOINT, R2E_JUDGE_API_KEY. See ENV.md.

pr_runtime · commit_runtime · cve_patchesCopy Anchor Link

Reward kind: test_execution (primary) + diff_similarity (fallback)

reward.txt: f2p_rate × p2p_rate, rounded to 6 decimal places.

reward-details.json (normal path: test output parsed successfully):

{
  "reward":                  0.833333,
  "resolved":                false,
  "command_resolved":        false,
  "f2p_total":               3,
  "f2p_passed":              2,
  "f2p_rate":                0.666667,
  "p2p_total":               5,
  "p2p_passed":              5,
  "p2p_rate":                1.0,
  "regressions":             [],
  "untracked_failed_count":  1,
  "untracked_failed":        ["tests/test_other.py::test_legacy"],
  "parse_status":            "ok",
  "runner":                  "pytest",
  "tests_parsed":            12,
  "exit_code":               1
}

reward-details.json (fallback path: test output unrecognised, no per-test status):

{
  "reward":            0.0,
  "resolved":          false,
  "command_resolved":  false,
  "parse_status":      "fallback_exitcode",
  "eval_trustworthy":  false,
  "runner":            "",
  "f2p_total":         3,
  "p2p_total":         5,
  "exit_code":         0
}

Field reference:

Grading outcomes before or around the tests. Each writes parse_status, resolved: false and eval_trustworthy: false:

Choosing between reward, resolved, and command_resolved:

• Training: use reward (dense, graded signal from reward.txt).
• Benchmark / leaderboard: use resolved (strict SWE-bench tracked resolution, unaffected by untracked flakiness).
• Strict eval (clean command required): use command_resolved.

code_instruct · equivalence_testsCopy Anchor Link

Reward kind: test_execution

reward.txt: 1.0 or 0.0. Binary.

The verifier runs the hidden pytest file against the submitted task_module.py as an unprivileged user and grades its JUnit report, read only after every process of the test user is gone. Reward is 1.0 iff pytest exits 0 and the report holds exactly the test IDs the reference solution passed at generation (tests/expected.json), every one passed: a skipped, missing or extra test fails. Tasks generated with skip_validation have no expected list and, with nothing measured, a 300 s per-run limit; their report is trusted as is (eval_trustworthy: false).

reward-details.json:

{
  "reward":            1.0,
  "resolved":          true,
  "command_resolved":  true,
  "eval_trustworthy":  true,
  "parse_status":      "ok",
  "tests_expected":    6,
  "tests_parsed":      6,
  "tests_passed":      6,
  "missing":           [],
  "unexpected":        [],
  "not_passed":        [],
  "exit_code":         0,
  "run_seconds":       1.82,
  "evaluation_status": "graded"
}

parse_status is "ok", "invalid_report: …" (no usable report, reward 0), "test_timeout", or one of the outcomes in the table above (submission_not_regular and submission_too_large for the submitted file). The mutant check runs here too, against the expected test IDs, with the same tamper_check fields.

Reading the reward for training:

reward = float(open("/logs/verifier/reward.txt").read().strip())  # 1.0 or 0.0

Per-pipeline summaryCopy Anchor Link