Historical internal evaluation records, inspected 5 October 2026.

50 selected SWE-bench Pro tasks; 40 resolved, 10 unresolved. Not an official full-benchmark score or independent rerun. Sampling procedure and representativeness have not been independently established.

pro-50-results.csv is an allowlisted column extract, not the unchanged original CSV. Original CSV SHA-256 is in batch-history.json. pro-50-receipts.json is derived from saved result objects and contains task/model/settings, outcomes and raw grader artifact hashes. The grader JSON and test-summary log copies in pro-50-harness are unchanged.

Model names are configuration identifiers as recorded. They are not independent provider/model identity verification. Runtime in-task checks and benchmark grading are separate: 48 of 50 records show in-task checks not_verified; two show verified. All 50 have separate grader records.

Costs are recorded model charges/fallback estimates, exclude AWS infrastructure and are not verified invoices. All 50 candidate run results mark model cost source estimated. A costEstimated=false CSV field indicates reported usage rather than a substituted cap; it does not establish invoice accuracy.

batch-history includes the 19 batches traced through the evaluation Slack channel, including cancelled batches and model-request failures. It is not a pooled score: datasets, samples, models, task overlaps and runner versions differ. The full launch metadata and Slack exports are not public artifacts.