GPT-6 Astra Benchmarks
A source-aware benchmark reference: vendor-reported results stay clearly separated from the independent tests this site still needs to run.
Important: these are OpenAI-reported numbers, not independent tests by this site. Production results can differ because of prompts, tools, system instructions and reasoning settings.
Selected official results
| Benchmark | Category | GPT-6 Astra | GPT-5.6 Sol | Claude Opus 5 |
|---|---|---|---|---|
| Agents' Last Exam | Computer use | 59.3% | 53.6% | 55.5% |
| OSWorld 2.0 | Computer use | 72.6% | 65.7% | 70.2% |
| ScreenSpot-Pro | Computer use | 92.7% | 76.9% | — |
| AutomationBench | Professional | 41.4% | 18.1% | 26.9% |
| BenchCAD | Professional | 95.9% | 83.3% | 82.1% |
| BrowseComp | Professional | 91.5% | 90.4% | 90.8% |
| MRCR v2 256K–512K | Long context | 100.0% | 91.5% | — |
| MRCR v2 512K–1M | Long context | 96.3% | 73.8% | — |
| ARC-AGI-3 | Reasoning | 99.9% | 7.8% | 30.2% |
| ARC-AGI-2 | Reasoning | 95.0% | 92.5% | 90.4% |
| ARC-AGI-1 | Reasoning | 98.5% | 97.5% | 97.5% |
| ExploitBench | Cybersecurity | 100.0% | 78.5% | — |
| ExploitGym | Cybersecurity | 42.4% | 30.3% | — |
Vendor-reported figures from OpenAI's GPT-6 Astra launch materials. “—” means the launch table did not publish a directly comparable value in that row.
What stands out
- Long context: OpenAI reports 96.3% on MRCR v2 at 512K–1M, compared with 73.8% for GPT-5.6 Sol in the same launch table.
- Computer use: OpenAI reports 72.6% on OSWorld 2.0 versus 65.7% for GPT-5.6 Sol and 70.2% for Claude Opus 5.
- Abstract reasoning: OpenAI reports 99.9% on ARC-AGI-3. Treat this as a vendor result until independently reproduced.
How to read these numbers
OpenAI notes that some GPT evaluations were run in research environments or API harnesses and can differ from production ChatGPT. Cross-vendor comparisons also need care because vendors can use different harnesses, tool setups and scoring conventions.
Independent test plan for this site
- Publish exact prompts, model IDs, reasoning settings and test date.
- Repeat tasks instead of reporting a single lucky run.
- Measure quality, latency and estimated API cost together.
- Keep vendor-published results visually separate from our own measurements.
Primary sources
Specifications and prices can change. We prefer first-party documentation and record the date each page was reviewed.