# GPT development comparison

Same preselected development cases, A/full, one attempt per model. No combined capability score or leaderboard claim.

Status: **complete**. Planned 18; launched 18; scored 18; infrastructure failures 0; missing 0; inflight 0.

| Model | Read exact | Write exact | Confirmed JSON objects / responses | Median scored-session seconds | Known API estimate | Unknown-cost sessions |
|---|---:|---:|---:|---:|---:|---:|
| openai/gpt-6-luna | 0/4 | 0/2 | 6/6 | 117.29 | $0.083433 | 0 |
| openai/gpt-6.1-terra | 0/4 | 1/2 | 6/6 | 147.94 | $0.746173 | 0 |
| openai/gpt-6.1-sol | 0/4 | 1/2 | 6/6 | 122.94 | $1.428556 | 0 |

Infrastructure failures are not failed capability answers. Known cost excludes unresolved charges; it is not a complete total when unknown-cost sessions exist. Latency includes the local session and API round trip.

## Per-family results

| Model | Family | Exact / scored | Functional accuracy | Scored parents |
|---|---|---:|---:|---:|
| openai-luna | compositional_synthesis | 0/2 | 71.9% | 2 |
| openai-luna | edit_effects | 0/2 | 45.6% | 2 |
| openai-luna | rule_recovery | 0/2 | 42.7% | 2 |
| openai-terra | compositional_synthesis | 1/2 | 97.3% | 2 |
| openai-terra | edit_effects | 0/2 | 53.4% | 2 |
| openai-terra | rule_recovery | 0/2 | 78.5% | 2 |
| openai-sol | compositional_synthesis | 1/2 | 95.8% | 2 |
| openai-sol | edit_effects | 0/2 | 86.7% | 2 |
| openai-sol | rule_recovery | 0/2 | 88.6% | 2 |

## Write control context

Static difficulty control on the identical selected targets. Recorded in B, it reads no weights or feedback; it is neither an A model score nor additional independent tasks.

The static zero-start/no-edit control has exact success **0/2**, target accuracy 70.1%, affected-case accuracy 0.0%, and retention 100.0%. No independent tasks are added.

Class imbalance makes raw Write accuracy insufficient evidence of successful construction.

| Model | Write target accuracy | Affected-case accuracy | Retention | Exact / scored |
|---|---:|---:|---:|---:|
| openai-luna | 71.9% | 7.6% | 99.6% | 0/2 |
| openai-terra | 97.3% | 96.1% | 97.7% | 1/2 |
| openai-sol | 95.8% | 93.4% | 96.7% | 1/2 |

## API tokens

Output includes API-reported reasoning; the reasoning column is a subset, not an additional billable output category.

| Model | Input total | Uncached | Cache write | Cache read | Output | Reasoning subset |
|---|---:|---:|---:|---:|---:|---:|
| openai-luna | 310938 | 18 | 310920 | 0 | 89132 | 87169 |
| openai-terra | 310938 | 18 | 310920 | 0 | 71501 | 67367 |
| openai-sol | 310938 | 18 | 310920 | 0 | 65122 | 59867 |

## Interpretation

- This small development comparison does not support a headline ranking or a combined capability score.
- Read is the mean of rule-recovery agreement and edit-effects before/after agreement, once per parent.
- Write accuracy alone is insufficient because the selected targets are class-imbalanced.
- Exact success uses the scored final cases. The two exact Write constructions also match all 1,024 domain entries.
- Median time includes the local session and API round trip. It is not isolated provider compute time.
- API costs are estimates for six scored sessions, using the run's recorded pricing and usage.
- The model names are provider aliases rather than immutable dated snapshots.
- Skill and family rows describe overlapping observations and must not be summed.