SIREN LABS · LIVE BENCH · WEEK OF OCT 12, 2026
Is AI building AI yet?
Five open-data instruments from this week’s Labs bench. Each one says what it counts and where the number comes from. None of them is a forecast.
The Recursion Meter
How much of the code that builds AI is now written by AI?
| REPO | AGENT | ALL | SHARE |
|---|---|---|---|
| pytorch/pytorch | 0 | 16 | 0.0% |
| huggingface/transformers | 0 | 146 | 0.0% |
| vllm-project/vllm | 28 | 524 | 5.3% |
| jax-ml/jax | 0 | 216 | 0.0% |
| ggml-org/llama.cpp | 2 | 190 | 1.1% |
Agent PRs = merged pulls from a coding agent’s GitHub app account or from a branch named with an agent’s default prefix (codex/, claude/, …), same roster as Takeover Watch. Counts are summed across separate searches, so the share is an upper-bound proxy. GitHub public search; same agent list as Takeover Watch.
The Task-Horizon Clock
How long a job can an AI agent finish on its own?
| MODEL | RELEASE | P50 HORIZON | SOTA |
|---|---|---|---|
| claude mythos preview early | 2026-04-07 | 17 h | yes |
| claude opus 4 6 | 2026-02-05 | 12 h | yes |
| gemini 3 1 pro | 2026-02-19 | 6.4 h | — |
| gpt 5 2 | 2025-12-11 | 5.9 h | yes |
| gpt 5 3 codex | 2026-02-05 | 5.8 h | — |
| gpt 5 4 | 2026-03-05 | 5.7 h | — |
| claude opus 4 5 | 2025-11-24 | 4.9 h | yes |
| gemini 3 pro | 2025-11-18 | 3.7 h | yes |
| gpt 5 1 codex max | 2025-11-19 | 3.7 h | — |
| gpt 5 2025 08 07 | 2025-08-07 | 3.4 h | yes |
| o3 | 2025-04-16 | 2.0 h | yes |
| claude 4 1 opus | 2025-08-05 | 1.7 h | — |
METR p50_horizon_length is the human task length in minutes where the model has a 50% chance of success. METR marks readings above 16 hours as less reliable with the current task suite. Source: METR time horizons (METR-Horizon-v1.1). Raw YAML.
Capability Jump Alarm
When does a new model land far off the trend?
No release sits more than two standard deviations above the disclosed-compute trend.
log10(training FLOP) vs publication date since 2018-01-01; flag residual > 2σ. Epoch AI notable-models dataset (CC BY). Only models with a disclosed training-compute figure from 2018 onward enter the fit. LMArena and Artificial Analysis leaderboards are named on the Labs card; this live cut uses Epoch’s open compute column until those boards expose a stable public series.
The Compute Clock
How fast is frontier training compute growing?
Epoch AI notable-models dataset (CC BY). Envelope = running maximum of disclosed training FLOP. Epoch CSV.
The Lab Silence Index
Do labs go quiet before a big release?
| LAB | DAYS SILENT | NEWS 7D | FLAG |
|---|---|---|---|
| OpenAI | 0 | 49 | NORMAL |
| Anthropic | 0 | 42 | NORMAL |
| Google DeepMind | 0 | 5 | NORMAL |
| xAI | 0 | 1 | NORMAL |
| Meta | 4 | 18 | NORMAL |
| DeepSeek | — | 21 | UNREAD |
| Mistral | — | 5 | QUIET |
| Qwen | — | 12 | UNREAD |
Silence is measured from the Leaders tracker (days since a sourced public statement). “Unusual” needs a few weeks of history of each lab’s own gap; until then a gap of 7+ days is marked long, not unusual. Scored after the fact against what shipped next once releases land in the newsroom. Principals from the Leaders tracker; news counts from the newsroom window.
Updated 2026-10-21 18:05 UTC. Open data only. Show the method. Count, don’t predict.