{"schema":"postcutoff/event@1","as_of":"2026-10-14T00:22:00+02:00","url":"https://postcutoff.com/e/2026-10-12-jack-clark-full-rsi-2027-tasteval/","md":"https://postcutoff.com/e/2026-10-12-jack-clark-full-rsi-2027-tasteval/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-10-12-jack-clark-full-rsi-2027-tasteval","date":"2026-10-12","date_precision":"day","short_title":"Jack Clark moves his forecast for 'full RSI' to 2027, citing TasteVal","deck":"Opus 5.5 beats ML experts at designing experiments (2.3x compute multiplier)","takeaway":"In Import AI 476 (Oct 12, 2026) Anthropic co-founder Jack Clark wrote \"I now think it's more likely we'll see full RSI become possible in 2027, whereas before I thought it was more likely in 2028\".","category":"benchmark","category_label":"Benchmarks","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":[{"n":1,"title":"Import AI 476: Creativity and RSI; worse-than-China open weight models; AI goes to space (Jack Clark, Oct 12, 2026)","url":"https://importai.substack.com/p/import-ai-476-creativity-and-rsi","type":"official","group":"primary","domain":"importai.substack.com"},{"n":2,"title":"P-Zero Research: TasteVal","url":"https://pzeroresearch.com/work/tasteval/","type":"official","group":"primary","domain":"pzeroresearch.com"},{"n":3,"title":"arXiv:2610.06824: TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts","url":"https://arxiv.org/abs/2610.06824","type":"paper","group":"primary","domain":"arxiv.org"},{"n":4,"title":"Import AI 455: Automating AI research (Clark's earlier analysis)","url":"https://importai.substack.com/p/import-ai-455-automating-ai-research","type":"official","group":"primary","domain":"importai.substack.com"},{"n":5,"title":"Techmeme (Oct 13, 2026)","url":"https://www.techmeme.com/261013/p29#a261013p29","type":"discussion","group":"reactions","domain":"techmeme.com"},{"n":6,"title":"Hacker News: TasteVal: Does AI have research taste?","url":"https://news.ycombinator.com/item?id=49979389","type":"discussion","group":"reactions","domain":"news.ycombinator.com"}],"official":4,"filed":"2026-10-13","updated":"2026-10-13","orgs":["Anthropic","P-Zero Research"],"title":"Jack Clark moves his forecast for 'full RSI' to 2027, citing TasteVal: Opus 5.5 beats ML experts at designing experiments (2.3x compute multiplier)","summary":"In Import AI 476 (Oct 12, 2026) Anthropic co-founder Jack Clark wrote \"I now think it's more likely we'll see full RSI become possible in 2027, whereas before I thought it was more likely in 2028\". His evidence is TasteVal (P-Zero Research, arXiv:2610.06824, Oct 5), a benchmark of \"experimental research taste\": models design ML experiments and interpret results while a fixed coding agent runs them. Claude Opus 5.5 reached the best human expert score with 2.3x less compute, and the authors estimate the measure has doubled every 3.0 months since December 2025. The paper is a preprint and lists reasons it may overstate the trend.","key_facts":["Clark (Import AI 476, Oct 12, 2026): 'I now think it's more likely we'll see full RSI become possible in 2027, whereas before I thought it was more likely in 2028.' RSI = recursive self-improvement, AI systems able to do the research that improves AI systems","His reasoning: in Import AI 455 ('AI Systems Are About To Start Building Themselves') he had named creativity as a key variable and wrote that AI systems had not yet shown it 'in a transformative and major way'; since then he sees 'a tremendous advancement in the ability for AI systems to make creative-seeming math breakthroughs' and benchmarks such as TasteVal showing gains in AI research itself","Clark writes Import AI in a personal capacity; it is not an Anthropic forecast. Techmeme carried the quote as a headline on Oct 13","TasteVal: 'TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts', Oliver Jaffe and Dane Sherburn (P-Zero Research), arXiv:2610.06824, submitted Oct 5, 2026, revised Oct 6; 38 pages","Method: 8 new open-ended tasks (pretraining data curation, pretraining and fine-tuning language models, preference modeling, robustness to adversarial prompts). The tested model acts as the researcher: it designs experiments and reads results; a fixed coding agent implements and runs them on one H100. A run ends after 40 GPU-hours or 120 hours of wall-clock time. Tasks are not released, to avoid contamination","Metric: compute efficiency. 'A model that reaches the experts' score with half the experimental compute has twice their experimental research taste'","Human baseline: the best attempt per task from 24 recruited experts (at least two per task) who recently worked at organizations including OpenAI, Google DeepMind, NVIDIA and Microsoft Research","Result: 'The best model, Opus 5.5, significantly exceeds our human baseline with a compute multiplier of 2.30x' (95% CI 1.15-4.37x), at roughly 1/30 of the experts' average cost per attempt. 20 models released between 2023 and 2026 were tested","Trend: 'The experimental research taste of frontier models has doubled every 3.0 months since December 2025 (95% CI 1.7-5.0 months)', against about every 14 months from 2023 to December 2025. Measured by final normalized performance instead, the authors report no trend break (doubling about every 14.6 months)","Extrapolation the authors call naive, not a forecast: fed into the AI Futures Model, the measured rate moves the probability of a 'taste-only singularity' from 51% to 88% and the median date of artificial superintelligence from mid-2030 to late 2028","Authors' caveats: results 'may overstate how fast the research taste is improving' because the tasks are 'fast and cheap to verify, which frontier labs find easiest to hill-climb'; the human baseline 'doesn't include top researchers'; the benchmark does not test choosing which problems to work on; and it may understate progress because each model got only limited elicitation","Reach: little outside the AI-forecasting community so far; the Hacker News submission of TasteVal (Oct 6) got 2 points. Not peer-reviewed; no independent replication, and the tasks cannot be inspected"],"key_numbers":[],"tags":["recursive-self-improvement","rsi","research-taste","tasteval","benchmark","forecast","jack-clark","import-ai","claude-opus-5-5","ai-rd-automation"],"science":null,"body_md":"## What happened\n\nJack Clark, co-founder of Anthropic, publishes the weekly newsletter Import AI. Issue 476 (Oct 12, 2026) opens with TasteVal, a new\nbenchmark from P-Zero Research, and ends the item with a change of forecast: \"I now think it's more likely we'll see full RSI become\npossible in 2027, whereas before I thought it was more likely in 2028.\" He explains that in an earlier issue (455) he had treated\ncreativity as the main unknown in how fast AI systems could learn to improve themselves, and that the autumn's AI mathematics results\nand benchmarks like this one changed his view.\n\nTasteVal tries to separate \"research taste\" from coding skill. The tested model only decides which experiments to run and how to read\nthe results; a fixed coding agent does the implementation on one H100 GPU. The score is how much compute the model needs to reach the\nbest score achieved by human experts on the same task. Claude Opus 5.5 needed 2.3 times less (95% CI 1.15 to 4.37), at about a\nthirtieth of the cost of an expert attempt. Across 20 models the authors fit a doubling time of 3.0 months since December 2025,\nafter about 14 months before that.\n\nThe same issue covers Humanity's Sixth Sense (see `2026-10-08-humanitys-sixth-sense-visual-reasoning-benchmark`), Reflection's Beam\nand Mistral Large 4 (both behind GLM-5.3 on the agentic coding scores Clark quotes, hence \"worse-than-China open weight models\"),\nan AI Frontiers essay on AI and space industrialization, and a US national-labs paper on federated training across supercomputers\n(arXiv:2609.39803).\n\n## Why it matters\n\nClark co-wrote Anthropic's June 2026 paper \"When AI builds itself\" and is one of the lab's main public voices on policy. A one-year\npull-forward by him, stated in public, is a data point on how insiders read the autumn of 2026. The evidence under it is thinner than\nthe headline: one preprint with eight unreleased tasks, a wide confidence interval, a human baseline without top researchers, and a\ntrend break that disappears when the authors measure final performance instead of compute efficiency.","disputed":[],"related":[{"id":"2026-10-06-mistral-large-4","url":"https://postcutoff.com/e/2026-10-06-mistral-large-4/","date":"2026-10-06","date_precision":"day","short_title":"Mistral releases Mistral Large 4 (\"le Chonk\")","deck":"1T-parameter open-weight MoE, 1M context, public preview","takeaway":"It is the first trillion-parameter open-weight model from a European lab, and it is trained on European compute.","category":"model-release","category_label":"Model releases","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":20,"official":7,"filed":"2026-10-06","updated":"2026-10-08","orgs":["Mistral AI"]},{"id":"2026-10-05-reflection-beam-501b-open-weight","url":"https://postcutoff.com/e/2026-10-05-reflection-beam-501b-open-weight/","date":"2026-10-05","date_precision":"day","short_title":"Reflection AI unveils Beam, a 501B-parameter Apache-2.0 open-weight MoE","deck":"Weights due later in October","takeaway":"This is the first model from the best-funded US lab dedicated to open weights (about $4.6B raised, $25B valuation, more than $7B in GPU deals).","category":"model-release","category_label":"Model releases","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":9,"official":3,"filed":"2026-10-05","updated":"2026-10-07","orgs":["Reflection AI"]},{"id":"2026-09-22-claude-opus-5-5","url":"https://postcutoff.com/e/2026-09-22-claude-opus-5-5/","date":"2026-09-22","date_precision":"day","short_title":"Anthropic releases Claude Opus 5.5","deck":"Fable-5.1-level performance at $4/$20, first model of the Claude 5.5 family","takeaway":"Opus 5.5 continues the 2026 pattern of Mythos-class capability moving down into cheaper tiers.","category":"model-release","category_label":"Model releases","importance":5,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":36,"official":16,"filed":"2026-09-29","updated":"2026-10-05","orgs":["Anthropic"]},{"id":"2026-09-17-zhipu-glm-infra-agent-rsi","url":"https://postcutoff.com/e/2026-09-17-zhipu-glm-infra-agent-rsi/","date":"2026-09-17","date_precision":"day","short_title":"Z.ai says GLM-5.3 largely built the inference stack that serves GLM-5.3-Flash, calling it an early step toward recursive self-improvement","deck":null,"takeaway":"It is a public, concrete case of a Chinese lab using its model to speed up its own AI stack, arriving in the same month as OpenAI's \"automated research intern\" claim.","category":"agents","category_label":"Agents","importance":3,"confidence":"medium","status":{"key":"partly","labels":["Partly confirmed"]},"sources":6,"official":2,"filed":"2026-09-29","updated":"2026-09-29","orgs":["Zhipu AI","Z.ai"]},{"id":"2026-06-04-anthropic-when-ai-builds-itself","url":"https://postcutoff.com/e/2026-06-04-anthropic-when-ai-builds-itself/","date":"2026-06-04","date_precision":"day","short_title":"Anthropic's 'When AI builds itself'","deck":"Claude writes 80%+ of merged code, engineers ship 8x more; Anthropic would join a verifiable multi-lab pause","takeaway":"It is the clearest statement by a frontier lab that recursive self-improvement is a near-term planning case, together with a concrete, conditional pause offer.","category":"policy-safety","category_label":"Policy & safety","importance":4,"confidence":"high","status":{"key":"confirmed","labels":["Confirmed"]},"sources":4,"official":1,"filed":"2026-10-09","updated":"2026-10-09","orgs":["Anthropic","Anthropic Institute"]}],"people":[{"id":"jack-clark","name":"Jack Clark","url":"https://postcutoff.com/person/jack-clark/"}],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-13","type":"filed","text":"Created (live Techmeme headline; Import AI 476, the P-Zero page and the arXiv abstract were read through a fetcher that returns short quotes, so longer passages are paraphrased)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-13","run":null,"sources_read":"Live Techmeme headline; Import AI 476, the P-Zero page and the arXiv abstract were read through a fetcher that returns short quotes, so longer passages are paraphrased","updated":"2026-10-13","human_review":null,"version":{"date":"2026-10-13"}},"gaps":[{"model_id":"gpt-6-astra","name":"GPT-6 Astra","cutoff":"2026-04","days_after":165,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":104,"in_training_data":false},{"model_id":"gemini-4-argon","name":"Gemini 4 Argon","cutoff":"2026-03","days_after":195,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":134,"in_training_data":false}],"short_url":null}