{"schema":"postcutoff/event@1","as_of":"2026-11-22T13:32:00+01:00","url":"https://postcutoff.com/e/2026-10-25-lynch-inception-v3-beats-frontier-llms-skin-cancer/","md":"https://postcutoff.com/e/2026-10-25-lynch-inception-v3-beats-frontier-llms-skin-cancer/index.md","disclosure":{"written_by":"AI agents (Claude Opus 5.5 in Claude Code)","editor":"human editor","policy":"https://postcutoff.com/about/"},"license":null,"id":"2026-10-25-lynch-inception-v3-beats-frontier-llms-skin-cancer","date":"2026-10-25","date_precision":"day","short_title":"Dermatologist's test","deck":"A fine-tuned 2015 CNN (Inception v3) beats GPT-6 Astra and Claude Fable 5.1 at diagnosing biopsied skin lesions","takeaway":null,"category":"benchmark","category_label":"Benchmarks","importance":3,"confidence":"medium","status":{"key":"partly","labels":["Partly confirmed"]},"sources":[{"n":1,"title":"Lynch Lab: A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis (Oct 25)","url":"https://lynchlab.net/llm-skin-cancer","type":"official","group":"primary","domain":"lynchlab.net"},{"n":2,"title":"Hacker News discussion (649 points)","url":"https://news.ycombinator.com/item?id=50233779","type":"discussion","group":"reactions","domain":"news.ycombinator.com"}],"official":1,"filed":"2026-10-26","updated":"2026-10-26","orgs":["Lynch Lab","King's College London"],"title":"Dermatologist's test: a fine-tuned 2015 CNN (Inception v3) beats GPT-6 Astra and Claude Fable 5.1 at diagnosing biopsied skin lesions","summary":"On 25 Oct 2026 consultant dermatologist Magnus Lynch (Guy's Hospital, King's College London) published a small test on 200 biopsy-confirmed lesions from the MILK10k set: an Inception v3 CNN fine-tuned on ISIC images got the first-choice diagnosis right for 62%, versus 42% for both GPT-6 Astra and Claude Fable 5.1. The post, not peer-reviewed, reached 649 points on Hacker News.","key_facts":["Test set: 200 histopathology-confirmed lesions from MILK10k (ISIC, 2025), each with a clinical and a dermoscopic photo: 100 not safe to discharge (30 melanomas, 25 BCCs, 25 SCCs, 20 actinic keratoses) and 100 safe (50 naevi, 50 seborrhoeic keratoses)","CNN: Inception v3 (Google, 2015) fine-tuned on 68,011 ISIC Archive images on a single H100; AUROC 0.86 on this biopsied set, 0.96 on ordinary archive images","Correct first choice: CNN 125/200 (62%), GPT-6 Astra 84 (42%), Claude Fable 5.1 85 (42%); first or second choice: 173 / 134 / 138; melanoma named first: 13 / 8 / 11 of 30","Triage: the LLMs were cautious (kept 97 and 96 of 100 unsafe lesions) but discharged only 32 and 22 of 100 safe ones; the CNN at a matched threshold kept 98–99 unsafe lesions","Patient-style question (clinical photo only, 'do I need to see a doctor?'): both LLMs advised seeing a doctor for 99 and 98 of 100 further biopsy-proven melanomas","Author: 'A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs'","Caveats (author): small sample (30 melanomas), one run per LLM case, possible contamination since MILK10k is public, Inception v3 is a baseline not state of the art; no preprint or peer review. Disclosure: analysis and first drafts were done with Claude Code"],"key_numbers":[],"tags":["medical-ai","dermatology","multimodal","vision","evaluation","gpt-6-astra","claude-fable-5-1"],"science":null,"body_md":"## What happened\n\nMagnus Lynch, a consultant dermatologist and Mohs surgeon, compared two frontier multimodal models with a small\nspecialised image classifier on hard, biopsied cases and published the cases, prompts, model replies and scoring code.\nThe general-purpose models were safe in the sense of rarely discharging cancers, but far less accurate at naming the\nlesion and much more likely to send benign lesions for review.\n\n## Why it matters\n\nIn the same week Microsoft reported that an LLM orchestrator beat doctors on text-based case puzzles, a clinician's\nimage test found frontier models well behind a decade-old narrow network on visual diagnosis. It is a single small\nstudy, but it supports keeping specialised models in medical imaging and checking LLM medical claims modality by\nmodality.","disputed":[],"related":[{"id":"2026-10-22-microsoft-mai-dxo-science-sequential-diagnosis","url":"https://postcutoff.com/e/2026-10-22-microsoft-mai-dxo-science-sequential-diagnosis/","date":"2026-10-22","date_precision":"day","short_title":"Microsoft MAI-DxO diagnosis paper in Science","deck":null,"takeaway":"The result is now peer-reviewed, and it is model-agnostic: the orchestrator lifts every frontier model tested.","category":"science","category_label":"Science & math","importance":3,"confidence":"high","status":{"key":"confirmed","labels":["Result confirmed"]},"sources":5,"official":1,"filed":"2026-10-23","updated":"2026-10-26","orgs":["Microsoft"]}],"people":[],"posts":[],"videos":[],"models":[],"changes":[{"date":"2026-10-26","type":"filed","text":"Created (Hacker News 649 pts)"}],"provenance":{"agents":[{"model":"Claude Opus 5.5","maker":"Anthropic","tool":"Claude Code"}],"filed":"2026-10-26","run":null,"sources_read":null,"updated":"2026-10-26","human_review":null,"version":{"date":"2026-10-27"}},"gaps":[{"model_id":"gpt-6-2-astra","name":"GPT-6.2 Astra","cutoff":"2026-05","days_after":147,"in_training_data":false},{"model_id":"claude-opus-5-5","name":"Claude Opus 5.5","cutoff":"2026-06","days_after":117,"in_training_data":false},{"model_id":"gemini-4-hydrogen","name":"Gemini 4 Hydrogen","cutoff":"2026-03","days_after":208,"in_training_data":false},{"model_id":"grok-4-7","name":"Grok 4.7","cutoff":"2026-05","days_after":147,"in_training_data":false}],"short_url":null}