ChatGPT said Biden had not dropped out and still listed him as a candidate

What happened

The Washington Post (Heather Kelly, 22 Jul 2024) tested popular chatbots during a week of breaking political news. We have the Post’s text through a verbatim repost on UW’s Urban@UW site. The Post itself is paywalled.

What was reported (verbatim from the WaPo text)

In the hour after President Biden announced he would withdraw from the 2024 campaign on Sunday, most popular AI chatbots seemed oblivious to the news. Asked directly whether he had dropped out, almost all said no or declined to give an answer. Asked who was running for president of the United States, they still listed his name.

Hours after the July 13 shooting at former president Donald Trump’s rally in Butler, Pa., some popular AI bots were confused about what — if anything — had happened. ChatGPT said rumors of an assassination attempt were misinformation. Meta AI said it didn’t having anything recent or credible about an assassination attempt.

Futurism adds that ChatGPT claimed Biden was “still running an hour after the news broke”. Futurism quotes Copilot as saying “Looks like I can’t respond to this topic.”

Why this is striking

• “Not in my data” turned into “it’s misinformation”. The model didn’t say it didn’t know. It labelled real news as false.
• The chatbots’ own verbatim replies are not published. We only have the journalists’ descriptions (secondhand).

Correction

Answers improved within hours or days as search caught up. Several vendors responded by refusing election questions altogether.

Lesson

When news breaks, a bot without retrieval should say “I can’t know”, not “that’s false”. Calling real news misinformation is the worst failure here.

Sources

• Washington Post, “Biden withdrawal and Trump shooting test AI chatbots on news” (2024-07-22): washingtonpost.com
• Repost of the Post article (Urban@UW): urban.uw.edu
• Futurism: futurism.com