Siri Gives Eagles 33 False Super Bowl Wins in Basic Knowledge Test

In what may not come as much of a surprise, a new test of Siri's knowledge of Super Bowl history has revealed significant accuracy issues with Apple's virtual assistant, suggesting Apple still has some way to go in overcoming challenges with Siri's ability to provide reliable information.

Should Apple Kill Siri Feature
In a methodical experiment, One Foot Tsunami's Paul Kafasis asked Siri who won each Super Bowl from I through LX and documented its responses. The results were strikingly poor, with Siri correctly identifying winners only 34% of the time – just 20 correct answers out of 58 played Super Bowls.

Perhaps most notably, Siri repeatedly and incorrectly credited the Philadelphia Eagles with 33 Super Bowl victories, despite the team having won only one championship in their history. The virtual assistant's responses ranged from providing information about wrong Super Bowls to offering completely unrelated football facts.

While Siri did manage a few streaks of accurate answers, including three consecutive correct responses for Super Bowls V through VII, it also had a remarkable string of 15 consecutive incorrect answers spanning Super Bowls XVII through XXXII.

In one telling instance, when asked about Super Bowl XVI, Siri offered to defer to ChatGPT - which then provided the correct answer. The contrast highlighted the limitations of Siri's own knowledge base compared to more advanced AI systems.

The test was conducted on iOS 18.2.1 with Apple Intelligence enabled, and similar results were found on both the upcoming iOS 18.3 beta and macOS 14.7.2, suggesting the issue extends across Apple's platforms. Kafasis generated a spreadsheet of the results in both Excel and PDF formats, which you can read here.

Separately, inspired by Kafasis' test, Daring Fireball's John Gruber tried some of his own sports queries with Siri and compared its responses to ChatGPT, Kagi, DuckDuckGo, and Google, all of which succeeded where Siri failed.

Perhaps worse for Apple, Gruber found that old Siri (i.e. before Apple Intelligence) did a better job at answering a question by declining to answer it, instead providing a list of web links. The first web result provided an accurate, if only partial, answer to the question, whereas new Siri, powered by Apple Intelligence, fared much worse. Gruber explains:

New Siri — powered by Apple Intelligence™ with ChatGPT integration enabled — gets the answer completely but plausibly wrong, which is the worst way to get it wrong. It's also inconsistently wrong — I tried the same question four times, and got a different answer, all of them wrong, each time. It's a complete failure.

"It's just incredible how stupid Siri is about a subject matter of such popularity," commented Gruber. "If you had guessed that Siri could get half the Super Bowls right, you lost, and it wasn't even that close."

Of course, this isn't the first time Siri has received heavy flak for its all-round performance, but Gruber's criticism about "plausibly wrong" answers to general knowledge questions ties back to the modern problem of hallucinating AI chatbots that spout misleading or flat-out wrong responses with complete confidence.

Apple is developing a much smarter version of Siri that utilizes advanced large language models, which should allow the personal assistant to better compete with chatbots like ChatGPT. A chatbot version of Siri would likely be able to hold ongoing conversations and provide the sort of help and insight as ChatGPT or Claude, but how well the integration will perform may be a concern, going on Siri's abysmal track record.

Apple is expected to announce LLM Siri as soon as 2025 at WWDC, but Apple won't launch it until several months after it's unveiled. That means LLM Siri would come in an update to iOS 19, with Apple planning for a spring 2026 launch.

Popular Stories

iOS 19 Mock WWDC25 Feature

iOS 19 Expected to Run on These iPhones

Monday March 31, 2025 5:28 pm PDT by
iOS 19 will not be available on the iPhone XR, iPhone XS, or the iPhone XS Max, according a private account on social media site X that has accurately provided information on device compatibility in the past. The iPhone XR, iPhone XS, and iPhone XS Max all have an A12 Bionic chip, so it looks like iOS 19 will discontinue support for that chip. All other iPhones that run iOS 18 are expected...
maxresdefault

Apple Releases iOS 18.4 With Priority Notifications, Ambient Music, New Emoji and More

Monday March 31, 2025 10:03 am PDT by
Apple today released iOS 18.4 and iPadOS 18.4, the fourth major updates to the iOS 18 and iPadOS 18 operating system updates that came out last year. iOS 18.4 and iPadOS 18.4 come two months after Apple released iOS 18.3 and iPadOS 18.3. Subscribe to the MacRumors YouTube channel for more videos. The new software can be downloaded on eligible iPhones and iPads over-the-air by going to...
iPhone 17 Pro 34ths Perspective

iPhone 17 Pro Launching Later This Year With These 10 New Features

Sunday March 23, 2025 10:00 am PDT by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of March 2025: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
AirPods Pro Firmware Feature

Apple Releases New Firmware for AirPods Pro 2 and AirPods 4

Monday March 31, 2025 11:27 am PDT by
Apple today released new firmware updates for all AirPods 4 and AirPods Pro 2 models. The new firmware is version 7E93, up from the 7B21 firmware that was installed on the AirPods Pro 2 and the 7B20 firmware available on the AirPods 4 and AirPods 4 with ANC. It is not immediately clear what new features or changes are included in the new firmware, but we'll update this article should we find ...
watchOS 11 Thumb 2 1

Apple Releases watchOS 11.4 With Sleep Alarm Update

Tuesday April 1, 2025 10:34 am PDT by
Apple today released watchOS 11.4, the fourth major update to the operating system that runs on the Apple Watch. watchOS 11.4 is compatible with the Apple Watch Series 6 and later, all Apple Watch Ultra models, and the Apple Watch SE 2. watchOS 11.4 can be downloaded on a connected iPhone by opening up the Apple Watch app and going to General > Software Update. To install the new software,...
macOS Sequoia Feature

Apple Releases macOS Sequoia 15.4 With Mail Categorization and More

Monday March 31, 2025 10:04 am PDT by
Apple today released macOS Sequoia 15.4, the fourth major update to the macOS Sequoia operating system that launched in September. macOS Sequoia 15.4 comes two months after the launch of macOS Sequoia 15.3. Mac users can download the ‌‌macOS Sequoia‌‌ update through the Software Update section of System Settings. It is available for free on all Macs able to run macOS 15. With...
top stories 2025 03 29

Top Stories: WWDC 2025 Announced, iPhone 17 Pro and iOS 19 Rumors, and More

Saturday March 29, 2025 6:00 am PDT by
Apple's big developer event is a little over two months away, and rumors about what we can expect to see in Apple's next major operating system updates are becoming increasingly frequent. A public release of iOS 18.4 is also imminent with a number of updates and improvements, although we won't be getting the major Apple Intelligence Siri upgrades that had reportedly been planned for this...
iOS 19 visionOS UI Elements

Apple Codename Provides Clue About iOS 19's Rumored New Design

Sunday March 30, 2025 6:40 am PDT by
Multiple sources have claimed that iOS 19 will introduce a new design with more translucent buttons, menus, notification banners, and more, and there is now another clue that points towards this glass-like appearance. Bloomberg's Mark Gurman today said the new design project is codenamed "Solarium" internally. A solarium is a room with glass walls that allow in plenty of sunlight, so this...

Top Rated Comments

brofkand Avatar
10 weeks ago
Siri has been and always will be useless for anything other than simple things like setting timers. Apple has not written good software in years. The fact that their platforms are still mostly usable speaks to how far ahead they were a decade+ ago.
Score: 36 Votes (Like | Disagree)
Eriamjh1138@DAN Avatar
10 weeks ago
So Siri has become as factual as TikTok and Facebook. Got it.

Deactivated.
Score: 18 Votes (Like | Disagree)
NightfallOrchid Avatar
10 weeks ago

This isn't about Apple as such. The entire idea behind this is entirely flawed and they are being dragged into the hype and relying on praying that i'll eventually work, which it won't. Every company in the LLM space is doing the same thing. It's an arms race based on swimming in excrement with the promise of a cake at the end (the cake is a lie) and the end game is drowning.
As the original post states, ChatGPT, Kagi, DuckDuckGo and Google are all capable of giving correct answers, for some reason only Siri isn’t… Siri with ChatGPT support is somehow worse than ChatGPT on its own
Score: 14 Votes (Like | Disagree)
rivalius13 Avatar
10 weeks ago
Go Birds.
Score: 13 Votes (Like | Disagree)
MVMNT Avatar
10 weeks ago


Attachment Image
Score: 13 Votes (Like | Disagree)
kiranmk2 Avatar
10 weeks ago

This isn't a Maps level fiasco, but it's not too far off.

Remember when Apple's selling point was "they're not first, but when they do something, they do it right"? Those were the days.

We've got a confluence of factors here:

* An industry-wide fixation on what is in many ways not very good technology (and an insistence on shoehorning it in everywhere).
* Apple having a particular weakness in this particular area, dating back to well before the machine learning age and showing no signs of improvement.
* FOMO on Apple's part - fear of Android/Samsung eating their lunch if they can't say they also do this stuff.

So we've got a technology where even the best implementations are pretty bad, and Apple's implementation is worse.
Exactly this. I know it's a trope on here, but this is exactly when a Steve Jobs-like personality is really needed. Famously, he wasn't interested in pandering to Wall Street, insisting that Apple would follow it's own path and wouldn't pay dividends / buy back shares, instead, relying on a constant pipeline of amazing products to grow the company and stock price.

Under Tim Cook, Apple became much more led by its investors (look at the value of dividends / buy backs over the last 10 years) and this whole AI/LLM rush to not just catch-up, but publicly announce their plans in this area almost a year in advance (context aware Siri was announced last June, but won't be available until iOS18.4 around April/May) screams that they are messaging to investors that they are following the industry trend, rather than taking that trend and creating a new trend with it as they used to do.
Score: 12 Votes (Like | Disagree)