Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

Siri continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both Siri and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
61 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
61 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
61 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
61 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
61 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
61 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Popular Stories

iPhone 15 Pro FineWoven

Apple Reportedly Stops Production of FineWoven Accessories

Sunday April 21, 2024 6:03 am PDT by
Apple has stopped production of FineWoven accessories, according to the Apple leaker and prototype collector known as "Kosutami." In a post on X (formerly Twitter), Kosutami explained that Apple has stopped production of FineWoven accessories due to its poor durability. The company may move to another non-leather material for its premium accessories in the future. Kosutami has revealed...
Provenance Emulator

PlayStation and SEGA Emulator for iPhone and Apple TV Coming to App Store [Updated]

Friday April 19, 2024 8:29 am PDT by
The lead developer of the multi-emulator app Provenance has told iMore that his team is working towards releasing the app on the App Store, but he did not provide a timeframe. Provenance is a frontend for many existing emulators, and it would allow iPhone and Apple TV users to emulate games released for a wide variety of classic game consoles, including the original PlayStation, GameCube, Wii,...
iOS 17 All New Features Thumb

iOS 17.5 Will Add These New Features to Your iPhone

Sunday April 21, 2024 3:00 am PDT by
The upcoming iOS 17.5 update for the iPhone includes only a few new user-facing features, but hidden code changes reveal some additional possibilities. Below, we have recapped everything new in the iOS 17.5 and iPadOS 17.5 beta so far. Web Distribution Starting with the second beta of iOS 17.5, eligible developers are able to distribute their iOS apps to iPhone users located in the EU...
apple vision pro orange

Apple Vision Pro Customer Interest Dying Down at Some Retail Stores

Monday April 22, 2024 2:12 am PDT by
Apple Vision Pro, Apple's $3,500 spatial computing device, appears to be following a pattern familiar to the AR/VR headset industry – initial enthusiasm giving way to a significant dip in sustained interest and usage. Since its debut in the U.S. in February 2024, excitement for the Apple Vision Pro has noticeably cooled, according to Bloomberg's Mark Gurman. Writing in his latest Power On...
top stories 20apr2024

Top Stories: Nintendo Emulators on App Store, Two New iOS 17 Features, and More

Saturday April 20, 2024 6:00 am PDT by
It was a big week for retro gaming fans, as iPhone users are starting to reap the rewards of Apple's recent change to allow retro game emulators on the App Store. This week also saw a new iOS 17.5 beta that will support web-based app distribution in the EU, the debut of the first hotels to allow for direct AirPlay streaming to room TVs, a fresh rumor about the impending iPad Air update, and...