Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

Siri continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both Siri and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
29 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
29 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
29 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
29 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
29 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
29 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Related Stories

studio buds family

Beats Studio Buds Debuting Today With Active Noise Cancellation, Stemless Design, and More for $150

Monday June 14, 2021 8:00 am PDT by
We've seen a lot of teasers about the Beats Studio Buds over the past month since they first showed up in Apple's beta software updates, and today they're finally official. The Beats Studio Buds are available to order today in red, white, and black ahead of a June 24 ship date, and they're priced at $149.99. The Studio Buds are the first Beats-branded earbuds to truly compete with AirPods...
airtag in hand

Apple Enhancing AirTags Anti-Stalking Measures With Android App and Shorter Sound Intervals

Thursday June 3, 2021 11:10 am PDT by
Apple is enhancing AirTags security to prevent stalking using the Bluetooth devices, Apple told CNET today. Apple is already sending out over-the-air updates to AirTags that will shorten the amount of time before an unknown AirTag alerts you if it is in your possession. At the current time, AirTags play a sound after three days of being away from their owner. After the update, AirTags will...
maxresdefault

Here's How Apple's New iPhone to iPhone Data Migration Feature Works in iOS 12.4

Tuesday July 23, 2019 1:20 pm PDT by
Apple this week released iOS 12.4, the newest version of iOS 12 available for iPhones and iPads. One of the new features in iOS 12.4 is an updated data migration option that uses device to device transfers rather than relying on iCloud. Apple didn't provide much information on the new data migration feature, so we thought we'd check it out in our latest YouTube video. Subscribe to the ...
youtube apple tv

YouTube Discontinuing 3rd-Generation Apple TV App, AirPlay Still Available

Wednesday February 3, 2021 3:09 pm PST by
YouTube is planning to stop supporting its YouTube app on the third-generation Apple TV models, where YouTube has long been available as a channel option. A 9to5Mac reader received a message about the upcoming app discontinuation, which is set to take place in March.Starting early March, the YouTube app will no longer be available on Apple TV (3rd generation). You can still watch YouTube on...
macos catalina legacy system extension alert

Apple Begins Warning Users That 'Legacy System Extensions' Won't Work With a Future Version of macOS

Wednesday March 25, 2020 9:53 am PDT by
Apple has shared a new support document that indicates kernel extensions — which it calls "legacy system extensions" — will not be compatible with a future version of macOS because they "aren't as secure or reliable as modern alternatives."System extensions are a category of software that works in the background to extend the functionality of your Mac. Some apps install kernel extensions, which...
os x mountain lion macs 16x9 2

Apple Makes OS X Lion and Mountain Lion Free to Download

Wednesday June 30, 2021 12:19 pm PDT by
Apple recently dropped the $19.99 fee for OS X Lion and Mountain Lion, making the older Mac updates free to download, reports Macworld. Apple has kept OS X 10.7 Lion and OS X 10.8 Mountain Lion available for customers who have machines limited to the older software, but until recently, Apple was charging $19.99 to get download codes for the updates. As of last week, these updates no...
airtag precision finding

AirTag Includes U1 Chip for 'Precision Finding' Feature

Tuesday April 20, 2021 12:11 pm PDT by
Apple's long-awaited AirTag was finally unveiled today, and as expected, the small circle-shaped accessories can be attached to items like wallets, keys, and more to allow them to be tracked in the Find My app. As was rumored ahead of release, each AirTag is equipped with a U1 chip, and on devices that also have U1 chips, there's a Precision Finding feature. U1 Ultra Wideband chips are...
iOS 15 General Feature Yellow

Everything New in iOS 15 Beta 6: SharePlay Disabled, Safari Redesigned and More

Tuesday August 17, 2021 2:12 pm PDT by
Apple released the sixth beta of iOS 15 just a week after the fifth beta, but the new update brings some of the most significant tweaks that we've seen to iOS 15 during the beta testing period. Safari Redesign Apple in iOS 15 beta 6 has added a toggle to move the Safari address bar to the top of the interface, which returns Safari to an iOS 14-like design and mitigates all of the Safari...
m1 macbook air

Kuo: Mini-LED MacBook Air Coming in Mid-2022

Thursday July 22, 2021 7:48 pm PDT by
Apple will release a new version of the MacBook Air around the middle of 2022, Apple analyst Ming-Chi Kuo said today in note to investors seen by MacRumors. The upcoming MacBook Air will feature a 13.3-inch mini-LED display, which would make it the second Mac to gain mini-LED technology after the 2021 MacBook Pro, which is rumored to include a mini-LED display and is expected to launch later ...
2016 macbook pro flexgate b

Apple Faces Another Class-Action MacBook Pro ‘Flexgate’ Lawsuit

Thursday August 20, 2020 6:43 am PDT by
Another class-action complaint has been lodged against Apple, which claims that the company was aware of a MacBook Pro design flaw that caused some devices to have backlight display issues (via Apple Insider). The so-called "flexgate" problem was present in some MacBook Pro models manufactured between 2016 and 2017. The problem appears as dark patches along the bottom of the MacBook Pro's...