Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

‌Siri‌ continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both ‌Siri‌ and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
23 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
23 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
23 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
23 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
23 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
23 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Top Stories

tile sticker e1570533758981

Tile CEO: 'We Welcome Competition From Apple, But We Think It Needs to Be Fair'

Tuesday May 4, 2021 9:51 am PDT by
Just after Apple announced its AirTags, Tile CEO CJ Prober relayed his concerns about competing with Apple in the tracking space, and said that Tile would ask Congress to investigate Apple's business practices specific to Find My and item trackers. Prober this week did an interview with Bloomberg, where he further expanded on Tile's complaints about Apple and why he feels that Tile is...
apple watch ecg

Apple Watch Likely to Gain Blood Pressure, Blood Glucose, and Blood Alcohol Monitoring

Monday May 3, 2021 4:03 am PDT by
The Apple Watch may gain the ability to measure blood pressure, blood glucose, and blood alcohol levels, according to newly-revealed information about one of Apple's chosen business partners. Apple has been revealed to be the largest customer of the British electronics start-up Rockley Photonics, The Telegraph reports. Rockley Photonics has developed non-invasive optical sensors for...
signal instagram ads3

Signal Shares the Instagram Ads Facebook Doesn't Want You to See

Wednesday May 5, 2021 1:29 am PDT by
Encrypted messaging app Signal has had a series of Instagram ads blocked from the social media platform, after it attempted to show users how much data the Facebook-owned company collects about them and how it's used to push targeted ads. In a blog post, Signal described how it generated the ads to show users why they were seeing them, simply by declaring upfront the information that the...
fortnite apple logo 2

Epic CEO Tim Sweeney Admits App Store's 30% Cut Is Similar to Consoles, Would Have Accepted Special Deal With Apple

Tuesday May 4, 2021 1:54 pm PDT by
Apple's legal battle with Epic Games is continuing on, and during the second day of the trial, Epic Games' CEO Tim Sweeney continued his testimony against Apple. Sweeney was grilled by Apple's lawyers, and made several points seemingly favorable to Apple. In addition to mentioning how he prefers Apple's iPhone and values Apple's privacy policies that he's aiming to dismantle, Sweeney...
snapchat dark mode

Snapchat Rolls Out Dark Mode on iOS

Wednesday May 5, 2021 1:17 am PDT by
Nearly two years following the release of iOS and iPadOS 13, which included native, built-in, and systemwide dark mode, Snapchat, one of the world's most prominent social media networks, has finally rolled out a dark mode theme for iOS users. Snapchat began testing a dark mode theme of its app design late last year with a small group of iOS users. Now, Snapchat says that as of this week, it...
maxresdefault

Hands-On With Brydge's 12.9-Inch iPad Pro Keyboard With Trackpad

Tuesday May 4, 2021 11:48 am PDT by
Brydge has been making keyboards for Apple's iPads for years now, and the newest model, the Brydge 12.9 MAX+, is compatible with the third, fourth, and fifth-generation iPad Pro models, so it works even with the new mini-LED iPad Pro. Subscribe to the MacRumors YouTube channel for more videos. In our latest YouTube video, we checked out the new Brydge 12.9 MAX+ to see if it's a viable...
iphone 13 pro max dummy notch

iPhone 13 Pro Max Dummy Model Depicts Smaller Notch

Tuesday May 4, 2021 1:08 pm PDT by
Apple's iPhone 13 models are expected to feature a slimmed down notch, marking the first major change to the TrueDepth camera system since it was introduced in the 2017 iPhone X. We're still months away from the launch of the iPhone 13, but Lewis Hilsenteger of Unbox Therapy managed to get an iPhone 13 Pro Max dummy model that represents what we can expect from the new 2021 device. Dummy...
Flat 2021 MacBook Pro Mockup Feature 1

Mini-LED Display Production Improving for Redesigned MacBook Pro Models Later This Year

Monday May 3, 2021 8:33 am PDT by
Apple supplier TSMT, a key vendor involved in the production of mini-LED displays in the newly announced 12.9-inch iPad Pro, has been able to address technical challenges for the production of mini-LED displays to be used in the upcoming 14 and 16-inch redesigned MacBook Pro models. As reported by DigiTimes, TSMT had initially been facing production constraints with the circuit board and...
netflix sign up

Apple Discussed 'Punitive Measures' Against Netflix for Dropping In-App Purchases

Wednesday May 5, 2021 11:03 am PDT by
As the Epic Games v. Apple trial progresses into its third day, Apple's internal documents and communications with various companies are continuing to surface, giving us some insight into the dealings that Apple has had around the App Store. Back in December 2018, Netflix stopped offering in-app subscription options for new or resubscribing members and instead began requiring them to sign up ...
airtags

AirTag Owners Bemoan Inability to Let Others Track Their Items Via Family Sharing

Tuesday May 4, 2021 5:12 am PDT by
Apple's new AirTag item trackers have been making their way into the hands of customers since Friday, and while the company has tried to describe the ways that they can be used to find lost items, many users are still surprised and disappointed to learn that the location of an AirTag can't be shared with other family members. On the face of it, sharing the location of an AirTag via Apple's...