Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

‌Siri‌ continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both ‌Siri‌ and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
20 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
20 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
20 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
20 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
20 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
20 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Top Stories

iphone 6 in hand

Apple Faces Another iPhone Lawsuit Over 'Programmed Obsolescence'

Monday March 1, 2021 6:44 am PST by
Apple faces a new class-action lawsuit that accuses it of deliberately releasing iOS updates that slowly reduce the performance of an iPhone, forcing customers to upgrade their devices. The lawsuit comes from the Portuguese Consumer Protection Agency, Deco Proteste (via Marketeer), which in a statement says that it will proceed with a case against the Cupertino tech giant because it...
PIA23764 RoverNamePlateonMars web

NASA Mars Perseverance Rover Uses Same PowerPC Chipset Found in 1998 G3 iMac

Tuesday March 2, 2021 3:21 am PST by
NASA's Perseverance rover, which recently made history landing on the surface of Mars, is powered by the same processor used in an iMac more than 23 years old. Image Credit: NASA As reported by NewScientist (via Gizmodo), the rover includes the PowerPC 750 processor, the same chip used in the G3 iMac in 1998. The main chipset is the same; however, there are differences between the...
magsafe charging brick feature

Prosser: iPhone 12 MagSafe Battery Pack to Feature Reverse Charging

Monday March 1, 2021 1:18 pm PST by
Apple is working on MagSafe-compatible battery packs that will be compatible with the iPhone 12 models, and leaker Jon Prosser today said on the Genius Bar podcast that one version of the battery pack will feature "reverse charging." According to Prosser, Apple is working on two versions of the battery packs, one that's a standard version and one that's a premium version with reverse...
rosetta 2

Rosetta May Be Removed From M1 Macs in Some Regions on macOS 11.3

Tuesday March 2, 2021 5:20 pm PST by
Installing the upcoming macOS 11.3 software update on an M1 Mac may result in Rosetta 2 being removed in one or more regions around the world. In the third beta of macOS 11.3 seeded to developers for testing today, MacRumors contributor Steve Moser uncovered new strings in the beta's code indicating that "Rosetta will be removed upon installing this update." Another new string reads "Rosetta ...
iphone 12 pro display video

iPhone 13 to Include 1TB Storage Option and LiDAR Across the Board, Says Wedbush Analyst

Monday March 1, 2021 4:00 am PST by
Apple's forthcoming iPhone 13 could include a 1TB storage option for some models and LiDAR Scanners across the entire lineup, according to a report from Wedbush analysts. In a new note to investors, seen by MacRumors, Wedbush analyst Daniel Ives said that initial Asian supply chain checks gave the firm "increased confidence" that Apple's 5G-driven product cycle would extend well into 2022,...
Apple Prefer Lightning Over USB C Feature

iPhone Sticking With Lightning Port Over USB-C for 'Foreseeable Future'

Tuesday March 2, 2021 9:32 am PST by
Apple will retain the Lightning connector on the iPhone for the "foreseeable future," with no intention of switching to USB-C, according to reliable analyst Ming-Chi Kuo. In spite of much of the industry moving toward USB-C, Apple will not be using it to replace the Lightning connector on the iPhone 13, or indeed on any iPhone model for the time being. In a note seen by MacRumors yesterday,...
iphone 12 120hz thumbnail feature

Kuo: iPhone 13 Lineup to Feature Smaller Notch and Larger Batteries, 120Hz Display for Pro Models, and More

Monday March 1, 2021 7:50 am PST by
iPhone 13 models will all feature a smaller notch, while the two Pro models will be equipped with low-power LTPO display technology for a 120Hz refresh rate, analyst Ming-Chi Kuo said today in a research note obtained by MacRumors. Subscribe to the MacRumors YouTube channel for more videos. Several other sources have previously claimed that some iPhone 13 models will support a 120Hz refresh...
Screen Shot 2021 03 03 at 11

MagSafe Charging Port for iPhone Appears in Apple Patent

Wednesday March 3, 2021 1:30 am PST by
In a newly granted patent, Apple envisions a type of connectivity port using an iteration of its magnetic MagSafe charger to charge an iPhone, potentially paving the way for a future without Lightning. Submitted to the United States Patent and Trademark Office on Tuesday (via Patently Apple), the patent outlines a three-pinned MagSafe charger, similar to the ones found in older MacBook...
apple transfer google photos 1

Apple Launches Service for Transferring iCloud Photos and Videos to Google Photos

Wednesday March 3, 2021 12:04 pm PST by
Apple this week introduced a new service that's designed to make it quick and easy for iCloud users to transfer their stored photos and videos to Google Photos. As outlined in an Apple support document, you can go to Apple's privacy website and sign in to see the "Transfer a copy of your data" option. If you select this and go through all the steps, Apple will transfer your iCloud photos and ...
galaxy s21 iphone 12 pro max front feature2

Kuo: Some 2022 iPhones to Abandon Notch in Favor of 'Punch-Hole' Display

Monday March 1, 2021 8:05 am PST by
At least some 2022 iPhone models will abandon the notch and switch to a "punch-hole display design" instead, similar to Samsung's recent high-end Galaxy smartphones, analyst Ming-Chi Kuo said today in a research note obtained by MacRumors. Kuo said the hole-punch display design will be adopted by high-end 2022 iPhones at a minimum, and if production yields are good enough, all 2022 iPhone...