Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google

Piper Jaffray analyst Gene Munster, who has regularly assessed Siri's accuracy in terms of correctly interpreting and answering queries, has issued the latest version of his Siri report card, noting that Siri has continued to improve under iOS 7, particularly in terms of being able to properly interpret questions being asked.

The primary improvement between Siri in iOS 6 and iOS 7 came from its performance in noisier areas, where Siri was able to translate better through a noisy environment (94% vs. 83% correct in Aug iOS 6). Based on our testing, iOS 7 Siri's net ability to answer questions (factors in questions not correctly understood) improved to 79% from 77% in August with iOS 6.

munster_siri_dec13_report
The improvements in Siri's understanding come alongside but separate from Apple's continued efforts to reduce its reliance on Google, as evidenced by Siri's shift to emphasize Bing, Wikipedia, and WolframAlpha results. In Munster's most recent testing, only 4 percent of Siri's answers relied on Google, compared to 17 percent in August testing and 27 percent in December 2012.

munster_siri_dec13.services

Click for larger

Munster also compares Siri to Google Now, finding that the two services are now essentially tied in effectiveness with a "C+" score after Siri held a substantial lead in the year-ago testing. In the latest testing, Google Now held small leads in content areas such as commerce and information, while Siri performed significantly better for operating system controls.

munster_siri_dec13_questions

Top Rated Comments

HenryDJP Avatar
113 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.

I get 90% accuracy, I don't know what's going on with you.
Score: 13 Votes (Like | Disagree)
NightFox Avatar
113 months ago
So even though its accuracy at understanding has gone up, the accuracy of its answers has actually got worse?
Score: 12 Votes (Like | Disagree)
lil' brudder Avatar
113 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
Score: 10 Votes (Like | Disagree)
NightFox Avatar
113 months ago
The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.

Never defend a technology! Technology is meant to develop to serve the requirements/limitations of existing human behaviour; human behaviour shouldn't be expected to adapt to meet the requirements/limitations of technology.
Score: 8 Votes (Like | Disagree)
Simmias Avatar
113 months ago
I have had the opposite experience. After seeing steady improvement in Siri, it has recently gotten worse and become almost unusable for me.

Specifically, I use Siri a lot to play music in the car. I will say "Play Song X" or "Play Album Y". Siri used to be pretty good at hearing the name of the song and finding it in my music library. Now it frequently hear the names incorrectly. Even when it does hear the correct name, it will play a song with a completely different name or simply say "Sorry, I couldn't find that in your music". What do you mean you couldn't find David Bowie in my music??? It's right there!

This regression is quite infuriating. Also, Siri still often fails to respond ("Sorry, there's something wrong") or take 20+ seconds to respond with the wrong answer. I believe we need to give all of Apple's web services time to gather data and make iterative improvements, but I have very little tolerance when they actually get worse over time.
Score: 7 Votes (Like | Disagree)
chirpie Avatar
113 months ago
I don't know, that net percentage increase (4% better) year over year seems kind of low to me. Am I the only one who feels that way? Not my field of study, so I'm not entirely sure if this is supposed to be impressive.
Score: 6 Votes (Like | Disagree)

Popular Stories

iPhone 14 Pro Purple Rear Flat MacRumors Exclusive

iPhone 14 Pro Predicted to Start With Increased 256GB Storage Alongside Rumored Price Increase

Wednesday August 10, 2022 11:14 am PDT by
Earlier today, analyst Ming-Chi Kuo claimed iPhone 14 Pro models will be more expensive than iPhone 13 Pro models. Kuo did not reveal exact pricing, but he said that the average selling price of all four iPhone 14 models will increase by about 15% overall. While higher prices would be disappointing for customers, it is possible the iPhone 14 Pro and iPhone 14 Pro Max will offer increased...
iPhone 14 Pro Lineup Feature Silver

Kuo: Apple to Increase Prices of iPhone 14 Pro Models

Wednesday August 10, 2022 8:22 am PDT by
Apple plans to increase the prices of iPhone 14 Pro models compared to iPhone 13 Pro models, according to analyst Ming-Chi Kuo. Kuo did not reveal exact pricing for the iPhone 14 Pro models. However, in a tweet today, he estimated that the average selling price of the iPhone 14 lineup as a whole will increase by about 15% compared to the iPhone 13 lineup. In the United States, the iPhone...
iPhone 14 Pro Purple Front and Back MacRumors Exclusive feature

iPhone 14 Is Just a Few Weeks Away: Three Tips to Prepare for the New iPhone

Wednesday August 10, 2022 4:08 am PDT by
The launch of the new iPhone 14 is just a few weeks away, meaning millions of iPhone customers will soon upgrade their existing iPhone or perhaps get an iPhone for the first time. Exclusive MacRumors iPhone 14 Pro renders by graphic designer Ian Zelbo Whether upgrading from an older model or this is your first iPhone, we've rounded up a few tips to help you prepare for the next flagship...
battery percentage ios 16

Here's Why the iPhone Battery Status Icon in iOS 16 Is So Controversial

Wednesday August 10, 2022 4:34 am PDT by
In the latest iOS 16 beta, Apple has updated the status bar battery icon on iPhones with Face ID to display the exact percentage remaining rather than just a visual representation of battery level, and while the change has been largely welcomed, some users are unhappy with the way it has been implemented. In iOS 15 and earlier, battery percent has not been present on iPhones that have...
Apple Watch Body Temperature Finished

'High-Accuracy' Apple Watch Temperature Sensor Revealed by Patent Filing Just Weeks Before Series 8 Unveiling

Wednesday August 10, 2022 5:39 am PDT by
Apple has been granted a patent for a temperature sensor suitable for the Apple Watch, just weeks before the company is expected to unveil the Apple Watch Series 8 with body temperature sensing capabilities. The newly granted patent, spotted by MyHealthyApple, was filed with the United States Patent and Trademark Office and is titled "Temperature gradient sensing in electronic devices."...