Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google

Piper Jaffray analyst Gene Munster, who has regularly assessed Siri's accuracy in terms of correctly interpreting and answering queries, has issued the latest version of his Siri report card, noting that Siri has continued to improve under iOS 7, particularly in terms of being able to properly interpret questions being asked.

The primary improvement between Siri in iOS 6 and iOS 7 came from its performance in noisier areas, where Siri was able to translate better through a noisy environment (94% vs. 83% correct in Aug iOS 6). Based on our testing, iOS 7 Siri's net ability to answer questions (factors in questions not correctly understood) improved to 79% from 77% in August with iOS 6.

munster_siri_dec13_report
The improvements in Siri's understanding come alongside but separate from Apple's continued efforts to reduce its reliance on Google, as evidenced by Siri's shift to emphasize Bing, Wikipedia, and WolframAlpha results. In Munster's most recent testing, only 4 percent of Siri's answers relied on Google, compared to 17 percent in August testing and 27 percent in December 2012.

munster_siri_dec13.services

Click for larger

Munster also compares Siri to Google Now, finding that the two services are now essentially tied in effectiveness with a "C+" score after Siri held a substantial lead in the year-ago testing. In the latest testing, Google Now held small leads in content areas such as commerce and information, while Siri performed significantly better for operating system controls.

munster_siri_dec13_questions

Top Rated Comments

HenryDJP Avatar
135 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.

I get 90% accuracy, I don't know what's going on with you.
Score: 13 Votes (Like | Disagree)
NightFox Avatar
135 months ago
So even though its accuracy at understanding has gone up, the accuracy of its answers has actually got worse?
Score: 12 Votes (Like | Disagree)
lil' brudder Avatar
135 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
Score: 10 Votes (Like | Disagree)
NightFox Avatar
135 months ago
The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.

Never defend a technology! Technology is meant to develop to serve the requirements/limitations of existing human behaviour; human behaviour shouldn't be expected to adapt to meet the requirements/limitations of technology.
Score: 8 Votes (Like | Disagree)
Simmias Avatar
135 months ago
I have had the opposite experience. After seeing steady improvement in Siri, it has recently gotten worse and become almost unusable for me.

Specifically, I use Siri a lot to play music in the car. I will say "Play Song X" or "Play Album Y". Siri used to be pretty good at hearing the name of the song and finding it in my music library. Now it frequently hear the names incorrectly. Even when it does hear the correct name, it will play a song with a completely different name or simply say "Sorry, I couldn't find that in your music". What do you mean you couldn't find David Bowie in my music??? It's right there!

This regression is quite infuriating. Also, Siri still often fails to respond ("Sorry, there's something wrong") or take 20+ seconds to respond with the wrong answer. I believe we need to give all of Apple's web services time to gather data and make iterative improvements, but I have very little tolerance when they actually get worse over time.
Score: 7 Votes (Like | Disagree)
chirpie Avatar
135 months ago
I don't know, that net percentage increase (4% better) year over year seems kind of low to me. Am I the only one who feels that way? Not my field of study, so I'm not entirely sure if this is supposed to be impressive.
Score: 6 Votes (Like | Disagree)

Popular Stories

iOS 18 Siri Integrated Feature

iOS 18 Will Add These New Features to Your iPhone

Friday April 12, 2024 11:11 am PDT by
iOS 18 is expected to be the "biggest" update in the iPhone's history. Below, we recap rumored features and changes for the iPhone. iOS 18 is rumored to include new generative AI features for Siri and many apps, and Apple plans to add RCS support to the Messages app for an improved texting experience between iPhones and Android devices. The update is also expected to introduce a more...
iPhone 16 Camera Lozenge 2 Colors

iPhone 16 Plus Rumored to Come in These 7 Colors

Wednesday April 10, 2024 3:52 am PDT by
Apple's iPhone 16 Plus may come in seven colors that either build upon the existing five colors in the standard iPhone 15 lineup or recast them in a new finish, based on a new rumor out of China. According to the Weibo-based leaker Fixed focus digital, Apple's upcoming larger 6.7-inch iPhone 16 Plus model will come in the following colors, compared to the colors currently available for the...
apple silicon feature joeblue

Macs to Get AI-Focused M4 Chips Starting in Late 2024

Thursday April 11, 2024 10:10 am PDT by
Apple will begin updating its Mac lineup with M4 chips in late 2024, according to Bloomberg's Mark Gurman. The M4 chip will be focused on improving performance for artificial intelligence capabilities. Last year, Apple introduced the M3, M3 Pro, and M3 Max chips all at once in October, so it's possible we could see the M4 lineup come during the same time frame. Gurman says that the entire...
top stories 13apr2024

Top Stories: M4 Mac Roadmap Leaked, New iPads in Second Week of May, and More

Saturday April 13, 2024 6:00 am PDT by
Apple's hardware roadmap was in the news this week, with things hopefully firming up for a launch of updated iPad Pro and iPad Air models next month while we look ahead to the other iPad models and a full lineup of M4-based Macs arriving starting later this year. We also heard some fresh rumors about iOS 18, due to be unveiled at WWDC in a couple of months, while we took a look at how things ...
new best buy blue

Best Buy Opens Up Sitewide Sale With Record Low Prices on M3 MacBook Air, iPad, and Much More

Saturday April 13, 2024 7:41 am PDT by
Best Buy this weekend has a big sale on Apple MacBooks and iPads, including new all-time low prices on the M3 MacBook Air, alongside the best prices we've ever seen on MacBook Pro, iPad, and more. Some of these deals require a My Best Buy Plus or My Best Buy Total membership, which start at $49.99/year. In addition to exclusive access to select discounts, you'll get free 2-day shipping, an...