Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google

Piper Jaffray analyst Gene Munster, who has regularly assessed Siri's accuracy in terms of correctly interpreting and answering queries, has issued the latest version of his Siri report card, noting that Siri has continued to improve under iOS 7, particularly in terms of being able to properly interpret questions being asked.

The primary improvement between Siri in iOS 6 and iOS 7 came from its performance in noisier areas, where Siri was able to translate better through a noisy environment (94% vs. 83% correct in Aug iOS 6). Based on our testing, iOS 7 Siri's net ability to answer questions (factors in questions not correctly understood) improved to 79% from 77% in August with iOS 6.

munster_siri_dec13_report
The improvements in Siri's understanding come alongside but separate from Apple's continued efforts to reduce its reliance on Google, as evidenced by Siri's shift to emphasize Bing, Wikipedia, and WolframAlpha results. In Munster's most recent testing, only 4 percent of Siri's answers relied on Google, compared to 17 percent in August testing and 27 percent in December 2012.

munster_siri_dec13.services

Click for larger

Munster also compares Siri to Google Now, finding that the two services are now essentially tied in effectiveness with a "C+" score after Siri held a substantial lead in the year-ago testing. In the latest testing, Google Now held small leads in content areas such as commerce and information, while Siri performed significantly better for operating system controls.

munster_siri_dec13_questions

Popular Stories

iPhone SE 4 Vertical Camera Feature

iPhone SE 4 Production Will Reportedly Begin Ramping Up in October

Tuesday July 23, 2024 2:00 pm PDT by
Following nearly two years of rumors about a fourth-generation iPhone SE, The Information today reported that Apple suppliers are finally planning to begin ramping up mass production of the device in October of this year. If accurate, that timeframe would mean that the next iPhone SE would not be announced alongside the iPhone 16 series in September, as expected. Instead, the report...
iPhone 17 Plus Feature

iPhone 17 Lineup Specs Detail Display Upgrade and New High-End Model

Monday July 22, 2024 4:33 am PDT by
Key details about the overall specifications of the iPhone 17 lineup have been shared by the leaker known as "Ice Universe," clarifying several important aspects of next year's devices. Reports in recent months have converged in agreement that Apple will discontinue the "Plus" iPhone model in 2025 while introducing an all-new iPhone 17 "Slim" model as an even more high-end option sitting...
Generic iPhone 17 Feature With Full Width Dynamic Island

Kuo: Ultra-Thin iPhone 17 to Feature A19 Chip, Single Rear Camera, Semi-Titanium Frame, and More

Wednesday July 24, 2024 9:06 am PDT by
Apple supply chain analyst Ming-Chi Kuo today shared alleged specifications for a new ultra-thin iPhone 17 model rumored to launch next year. Kuo expects the device to be equipped with a 6.6-inch display with a current-size Dynamic Island, a standard A19 chip rather than an A19 Pro chip, a single rear camera, and an Apple-designed 5G chip. He also expects the device to have a...
iPhone 16 Pro Sizes Feature

iPhone 16 Series Is Less Than Two Months Away: Everything We Know

Thursday July 25, 2024 5:43 am PDT by
Apple typically releases its new iPhone series around mid-September, which means we are about two months out from the launch of the iPhone 16. Like the iPhone 15 series, this year's lineup is expected to stick with four models – iPhone 16, iPhone 16 Plus, iPhone 16 Pro, and iPhone 16 Pro Max – although there are plenty of design differences and new features to take into account. To bring ...
icloud private relay outage

iCloud Private Relay Experiencing Outage

Thursday July 25, 2024 3:18 pm PDT by
Apple’s iCloud Private Relay service is down for some users, according to Apple’s System Status page. Apple says that the iCloud Private Relay service may be slow or unavailable. The outage started at 2:34 p.m. Eastern Time, but it does not appear to be affecting all iCloud users. Some impacted users are unable to browse the web without turning iCloud Private Relay off, while others are...
iPhone 17 Plus Feature Purple

iPhone 17 Rumored to Feature Mechanical Aperture

Tuesday July 23, 2024 9:32 am PDT by
Apple is planning to release at least one iPhone 17 model next year with mechanical aperture, according to a report published today by The Information. The mechanical system would allow users to adjust the size of the iPhone 17's aperture, which refers to the opening of the camera lens through which light enters. All existing iPhone camera lenses have fixed apertures, but some Android...

Top Rated Comments

HenryDJP Avatar
139 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.

I get 90% accuracy, I don't know what's going on with you.
Score: 13 Votes (Like | Disagree)
NightFox Avatar
139 months ago
So even though its accuracy at understanding has gone up, the accuracy of its answers has actually got worse?
Score: 12 Votes (Like | Disagree)
lil' brudder Avatar
139 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
Score: 10 Votes (Like | Disagree)
NightFox Avatar
139 months ago
The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.

Never defend a technology! Technology is meant to develop to serve the requirements/limitations of existing human behaviour; human behaviour shouldn't be expected to adapt to meet the requirements/limitations of technology.
Score: 8 Votes (Like | Disagree)
Simmias Avatar
139 months ago
I have had the opposite experience. After seeing steady improvement in Siri, it has recently gotten worse and become almost unusable for me.

Specifically, I use Siri a lot to play music in the car. I will say "Play Song X" or "Play Album Y". Siri used to be pretty good at hearing the name of the song and finding it in my music library. Now it frequently hear the names incorrectly. Even when it does hear the correct name, it will play a song with a completely different name or simply say "Sorry, I couldn't find that in your music". What do you mean you couldn't find David Bowie in my music??? It's right there!

This regression is quite infuriating. Also, Siri still often fails to respond ("Sorry, there's something wrong") or take 20+ seconds to respond with the wrong answer. I believe we need to give all of Apple's web services time to gather data and make iterative improvements, but I have very little tolerance when they actually get worse over time.
Score: 7 Votes (Like | Disagree)
chirpie Avatar
139 months ago
I don't know, that net percentage increase (4% better) year over year seems kind of low to me. Am I the only one who feels that way? Not my field of study, so I'm not entirely sure if this is supposed to be impressive.
Score: 6 Votes (Like | Disagree)