Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google

Piper Jaffray analyst Gene Munster, who has regularly assessed Siri's accuracy in terms of correctly interpreting and answering queries, has issued the latest version of his Siri report card, noting that Siri has continued to improve under iOS 7, particularly in terms of being able to properly interpret questions being asked.

The primary improvement between Siri in iOS 6 and iOS 7 came from its performance in noisier areas, where Siri was able to translate better through a noisy environment (94% vs. 83% correct in Aug iOS 6). Based on our testing, iOS 7 Siri's net ability to answer questions (factors in questions not correctly understood) improved to 79% from 77% in August with iOS 6.

munster_siri_dec13_report
The improvements in Siri's understanding come alongside but separate from Apple's continued efforts to reduce its reliance on Google, as evidenced by Siri's shift to emphasize Bing, Wikipedia, and WolframAlpha results. In Munster's most recent testing, only 4 percent of Siri's answers relied on Google, compared to 17 percent in August testing and 27 percent in December 2012.

munster_siri_dec13.services

Click for larger

Munster also compares Siri to Google Now, finding that the two services are now essentially tied in effectiveness with a "C+" score after Siri held a substantial lead in the year-ago testing. In the latest testing, Google Now held small leads in content areas such as commerce and information, while Siri performed significantly better for operating system controls.

munster_siri_dec13_questions

Top Rated Comments

(View all)
Avatar
88 months ago

There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.


I get 90% accuracy, I don't know what's going on with you.
Score: 13 Votes (Like | Disagree)
Avatar
88 months ago
So even though its accuracy at understanding has gone up, the accuracy of its answers has actually got worse?
Score: 12 Votes (Like | Disagree)
Avatar
88 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
Score: 10 Votes (Like | Disagree)
Avatar
88 months ago

The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.


Never defend a technology! Technology is meant to develop to serve the requirements/limitations of existing human behaviour; human behaviour shouldn't be expected to adapt to meet the requirements/limitations of technology.
Score: 8 Votes (Like | Disagree)
Avatar
88 months ago
I have had the opposite experience. After seeing steady improvement in Siri, it has recently gotten worse and become almost unusable for me.

Specifically, I use Siri a lot to play music in the car. I will say "Play Song X" or "Play Album Y". Siri used to be pretty good at hearing the name of the song and finding it in my music library. Now it frequently hear the names incorrectly. Even when it does hear the correct name, it will play a song with a completely different name or simply say "Sorry, I couldn't find that in your music". What do you mean you couldn't find David Bowie in my music??? It's right there!

This regression is quite infuriating. Also, Siri still often fails to respond ("Sorry, there's something wrong") or take 20+ seconds to respond with the wrong answer. I believe we need to give all of Apple's web services time to gather data and make iterative improvements, but I have very little tolerance when they actually get worse over time.
Score: 7 Votes (Like | Disagree)
Avatar
88 months ago
I don't know, that net percentage increase (4% better) year over year seems kind of low to me. Am I the only one who feels that way? Not my field of study, so I'm not entirely sure if this is supposed to be impressive.
Score: 6 Votes (Like | Disagree)

Top Stories

Here's How You Can Download iOS 14 and iPadOS 14 Around the World [It's Out]

Wednesday September 16, 2020 2:36 am PDT by
Apple's official public release of iOS 14 and iPadOS 14 dropped on Wednesday, September 16, just a day after the company released the Golden Master to third-party developers. Also set to be made available to the general public for the first time are watchOS 7 and tvOS 14. Getting Started With iOS 14 Video Click image to watch iOS 14 Getting Started While that's left a lot of developers...

When Will the iPhone 12 Launch? Here's What We Know

Wednesday September 16, 2020 6:12 am PDT by
Yesterday's "Time Flies" Apple event saw the release of the Apple Watch Series 6, Apple Watch SE, iPad 8, and iPad Air 4, but no new iPhone models. Rumors before the event strongly alleged that it would not see the unveiling of new iPhones, with many reports pointing to an October launch. The lack of new iPhone models yesterday seems to confirm that the iPhone 12 lineup will not appear...

Apple Releases iOS 14 and iPadOS 14 With Home Screen Redesign, App Library, Compact UI, Translate App, Scribble Support, App Clips, and More

Wednesday September 16, 2020 12:48 pm PDT by
Apple has released iOS 14 and iPadOS 14, the newest operating system updates designed for the iPhone and iPad. As with all of Apple's software updates, iOS 14 and iPadOS 14 can be downloaded for free. iOS 14 is available on the iPhone 6s and later, while iPadOS 14 is available on the iPad Air 2 and later. The updates are available on all eligible devices over-the-air in the Settings app. To ...

Apple Updates AirPods 2 and AirPods Pro Firmware to Version 3A283

Monday September 14, 2020 11:24 am PDT by
Apple today released new 3A283 firmware updates for the second-generation AirPods and the AirPods Pro. The second-generation AirPods are being updated from the 2D15 firmware they were previously running, while the AirPods Pros are being updated from the 2D27 firmware they had installed previously. Apple does not provide details on what's included in refreshed firmware so we don't know what's ...

Spotify Says Apple One Bundle Will Cause 'Irreparable Harm to Developer Community'

Tuesday September 15, 2020 12:26 pm PDT by
Apple today announced Apple One, a series of new subscription bundles that provide access to various Apple services at a combined monthly price. In response, Spotify sent out statements (via Peter Kafka) to the press decrying Apple's anti-competitive behavior and calling on "competition authorities" to stop Apple before it is able to cause "irreparable harm" to developers.Once again, Apple...

iOS 14, iPadOS 14, watchOS 7, and tvOS 14 Will Be Released September 16

Tuesday September 15, 2020 11:06 am PDT by
Apple today announced that iOS 14, iPadOS 14, watchOS 7, and tvOS 14 will be officially released on Wednesday, September 16. Apple has seeded the final Golden Master beta versions of each update to developers in advance. ‌iOS 14‌ and iPadOS 14 introduce dozens of new features, including a redesigned Home Screen that supports widgets on the iPhone and iPad for the first time. For more...

New AirPods Pro Firmware Introduces Spatial Audio Support and Automatic Switching

Monday September 14, 2020 12:22 pm PDT by
The new 3A283 firmware that Apple released for the AirPods Pro today appears to introduce support for Spatial Audio, a new feature coming to the higher-end AirPods with iOS 14. Multiple reports on Twitter and the MacRumors forums indicate that the firmware update adds a "Spatial Audio" option to the Control Center, which can be activated to enable the feature. Note that using Spatial Audio...

Apple Releases Safari 14 for Mac Ahead of macOS Big Sur Launch

Wednesday September 16, 2020 1:40 pm PDT by
macOS Big Sur didn't launch alongside iOS 14, iPadOS 14, tvOS 14, and watchOS 7 today, with the update coming later this fall, but Apple did release the Safari 14 update for macOS Catalina and macOS Mojave users. Safari 14 brings improved performance, customizable start pages, a Privacy Report to see which cross-site trackers are being blocked, and a new tab bar design that provides tab...

Full Transcript of Apple's 'Time Flies' Event With Apple Watch and iPad Updates

Tuesday September 15, 2020 8:46 am PDT by
Apple's virtual "Time Flies" event kicks off today at 10:00 a.m. Pacific Time, with Apple expected to debut new Apple Watch and iPad models, but it appears we may not see the iPhone 12 until next month. While we're not expecting to see new iPhones today, Apple's software updates for its various platforms are likely nearly ready for launch, so we may be hearing more about them today. Check...

Everything Apple Announced at Today's iPad and Apple Watch Event in Just Seven Minutes

Tuesday September 15, 2020 2:19 pm PDT by
Apple today held its annual September event, but this year was a bit different because no new iPhones were announced. The "Time Flies" event focused on the Apple Watch Series 6 and the iPad Air, and saw the debut of a new Apple One services bundle. Subscribe to the MacRumors YouTube channel for more videos. It took Apple an hour to introduce the new devices, but we've recapped the event in...