Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google

Piper Jaffray analyst Gene Munster, who has regularly assessed Siri's accuracy in terms of correctly interpreting and answering queries, has issued the latest version of his Siri report card, noting that Siri has continued to improve under iOS 7, particularly in terms of being able to properly interpret questions being asked.

The primary improvement between Siri in iOS 6 and iOS 7 came from its performance in noisier areas, where Siri was able to translate better through a noisy environment (94% vs. 83% correct in Aug iOS 6). Based on our testing, iOS 7 Siri's net ability to answer questions (factors in questions not correctly understood) improved to 79% from 77% in August with iOS 6.

munster_siri_dec13_report
The improvements in Siri's understanding come alongside but separate from Apple's continued efforts to reduce its reliance on Google, as evidenced by Siri's shift to emphasize Bing, Wikipedia, and WolframAlpha results. In Munster's most recent testing, only 4 percent of Siri's answers relied on Google, compared to 17 percent in August testing and 27 percent in December 2012.

munster_siri_dec13.services

Click for larger

Munster also compares Siri to Google Now, finding that the two services are now essentially tied in effectiveness with a "C+" score after Siri held a substantial lead in the year-ago testing. In the latest testing, Google Now held small leads in content areas such as commerce and information, while Siri performed significantly better for operating system controls.

munster_siri_dec13_questions

Top Rated Comments

HenryDJP Avatar
93 months ago

There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.


I get 90% accuracy, I don't know what's going on with you.
Score: 13 Votes (Like | Disagree)
NightFox Avatar
93 months ago
So even though its accuracy at understanding has gone up, the accuracy of its answers has actually got worse?
Score: 12 Votes (Like | Disagree)
lil' brudder Avatar
93 months ago
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
Score: 10 Votes (Like | Disagree)
NightFox Avatar
93 months ago

The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.


Never defend a technology! Technology is meant to develop to serve the requirements/limitations of existing human behaviour; human behaviour shouldn't be expected to adapt to meet the requirements/limitations of technology.
Score: 8 Votes (Like | Disagree)
Simmias Avatar
93 months ago
I have had the opposite experience. After seeing steady improvement in Siri, it has recently gotten worse and become almost unusable for me.

Specifically, I use Siri a lot to play music in the car. I will say "Play Song X" or "Play Album Y". Siri used to be pretty good at hearing the name of the song and finding it in my music library. Now it frequently hear the names incorrectly. Even when it does hear the correct name, it will play a song with a completely different name or simply say "Sorry, I couldn't find that in your music". What do you mean you couldn't find David Bowie in my music??? It's right there!

This regression is quite infuriating. Also, Siri still often fails to respond ("Sorry, there's something wrong") or take 20+ seconds to respond with the wrong answer. I believe we need to give all of Apple's web services time to gather data and make iterative improvements, but I have very little tolerance when they actually get worse over time.
Score: 7 Votes (Like | Disagree)
chirpie Avatar
93 months ago
I don't know, that net percentage increase (4% better) year over year seems kind of low to me. Am I the only one who feels that way? Not my field of study, so I'm not entirely sure if this is supposed to be impressive.
Score: 6 Votes (Like | Disagree)

Top Stories

lg wing

LG Considering Exit From Smartphone Business, Halts LCD Production for iPhone

Wednesday January 20, 2021 5:38 am PST by
LG is considering exiting the smartphone business entirely amid declining shipments and accrued losses of $4.5 billion over the past five years (via The Korea Herald). LG CEO Kwon Bong-Seok cautioned staff earlier today that the company is re-evaluating its presence in the smartphone industry: Since the competition in the global market for mobile devices is getting fiercer, it is about...
google maps detailed street level e1611052089473

Google Maps Gains Enhanced Street-Level Detail in Four Major Cities

Tuesday January 19, 2021 2:34 am PST by
Google Maps has quietly been updated to include significantly more detailed street-level information in a handful of key cities around the world. Upon zooming in, Google's maps for Central London, Tokyo, San Francisco, and New York now benefit from shapes and widths that match the scale of roads more accurately. Meanwhile, enhanced graphical representations of sidewalks, crosswalks,...
2019 mac pro side and front

Tim Cook Gifted Donald Trump 'First' 2019 Mac Pro

Wednesday January 20, 2021 5:45 pm PST by
Apple CEO Tim Cook gifted former United States President Donald Trump with the first 2019 Mac Pro that came off of the assembly line in Austin, Texas, according to a financial disclosure report that was released today (via The Verge). "Mac Pro Computer, the first created at the Flex Factory in Austin, Texas," reads the entry, which values the machine at $5,999, the base price for a Mac Pro....
airpods max sim ejector

AirPods Max Headband Removable With Just a SIM Ejector Tool, Hinting at Interchangeable Headbands

Tuesday January 19, 2021 8:25 am PST by
It is possible to remove the headband of AirPods Max with just a standard SIM card ejector tool, hinting at the possibility of interchanging headbands to achieve a different colorway. Image via Prelook In December, MacRumors revealed the large variety of AirPods Max ear cushion color combinations when it became clear that they were magnetically attatched and available for sale separately....
iphone 5s black slate

Images of Unreleased iPhone 5s in Black and Slate Shared Online

Sunday January 17, 2021 9:47 am PST by
Twitter user @DongleBookPro has today shared images of a prototype iPhone 5s in an unreleased Black and Slate color. The iPhone 5s was launched in September 2013. The device featured Touch ID, a 64-bit processor, and a True Tone LED flash for the first time. Other new features included a five-element lens with an f/2.2 aperture, a 15 percent larger camera sensor, Burst Mode, and Slo-Mo...
iphone 12 vs iphone 12 mini

Apple Shifting Some Production From iPhone 12 mini to iPhone 12 Pro to Meet Demand

Wednesday January 20, 2021 8:12 am PST by
Apple has reportedly cut production of the iPhone 12 mini by two million units to create more manufacturing capacity for the iPhone 12 Pro, according to a new Morgan Stanley investment note seen by PED30. Apple is believed to have made the switch for the first quarter of 2021 in an effort to combat continuing lead times for the more popular iPhone 12 Pro. iPhone 12 Pro lead times remain ...
Apple VR Feature

Bloomberg: Apple's First AR/VR Headset 'Pricey, Niche Precursor' to More Ambitious AR Glasses and Could Launch Next Year

Thursday January 21, 2021 3:27 am PST by
Apple's first virtual reality headset will be a "pricey, niche precursor" to a more ambitious augmented reality product, according to a new report from Bloomberg's Mark Gurman. As a mostly virtual reality device, it will display an all-encompassing 3-D digital environment for gaming, watching video and communicating. AR functionality, the ability to overlay images and information over a view...
shot on iphone 12 apple

Apple Highlights Photos Shot by iPhone 12 Users: Portraits, Cityscapes, and More

Tuesday January 19, 2021 6:05 am PST by
Apple today shared a gallery of photos shot by customers using the iPhone 12 mini, iPhone 12, iPhone 12 Pro, and iPhone 12 Pro Max, with scenes including cityscapes, landscapes, portraits of people, and more at day and night. Shot on iPhone 12 Pro Max by "NKCHU" in China (top) and shot on iPhone 12 Pro Max by Rohit Vohra in India (bottom) iPhone 12 mini and iPhone 12 models have a dual camera ...
Apple and Hyundai feature

Apple Car Production Again Linked to Kia Motor's US Plant in Georgia

Tuesday January 19, 2021 4:19 am PST by
Hyundai intends to transition the company's Apple Car involvement to its Kia brand as part of an internal arrangement that could see production move to the U.S., according to a new report today. On Sunday, Korea IT News reported that Apple and Hyundai are seeking a partnership agreement for the upcoming Apple Car by March, and that the electric vehicles could be made at a Georgia factory...
iPhone 13 Notch Feature

iPhone 13 Rumored to Feature Smaller Notch, Pro Model Cameras to Use Larger Image Sensor

Thursday January 21, 2021 1:38 am PST by
Apple's iPhone 13 series will feature a redesigned Face ID system that will allow for a smaller notch at the top of the screen, according to a new report today. The rumor comes via hit-and-miss Taiwanese industry publication DigiTimes, whose supply chain sources also claim that the ultra wide-angle lens in Apple's next-generation iPhones is due for an upgrade. The next-generation iPhones'...