A patent application published by the United States Patent and Trademark Office today details a new Apple service where users could make inquiries and talk with the company's AI assistant Siri through Messages (via AppleInsider). The new patent is similar to a filing the USPTO published late last year, but now includes deeper integration with audio, video, and image files.

Similar to chatbots in Facebook Messenger and other texting services, Apple's patent describes a Siri that could perform her current duties without the user having to speak aloud, which could be helpful in certain public situations.

The "Intelligent Automated Assistant in a Messaging Environment" could respond to text, audio, images, and video when sent to it by the user, which Apple said would result in "a richer interactive experience between a user and a digital assistant." The patent gives a few examples of a conversation held between Siri and a user in Messages, with the user asking questions regarding calorie content in food, upcoming meetings, and even asking Siri to text a friend.

siri chatbot 2
Interesting applications include a thread where a user texts Siri a picture of a car or a bottle of wine, and Siri sees the images and can intelligently respond to the user's inquiries about them. For the car, the user asks Siri for details on pricing for a specific model using only an image, and Siri searches the internet and returns the relevant MSRP information.

The bottle of wine image is used as an example to show Siri's memory functions, where a user asks Siri to remember their favorite wine, which she can resurface at a later date. Siri sees the wine image, reads the label, and can then respond to a user's question in text format about the brand and even year it was made.

siri chatbot 6
Other image-related inquiries include "Where is this place?" and "What insect is this?", to which Siri would respond "This is the country Algeria" and "This is an earwig," respectively. Audio and video could also be recognized by Siri, including simple Shazam-like questions related to songs and the content of shared videos.

Apple points out in its patent that thanks to the chronological format of texting, users would be able to "review previous interactions" with Siri, unlike how current Siri conversations disappear immediately after they conclude. Subsequently, Siri would be able to use that history to become smarter and "define a wider range of tasks."

The messaging platform can enable multiple modes of input (e.g., text, audio, images, video, etc.) to be sent and received. As described herein, this can increase the functionality and capabilities of the digital assistant, thereby providing a richer interactive experience between a user and a digital assistant.

A digital assistant in a message environment can thus enable greater accessibility to the digital assistant. In particular, the digital assistant can be accessible in noisy environments or in environments where audio output is not desired (e.g., the library). Moreover, the chronological format enables a user to conveniently review previous interactions with the digital assistant and utilize the contextual history associated with the previous interactions to define a wider range of tasks.

The patent includes a description where Siri would be "a participant in a multi-party conversation," allowing group chats to use Apple's AI simultaneously. Apple gives an example where one user asks Siri to list nearby Chinese restaurants to begin making the group's dinner plans, and then another user responds by asking Siri to whittle down the list to only include the cheapest places. One user's personal Siri can even be asked to remind other participants of the upcoming dinner.

siri chatbot 8
Apple is believed to be working on an "enhanced Siri" that might launch in iOS 11 this fall, but the exact specifications as to what would make the new Siri "enhanced" have never been divulged. A questionable rumor in March stated that deep Siri integration is coming to Messages in iOS 11, but the source of the news -- The Verifier -- doesn't have a previous track record of reporting accurate rumors.

Chatbots are certainly growing in popularity so it wouldn't be too surprising if Apple introduced some kind of text-based Siri interface, particularly considering the multiple patents the company has published on the topic. Still, as with all patents it's best to look at Apple's new filing as an intriguing insight into what the company might be working on for the future, rather than proof of an impending launch.

Top Rated Comments

lincolntran Avatar
51 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
bc lots of people prefer to not talking in to a phone in public places or quite places.
Score: 6 Votes (Like | Disagree)
NT1440 Avatar
51 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
.....are you asserting that somehow working on one feature means that the core of Siri isn't being worked on?
Score: 5 Votes (Like | Disagree)
Scottsoapbox Avatar
51 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
Score: 3 Votes (Like | Disagree)
Scottsoapbox Avatar
51 months ago
. . . . . . . . [image of car]

What should I do with it?

. . . . . . . . How much does it cost?

OK, now playing songs by Lady Gaga.
Score: 3 Votes (Like | Disagree)
lazyrighteye Avatar
51 months ago
Low-hanging Siri jokes aside, this seems interesting. I welcome any traction on Siri development. And while today's Siri makes the thought of Siri inquiries via photo or video seems laughable, the next update to Siri could make this concept more plausible.

As one who tries to interact with Siri as much as possible, this Messages concept could be nice for a couple of reasons.

1. An archived Siri thread.
I'm often frustrated when after conducting a Siri inquiry and then leaving the Siri window to (say) check a link, I can't get back to my initial Siri interaction to review, continue or amend. Even a standalone Siri "app" could be interesting. Could offer a simple/intuitive way for users to "get back" to their Siri-ing via Home button double-click to reveal a Siri slide in the app switcher.

2. The ability to access Siri via text input.
There are definitely scenarios where accessing Siri via voice isn't always desirable. Being able to interact with her via text is a welcome option.

Will be interesting to see what Apple has up their sleeve regarding Siri. Will
Be nice if they update us on Siri dev at WWDC. While it can never been good enough, any development/advancement of Siri will be welcomed.
Score: 2 Votes (Like | Disagree)
lincolntran Avatar
51 months ago
And most people prefer google search to Siri search so...
So now we have 2 choices (and many others) instead of google, choices are good.
Score: 1 Votes (Like | Disagree)

Top Stories

samsung experience 1

Samsung's 'iTest' Lets You Try a Galaxy Device on Your iPhone

Thursday April 8, 2021 12:42 pm PDT by
Samsung has launched "iTest," an interactive website experience that's designed to allow iPhone users to test out Android on a Galaxy device, or "sample the other side," as Samsung puts it. Subscribe to the MacRumors YouTube channel for more videos. The iTest website is being advertised in New Zealand, according to a MacRumors reader who came across the feature. Visiting the iTest website on...
pixel watch prosser leak

Google Pixel Watch Allegedly Leaks with Circular Design, Rumored to Launch in October

Monday April 12, 2021 2:49 am PDT by
Renders of Google's first smartwatch, codenamed "Rohan," have been shared by Jon Prosser, showing that Google plans to adopt a circular design for its flagship wearable watch. Prosser shared the renders in an episode of his YouTube show "Front Page Tech," in which he claims they were made based on marketing material he had seen from a source within Google. The renders show that the Pixel...
AppleTV and HomePod Feature

Bloomberg: Apple Working on New Apple TV With Integrated HomePod Speaker and FaceTime Camera

Monday April 12, 2021 3:32 am PDT by
Apple is working on a combined Apple TV with HomePod speaker that has a camera for video calls through a connected television set, according to Bloomberg's Mark Gurman. From the report: The company is working on a product that would combine an Apple TV set-top box with a HomePod speaker and include a camera for video conferencing through a connected TV and other smart-home functions,...
nba tracking prompt orange

Two-Thirds of iPhone Users Expected to Block Ad Tracking

Friday April 9, 2021 7:19 am PDT by
As many as 68 percent of iPhone users are expected to deny advertisers permission to track them thanks to Apple's App Tracking Transparency feature, in what is beginning to look like a significant blow to the advertising industry (via AdWeek). With the launch of iOS 14.5, apps will have to receive explicit user permission before accessing an iPhone's advertising identifier or IDFA, which is...
fake airpods 3

Counterfeit 'AirPods 3' Hit the Market Prior to Official Announcement

Friday April 9, 2021 2:45 am PDT by
Apple is expected to launch the third iteration of AirPods in the third quarter of this year. Rumors and reports suggest the new AirPods will feature an updated design more in line with the AirPods Pro, but lacking in "Pro" features such as active noise cancellation. Despite AirPods 3 not yet being officially announced by Apple, counterfeit products of the unreleased earbuds have already hit ...
Top Stories 56 Feature

Top Stories: 'Find My' Expansion, iPhone 13 Pro Mockup, Largest-Ever iMac?

Saturday April 10, 2021 6:00 am PDT by
Apple's AirTags may still be nowhere to be seen nearly two years after signs of them were first discovered, but Apple this week launched its Find My network accessory program that will let third-party devices integrate with the Find My app on Apple's platforms to make it easy to keep track of your items. This week also saw fresh rumors about the upcoming "iPhone 13" and new iMacs, while...
ehric

iPhone 12 Mini Missing From Top 5 Best Selling Smartphone List of January 2021

Friday April 9, 2021 4:58 am PDT by
According to market data compiled by Counterpoint Research, Apple's smallest iPhone since the 2016 iPhone SE, the iPhone 12 mini, struggled to obtain a spot in the top five list of best-selling smartphones in January of this year. According to the market data, the iPhone 12 mini came in eighth place for the best-selling smartphone worldwide in the first month of the year. However, the iPhone ...
sonny 2021 ipad mini pro dummies

Leaked Dummy Units Show iPad Mini 6 With Thick Bezels and Home Button, New iPad Pro Models

Thursday April 8, 2021 2:11 am PDT by
Rumors suggest Apple will release refreshed versions of the iPad mini and iPad Pro models in the first half of this year, potentially as soon as this month, and a new leak today has provided us with a possible preview of what to expect in terms of the devices' overall design and camera prospects. Tech leaker and Apple blogger Sonny Dickson this morning shared images on Twitter showing dummy ...
Google maps feaure green

Google Maps App for iOS Finally Updated After Four Months

Monday April 12, 2021 10:03 am PDT by
Following the completed rollout of App Privacy labels for its App Store apps, Google today updated the Google Maps app for the first time in four months. Apple in December began requiring all new app submissions and app updates to include App Privacy labels, detailing the data that is collected by the app so consumers know what they're sharing. Google didn't begin implementing App Privacy ...
epic iap feature 3

Tim Cook Says App Store Would Become a 'Flea Market' if Third-Party Payment Systems Were Allowed

Monday April 12, 2021 9:41 am PDT by
In a recent interview with the Toronto Star, Apple CEO Tim Cook spoke about a wide variety of topics, ranging from App Tracking Transparency to Apple's ongoing legal battle over App Store policies with Fortnite creator Epic Games. Notably, Cook said that Epic Games' desire for Apple to let developers offer their own payment systems in apps "would make the App Store a flea market":At the...