A patent application published by the United States Patent and Trademark Office today details a new Apple service where users could make inquiries and talk with the company's AI assistant Siri through Messages (via AppleInsider). The new patent is similar to a filing the USPTO published late last year, but now includes deeper integration with audio, video, and image files.

Similar to chatbots in Facebook Messenger and other texting services, Apple's patent describes a Siri that could perform her current duties without the user having to speak aloud, which could be helpful in certain public situations.

The "Intelligent Automated Assistant in a Messaging Environment" could respond to text, audio, images, and video when sent to it by the user, which Apple said would result in "a richer interactive experience between a user and a digital assistant." The patent gives a few examples of a conversation held between Siri and a user in Messages, with the user asking questions regarding calorie content in food, upcoming meetings, and even asking Siri to text a friend.

siri chatbot 2
Interesting applications include a thread where a user texts Siri a picture of a car or a bottle of wine, and Siri sees the images and can intelligently respond to the user's inquiries about them. For the car, the user asks Siri for details on pricing for a specific model using only an image, and Siri searches the internet and returns the relevant MSRP information.

The bottle of wine image is used as an example to show Siri's memory functions, where a user asks Siri to remember their favorite wine, which she can resurface at a later date. Siri sees the wine image, reads the label, and can then respond to a user's question in text format about the brand and even year it was made.

siri chatbot 6
Other image-related inquiries include "Where is this place?" and "What insect is this?", to which Siri would respond "This is the country Algeria" and "This is an earwig," respectively. Audio and video could also be recognized by Siri, including simple Shazam-like questions related to songs and the content of shared videos.

Apple points out in its patent that thanks to the chronological format of texting, users would be able to "review previous interactions" with Siri, unlike how current Siri conversations disappear immediately after they conclude. Subsequently, Siri would be able to use that history to become smarter and "define a wider range of tasks."

The messaging platform can enable multiple modes of input (e.g., text, audio, images, video, etc.) to be sent and received. As described herein, this can increase the functionality and capabilities of the digital assistant, thereby providing a richer interactive experience between a user and a digital assistant.

A digital assistant in a message environment can thus enable greater accessibility to the digital assistant. In particular, the digital assistant can be accessible in noisy environments or in environments where audio output is not desired (e.g., the library). Moreover, the chronological format enables a user to conveniently review previous interactions with the digital assistant and utilize the contextual history associated with the previous interactions to define a wider range of tasks.

The patent includes a description where Siri would be "a participant in a multi-party conversation," allowing group chats to use Apple's AI simultaneously. Apple gives an example where one user asks Siri to list nearby Chinese restaurants to begin making the group's dinner plans, and then another user responds by asking Siri to whittle down the list to only include the cheapest places. One user's personal Siri can even be asked to remind other participants of the upcoming dinner.

siri chatbot 8
Apple is believed to be working on an "enhanced Siri" that might launch in iOS 11 this fall, but the exact specifications as to what would make the new Siri "enhanced" have never been divulged. A questionable rumor in March stated that deep Siri integration is coming to Messages in iOS 11, but the source of the news -- The Verifier -- doesn't have a previous track record of reporting accurate rumors.

Chatbots are certainly growing in popularity so it wouldn't be too surprising if Apple introduced some kind of text-based Siri interface, particularly considering the multiple patents the company has published on the topic. Still, as with all patents it's best to look at Apple's new filing as an intriguing insight into what the company might be working on for the future, rather than proof of an impending launch.

Top Rated Comments

lincolntran Avatar
55 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
bc lots of people prefer to not talking in to a phone in public places or quite places.
Score: 6 Votes (Like | Disagree)
NT1440 Avatar
55 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
.....are you asserting that somehow working on one feature means that the core of Siri isn't being worked on?
Score: 5 Votes (Like | Disagree)
Scottsoapbox Avatar
55 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
Score: 3 Votes (Like | Disagree)
Scottsoapbox Avatar
55 months ago
. . . . . . . . [image of car]

What should I do with it?

. . . . . . . . How much does it cost?

OK, now playing songs by Lady Gaga.
Score: 3 Votes (Like | Disagree)
lazyrighteye Avatar
55 months ago
Low-hanging Siri jokes aside, this seems interesting. I welcome any traction on Siri development. And while today's Siri makes the thought of Siri inquiries via photo or video seems laughable, the next update to Siri could make this concept more plausible.

As one who tries to interact with Siri as much as possible, this Messages concept could be nice for a couple of reasons.

1. An archived Siri thread.
I'm often frustrated when after conducting a Siri inquiry and then leaving the Siri window to (say) check a link, I can't get back to my initial Siri interaction to review, continue or amend. Even a standalone Siri "app" could be interesting. Could offer a simple/intuitive way for users to "get back" to their Siri-ing via Home button double-click to reveal a Siri slide in the app switcher.

2. The ability to access Siri via text input.
There are definitely scenarios where accessing Siri via voice isn't always desirable. Being able to interact with her via text is a welcome option.

Will be interesting to see what Apple has up their sleeve regarding Siri. Will
Be nice if they update us on Siri dev at WWDC. While it can never been good enough, any development/advancement of Siri will be welcomed.
Score: 2 Votes (Like | Disagree)
lincolntran Avatar
55 months ago
And most people prefer google search to Siri search so...
So now we have 2 choices (and many others) instead of google, choices are good.
Score: 1 Votes (Like | Disagree)

Top Stories

Flat 2021 MacBook Pro Mockup Feature

Unreleased Apple Macs and Apple Watches Listed in Eurasian Database Ahead of Fall Product Launches

Monday August 2, 2021 9:34 am PDT by
Apple is preparing for a slew of fall product launches according to new filings that showed up today in the Eurasian Economic Commission database. There are listings for new Mac and Apple Watch models, all of which have previously unknown model identifiers that indicate that they're upcoming devices. There are six new Apple Watch identifiers, including A2473, A2474, A2475, A2476, A2477, and...
ifixit iphone12 mini

Apple to Make Space for Larger Batteries in iPhones, iPads, and MacBooks By Adopting Slimmer Peripheral Chips

Monday August 2, 2021 2:12 am PDT by
For future iPhones, iPads, and MacBooks, Apple plans to use smaller internal components in an effort to increase the size of the device's battery, according to DigiTimes. Image Credit: iFixit Specifically, Apple plans to "significantly increase the adoption" of IPDs or integrated passive devices for the peripheral chips in its products. These news chips will be slimmer in size and allow for...
Apple Watch 7 Unreleased Feature

Apple Watch Series 7 to Focus on One Major Upgrade

Wednesday August 4, 2021 2:12 am PDT by
The upcoming Apple Watch Series 7 will focus on one important feature in an attempt to tempt existing Apple Watch users that have an older device to upgrade, according to recent reports. Apple may skip adding new health sensors to this year's Apple Watch Series 7 in favor of improving the device's battery life. The company is said to be adopting new double-sided System in Package (SiP)...
magic keyboard touch id

Apple Makes Magic Keyboard With Touch ID Available for Separate Purchase

Tuesday August 3, 2021 5:22 am PDT by
Apple has made the Magic Keyboard with Touch ID, which previously was only available with the purchase of the new 24-inch iMac, available for purchase individually for $149. Apple also retails the Magic Keyboard with Touch ID and a numeric keypad for $179. A standard Magic Keyboard without Touch ID or a numeric keypad is available for $99, and a new Magic Trackpad for $129. One major...
iPhone 13 Wi Fi 6E feature update

Wi-Fi 6E Explained: What It Could Mean for iPhone 13 and Beyond

Monday August 2, 2021 8:00 am PDT by
The iPhone 13 is widely expected to come with Wi-Fi 6E capabilities, and while it may seem rather nuanced to the average consumer, with only improved speeds and being "up to date" in the realm of Wi-Fi technology, it's actually a fairly significant improvement, laying the groundwork for much of what we know the future holds. To truly understand Wi-Fi 6E, MacRumors sat down for an exclusive...
applestoredown

Apple's Online Store Temporarily Down [Update: Back Up]

Tuesday August 3, 2021 4:01 pm PDT by
Apple's online Apple Store is down at the current time, and attempting to access it to make a purchase gives the standard "Be Right Back" message. Given that it's a Tuesday night/afternoon in the United States and we're not expecting any new products this week, it's likely that this is a temporary maintenance outage that is not related to a new product release. The Apple Store app is also...
themorningshowcarrell

Apple Decided Not to Buy Reese Witherspoon's 'Hello Sunshine' Media Company

Monday August 2, 2021 2:02 pm PDT by
Reese Witherspoon's media company "Hello Sunshine" recently courted various buyers, and while Apple was one of parties interested in buying Hello Sunshine, the Cupertino company did not end up going through with the purchase. Hello Sunshine was valued at around $900 million thanks to its involvement in popular series like The Morning Show," "Big Little Lies," and "Little Fires Everywhere,"...
iPhone 13 Dummy Thumbnail 2

Apple Brings China's Luxshare Precision into iPhone 13 Supply Chain to Meet Production Targets

Wednesday August 4, 2021 12:19 am PDT by
Apple is tapping more Chinese suppliers as it seeks to meet ambitious targets for iPhone 13 production, according to a new report by Nikkei Asia. Apple is set to produce between 90 million and 95 million iPhones through January, according to a previous Nikkei report, and China's Luxshare Precision Industry has won 3% of orders away from Taiwanese rivals Foxconn and Pegatron. Luxshare will...
mac pro new graphics

Apple Introduces New High-End Graphics Options for Mac Pro

Tuesday August 3, 2021 7:34 am PDT by
Apple today began offering new high-end graphics upgrade options for both the tower and rack versions of the Mac Pro desktop computer. This comes on the same day that Apple started selling the Magic Keyboard with Touch ID on a standalone basis. As noted by CNN Underscored's Jake Krol, the Mac Pro can now be configured with new AMD Radeon Pro W6800X, W6800X Duo, or W6900X graphics when...
General Apps Messages

Android iMessage Competitor Puts Pressure on Apple

Friday July 30, 2021 3:15 am PDT by
Google and the three major U.S. carriers, including Verizon, AT&T, and T-Mobile, will all support a new communications protocol on Android smartphones starting in 2022, a move that puts pressure on Apple to adopt a new cross-platform messaging standard and may present a challenge to iMessage. Verizon recently announced that it is planning to adopt Messages by Google as its default messaging...