Chatbot-Like Siri Patent Includes Intelligent Image, Video, and Audio Recognition Within Messages - MacRumors
Skip to Content

Chatbot-Like Siri Patent Includes Intelligent Image, Video, and Audio Recognition Within Messages

by

A patent application published by the United States Patent and Trademark Office today details a new Apple service where users could make inquiries and talk with the company's AI assistant Siri through Messages (via AppleInsider). The new patent is similar to a filing the USPTO published late last year, but now includes deeper integration with audio, video, and image files.

Similar to chatbots in Facebook Messenger and other texting services, Apple's patent describes a Siri that could perform her current duties without the user having to speak aloud, which could be helpful in certain public situations.

The "Intelligent Automated Assistant in a Messaging Environment" could respond to text, audio, images, and video when sent to it by the user, which Apple said would result in "a richer interactive experience between a user and a digital assistant." The patent gives a few examples of a conversation held between Siri and a user in Messages, with the user asking questions regarding calorie content in food, upcoming meetings, and even asking Siri to text a friend.

siri chatbot 2
Interesting applications include a thread where a user texts Siri a picture of a car or a bottle of wine, and Siri sees the images and can intelligently respond to the user's inquiries about them. For the car, the user asks Siri for details on pricing for a specific model using only an image, and Siri searches the internet and returns the relevant MSRP information.

The bottle of wine image is used as an example to show Siri's memory functions, where a user asks Siri to remember their favorite wine, which she can resurface at a later date. Siri sees the wine image, reads the label, and can then respond to a user's question in text format about the brand and even year it was made.

siri chatbot 6
Other image-related inquiries include "Where is this place?" and "What insect is this?", to which Siri would respond "This is the country Algeria" and "This is an earwig," respectively. Audio and video could also be recognized by Siri, including simple Shazam-like questions related to songs and the content of shared videos.

Apple points out in its patent that thanks to the chronological format of texting, users would be able to "review previous interactions" with Siri, unlike how current Siri conversations disappear immediately after they conclude. Subsequently, Siri would be able to use that history to become smarter and "define a wider range of tasks."

The messaging platform can enable multiple modes of input (e.g., text, audio, images, video, etc.) to be sent and received. As described herein, this can increase the functionality and capabilities of the digital assistant, thereby providing a richer interactive experience between a user and a digital assistant.

A digital assistant in a message environment can thus enable greater accessibility to the digital assistant. In particular, the digital assistant can be accessible in noisy environments or in environments where audio output is not desired (e.g., the library). Moreover, the chronological format enables a user to conveniently review previous interactions with the digital assistant and utilize the contextual history associated with the previous interactions to define a wider range of tasks.

The patent includes a description where Siri would be "a participant in a multi-party conversation," allowing group chats to use Apple's AI simultaneously. Apple gives an example where one user asks Siri to list nearby Chinese restaurants to begin making the group's dinner plans, and then another user responds by asking Siri to whittle down the list to only include the cheapest places. One user's personal Siri can even be asked to remind other participants of the upcoming dinner.

siri chatbot 8
Apple is believed to be working on an "enhanced Siri" that might launch in iOS 11 this fall, but the exact specifications as to what would make the new Siri "enhanced" have never been divulged. A questionable rumor in March stated that deep Siri integration is coming to Messages in iOS 11, but the source of the news -- The Verifier -- doesn't have a previous track record of reporting accurate rumors.

Chatbots are certainly growing in popularity so it wouldn't be too surprising if Apple introduced some kind of text-based Siri interface, particularly considering the multiple patents the company has published on the topic. Still, as with all patents it's best to look at Apple's new filing as an intriguing insight into what the company might be working on for the future, rather than proof of an impending launch.

Top Rated Comments

lincolntran Avatar
124 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
bc lots of people prefer to not talking in to a phone in public places or quite places.
Score: 6 Votes (Like | Disagree)
NT1440 Avatar
124 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
.....are you asserting that somehow working on one feature means that the core of Siri isn't being worked on?
Score: 5 Votes (Like | Disagree)
Scottsoapbox Avatar
124 months ago
. . . . . . . . [image of car]

What should I do with it?

. . . . . . . . How much does it cost?

OK, now playing songs by Lady Gaga.
Score: 3 Votes (Like | Disagree)
Scottsoapbox Avatar
124 months ago
So instead of fixing Siri's speech interface Apple wants you to type in queries...

...so different than Googling.
Score: 3 Votes (Like | Disagree)
lazyrighteye Avatar
124 months ago
Low-hanging Siri jokes aside, this seems interesting. I welcome any traction on Siri development. And while today's Siri makes the thought of Siri inquiries via photo or video seems laughable, the next update to Siri could make this concept more plausible.

As one who tries to interact with Siri as much as possible, this Messages concept could be nice for a couple of reasons.

1. An archived Siri thread.
I'm often frustrated when after conducting a Siri inquiry and then leaving the Siri window to (say) check a link, I can't get back to my initial Siri interaction to review, continue or amend. Even a standalone Siri "app" could be interesting. Could offer a simple/intuitive way for users to "get back" to their Siri-ing via Home button double-click to reveal a Siri slide in the app switcher.

2. The ability to access Siri via text input.
There are definitely scenarios where accessing Siri via voice isn't always desirable. Being able to interact with her via text is a welcome option.

Will be interesting to see what Apple has up their sleeve regarding Siri. Will
Be nice if they update us on Siri dev at WWDC. While it can never been good enough, any development/advancement of Siri will be welcomed.
Score: 2 Votes (Like | Disagree)
dampfnudel Avatar
124 months ago
I've never noticed this before, but do they always put a real carrier name on these patent mockups? I thought they usually leave them intentionally vague. Looks like Apple is a fan of T-Mobile as well!
Well, Apple's definitely not a fan of AT&T and T-Mobile has a reputation of being the cool carrier that tries to be more like the best European/Asian wireless carriers, giving customers more choices, features that you would expect in 2017 like seamless/inexpensive international data roaming, and a better value.
Score: 1 Votes (Like | Disagree)

Popular Stories

iOS 27

Apple Releases iOS 27.2 With New Siri AI Languages

Monday October 26, 2026 11:32 am PDT by
Apple today released iOS 27.2 and iPadOS 27.2, the first major updates to the iOS 27 and iPadOS 27 operating systems that came out in September. iOS 27.2 is available for all iPhones able to run iOS 27, including the iPhone Duo, while iPadOS 27.2 is available for all iPads able to run iPadOS 27, including the newly announced OLED iPad mini. The iOS 27.2 update expands Siri AI to new...
Apple TV 4K 4th Generation feature0

Apple Releases tvOS 27.2.1 With Siri AI for Older Apple TVs

Friday November 13, 2026 11:26 am PST by
Apple today released tvOS 27.2.1, an update to the tvOS 27 operating system that launched in September. tvOS 27.2.1 comes nearly three weeks after Apple released tvOS 27.2. tvOS 27.2.1 brings Siri AI support to the 2nd- and 3rd-generation Apple TV 4K models, but it is available for all Apple TV 4K models that support tvOS 27. Apple announced Siri AI for the Apple TV when debuting the...
Apple TV 4K Thumb 2

Hands-On With the New Apple TV 4K and Siri AI

Thursday November 12, 2026 4:41 pm PST by
The 4th-generation Apple TV 4K with an A19 chip and Siri AI support is available starting on November 13, and we were able to spend some time with it ahead of launch. Subscribe to the MacRumors YouTube channel for more videos. Siri AI is the headline feature, and while it's not perfect, it works better than the prior version of Siri. Siri was able to pull up multiple movie and TV show...
Latest Stories
iFixit Says Apple's HomeView is Unnecessarily Hard to Repair
iFixit Says Apple's HomeView is Unnecessarily Hard to Repair
2 hours ago
Final Cut Pro Now Includes MotionVFX for Creator Studio Users
Final Cut Pro Now Includes MotionVFX for Creator Studio Users
3 hours ago
Apple Blames Internal Update for Outage, Says It Wasn't a Cyberattack
Apple Blames Internal Update for Outage, Says It Wasn't a Cyberattack
4 hours ago
Nomad Launches Black Friday Sale With Up to 30% Off Apple Accessories
Nomad Launches Black Friday Sale With Up to 30% Off Apple Accessories
5 hours ago
Google Wallet Available on iPhone, But Without Tap to Pay
Google Wallet Available on iPhone, But Without Tap to Pay
5 hours ago
Apple Accounts, App Store and Apple Music Experiencing Issues
Apple Accounts, App Store and Apple Music Experiencing Issues
6 hours ago
Apple Moves Ahead With Better Dual SIM Support for Apps
Apple Moves Ahead With Better Dual SIM Support for Apps
7 hours ago
Former Apple Chairman Plans to Sell $32 Million in Apple Stock
Former Apple Chairman Plans to Sell $32 Million in Apple Stock
7 hours ago
Apple Accused of Silencing iPhone Duo Supplier Over 'Crackgate' Issue
Apple Accused of Silencing iPhone Duo Supplier Over 'Crackgate' Issue
8 hours ago
Apple Store App Now Optimized for iPhone Duo
Apple Store App Now Optimized for iPhone Duo
8 hours ago
Apple Says iCloud+ Plans Will Not Include Increased Siri AI Usage in UK
Apple Says iCloud+ Plans Will Not Include Increased Siri AI Usage in UK
9 hours ago
Amazon Offering Early Black Friday Discounts on TVs, Headphones, and Charging Devices
Amazon Offering Early Black Friday Discounts on TVs, Headphones, and Charging Devices
9 hours ago
Apple Now Sells DIY Repair Parts for iPhone Duo
Apple Now Sells DIY Repair Parts for iPhone Duo
9 hours ago
iPhone 20 Pro Screen Protectors Show Rumored Curved Design
iPhone 20 Pro Screen Protectors Show Rumored Curved Design
10 hours ago
MacBook Neo Black Friday Deals Arrive With $49 Discounts on Nearly Every Model at Amazon
MacBook Neo Black Friday Deals Arrive With $49 Discounts on Nearly Every Model at Amazon
10 hours ago
iPhone Users Targeted in Major Website Security Breach
iPhone Users Targeted in Major Website Security Breach
11 hours ago
Apple Names 'Baby, This Is Keke Palmer' Podcast of the Year
Apple Names 'Baby, This Is Keke Palmer' Podcast of the Year
11 hours ago
Apple Tests Bringing Back a Softer Side of Liquid Glass
Apple Tests Bringing Back a Softer Side of Liquid Glass
12 hours ago
Apple Close to Rare Long-Term Memory Deal With Samsung
Apple Close to Rare Long-Term Memory Deal With Samsung
13 hours ago
Everything New in iOS 27.3 Beta 2
Everything New in iOS 27.3 Beta 2
23 hours ago
Apple Plans to Release These 18 New Products Next Year
Apple Plans to Release These 18 New Products Next Year
1 day ago
New Apple TV 4K Discount Returns at Amazon
New Apple TV 4K Discount Returns at Amazon
1 day ago
iPhone Duo Orders Will Now Arrive Before the Holidays
iPhone Duo Orders Will Now Arrive Before the Holidays
1 day ago
Hands-On With the iPhone Duo's New Face Unlock
Hands-On With the iPhone Duo's New Face Unlock
1 day ago
macOS 27.3 Warns Against Giving AI Apps Full Mac Control
macOS 27.3 Warns Against Giving AI Apps Full Mac Control
1 day ago
iPad and Mac Get Apple Intelligence in China in 27.3 Betas
iPad and Mac Get Apple Intelligence in China in 27.3 Betas
1 day ago
Apple News+ Puzzles Come to UK, Australia, and French Canada
Apple News+ Puzzles Come to UK, Australia, and French Canada
1 day ago
macOS Golden Gate 27.3 Beta 2 Now Available for Developers
macOS Golden Gate 27.3 Beta 2 Now Available for Developers
1 day ago
Apple Celebrates Pokémon With iPhone Wallpapers and Special Gift Card
Apple Celebrates Pokémon With iPhone Wallpapers and Special Gift Card
1 day ago
Apple TV App Gets Video Podcasts in iOS 27.3
Apple TV App Gets Video Podcasts in iOS 27.3
1 day ago