iOS 15's Live Text Feature Lets You Digitize Written Notes, Call a Number on a Sign, Translate a Menu, and Much More

In iOS 15, Apple is introducing a new feature called Live Text that can recognize text when it appears in your camera's viewfinder or in a photo you've taken and let you perform several actions with it.

Apple iPadPro iPadOS15 photos LiveText 060721 big
For example, Live Text allows you to capture a phone number from a storefront with the option to place a call, or look up a location name in Maps to get directions. It also incorporates optical character recognition, so you can search for a picture of a handwritten note in your photos and save it as text.

Live Text's content awareness extends to everything from QR codes to emails that appear in pictures, and this on-device intelligence feeds into Siri suggestions, too.

ios15 live text
For instance, if you take a picture that shows an email address and then open the Mail app and start composing a message, ‌Siri‌'s keyboard suggestions will offer up the option to add "Email from Camera" to the To field of your message.

Other Live Text options include the ability to copy text from the camera viewfinder or photos for pasting elsewhere, share it, look it up in the dictionary, and translate it for you into English, Chinese (both simplified and traditional), French, Italian, German, Spanish, or Portuguese.

live text translate
It can even sort your photos by location, people, scene, objects, and more, by recognizing the text in pictures. For example, searching for a word or phrase in Spotlight search will bring up pictures from your Camera Roll in which that text occurs.

Live Text works in Photos, Screenshot, Quick Look, and Safari and in live previews with Camera. In the Camera app, it's available whenever you point your iPhone's camera at anything that displays text, and is indicated by a small icon that appears in the bottom right corner whenever textual content is recognized in the viewfinder. Tapping the icon lets you tap recognized text and perform an action with it. A similar icon appears in the ‌Photos‌ app when you're viewing a shot image.

visual look up ios 15
In another neural engine feature, Apple is introducing something called Visual Look Up that lets you take photos of objects and scenes to get more information from them. Point your ‌iPhone‌'s camera at a piece of art, flora, fauna, landmarks, or books, and the Camera will indicate with an icon that it recognizes the content and has relevant ‌Siri‌ Knowledge that can add context.

Since Live Text relies heavily on Apple's neural engine, the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an ‌iPhone‌ X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.

The ‌iOS 15‌ beta is currently in the hands of developers, with a public beta set to be released next month. The official launch of ‌iOS 15‌ is scheduled for the fall.

Related Roundups: iOS 15, iPadOS 15
Related Forum: iOS 15

Top Rated Comments

ruka.snow Avatar
19 weeks ago
Let's see them digitise doctors notes and prescriptions.
Score: 19 Votes (Like | Disagree)
Unggoy Murderer Avatar
19 weeks ago

They don’t want to. Hence why they aren’t enabling it for the intel macs. Just a cash grab as they hope everybody will sell their newish intel Mac, which they paid handsomely for, at a huge loss and then go and pay handsomely again for a new Apple silicon Mac.
I'm sure Apple could get it running on the A11, however people apparently forget there will be trade-offs to enable that to happen. What if enabling it on the A11 halved the battery life, would you want that, would that be a trade-off you'd be willing to make? Or perhaps the performance of the phone drops by 25%, how about that?

Apple can't (and shouldn't be expected to) support older hardware with every single new feature that's released. Apple do a good job of supporting older hardware far better than the competitors do (the iPhone 6s, a 2015 phone, gets iOS 15), they'll get the benefit of at least some new features and improvements.

I'm sure some people will want to upgrade their hardware to support the latest software additions, but I (and I'm sure like a lot of other folks like me with newish machines), won't be bothered enough. Nice features, but I won't lose sleep over not having them.


Apple isn’t the company from heaven as most seem to think.
Of course they're not - Apple is a money making enterprise.
Score: 14 Votes (Like | Disagree)
EmotionalSnow Avatar
19 weeks ago

Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
The later phones have an improved Neural Engine. I'm guessing that these are required in order to ensure good performance.
Score: 8 Votes (Like | Disagree)
Unggoy Murderer Avatar
19 weeks ago

The old World Lens app (and subsequent iterations after Google acquired it) were able to both capture and translate text in real-time using substantially slower hardware.

Live text would just be this, without the translation (whether on device or not). I can’t see why the neural engine would be an absolute requirement, even if you try to make a ‘performance’ argument.
Try using Word Lens for any sustained period - it absolutely hammered your battery. And just to be clear, that isn't a criticism of Word Lens - it was an exceptional app for its time, it just couldn't take advantage of optimised silicon.

The performance argument is *the* argument. Apple more than likely could implement it on all hardware, but what it it caused your battery life to be half of usual, or Safari and camera apps to be considerably slower while it runs CPU-bound neural networks on every image downloaded, or every camera frame?

Also worth noting, text recognition isn't a trivial task, especially with handwriting.
Score: 8 Votes (Like | Disagree)
farewelwilliams Avatar
19 weeks ago
implemented better than android
Score: 5 Votes (Like | Disagree)
LukeHarrison Avatar
19 weeks ago

Since Live Text relies heavily on Apple's neural engine, so the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an iPhone X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.
Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
Score: 5 Votes (Like | Disagree)

Related Stories

live text macos monterey feature

macOS Monterey Beta 4 Brings Live Text to Intel-Based Macs

Tuesday July 27, 2021 10:54 am PDT by
The latest beta update of macOS Monterey, released to developers today, has brought Live Text functionality to Intel-based Mac computers, removing the requirement for users to use an M1 Apple silicon Mac to utilize the feature, according to Rene Ritchie. Live Text, included in the forthcoming macOS Monterey, iOS 15, and iPadOS 15 updates, allows users to interact with text within images,...
iOS 15 icon on phone

Apple Releases iOS 15 and iPadOS 15 With Safari Updates, Focus Mode, Live Text, iCloud+, System-Wide Translate, On-Device Siri and More

Monday September 20, 2021 10:04 am PDT by
Apple today released iOS 15 and iPadOS 15, the newest operating system updates designed for the iPhone, iPad, and iPod touch. As with all of Apple's software updates, iOS and iPadOS 15 can be downloaded at no cost. iOS 15 is available on the iPhone 6s and later while iPadOS 15 is available on the iPad Air 2 and later. The new software can be downloaded on eligible devices over-the-air by...
iWork macOS Trio Feature

Apple Updates Pages, Numbers, and Keynote With New Features for iOS 15 and macOS Monterey

Tuesday September 28, 2021 9:09 am PDT by
Apple today announced updates to its iWork suite of apps with new features, including some that are limited to iOS 15 and macOS Monterey. This includes updates to the Pages, Numbers, and Keynote apps across the iPhone, iPad, and Mac. Newly redesigned iWork icons on the Mac Apple says Keynote now offers an option to show a live camera view of the presenter directly in slides across the iPhone, ...
iOS 15 icon on phone

Apple Seeds iOS and iPadOS 15 Release Candidates to Developers

Tuesday September 14, 2021 11:48 am PDT by
Apple today seeded the release candidate versions of iOS and iPadOS 15 to developers for testing purposes, with the updates coming two weeks after Apple released the eighth betas. Registered developers can download the profile for the iOS and iPadOS betas from the Apple Developer Center, and once the profile is installed, beta updates will be available over the air. iOS 15 is a major...
macos monterey tidbits feature copy

Apple Releases New macOS 12 Monterey Public Beta

Thursday October 7, 2021 10:29 am PDT by
Apple today seeded a new public beta of the macOS 12 Monterey beta to public beta testers, allowing non-developers to test the new macOS Monterey software ahead of its public release. The update comes just a day after Apple provided the ninth beta to developers. Public beta testers can download the macOS 12 Monterey update from the Software Update section of the System Preferences app after ...
macOS Monterey on MBP Feature

Apple Seeds macOS Monterey Release Candidate to Developers

Monday October 18, 2021 11:10 am PDT by
Apple today seeded the release candidate version of macOS Monterey, the newest version of the macOS operating system. The eleventh beta comes one week after Apple released the tenth macOS Monterey beta. Registered developers can download the beta through the Apple Developer Center and once the appropriate profile is installed, betas will be available through the Software Update mechanism in...
macos monterey tidbits feature copy

Apple Releases New macOS 12 Monterey Public Beta

Wednesday September 29, 2021 10:21 am PDT by
Apple today seeded a new public beta of the macOS 12 Monterey beta to public beta testers, allowing non-developers to test the new macOS Monterey software ahead of its public release. The update comes just a few day after Apple provided the eighth beta to developers. Public beta testers can download the macOS 12 Monterey update from the Software Update section of the System Preferences app...
macOS Monterey on MBP Feature

Apple Seeds Tenth Beta of macOS Monterey to Developers

Wednesday October 13, 2021 10:13 am PDT by
Apple today seeded the tenth developer beta of macOS Monterey, the newest version of the macOS operating system. The tenth beta comes one week after Apple released the ninth macOS Monterey beta. Registered developers can download the beta through the Apple Developer Center and once the appropriate profile is installed, betas will be available through the Software Update mechanism in System...
iOS 15 icon on phone

Apple Seeds Eighth Betas of iOS and iPadOS 15 to Developers [Public Beta Available]

Tuesday August 31, 2021 10:03 am PDT by
Apple today seeded the eighth betas of iOS and iPadOS 15 to developers for testing purposes, with the updates coming one week after Apple released the seventh betas. Registered developers can download the profile for the iOS and iPadOS betas from the Apple Developer Center, and once the profile is installed, beta updates will be available over the air. iOS 15 is a major update that...
iOS 15 icon on phone

Apple Seeds Seventh Betas of iOS and iPadOS 15 to Developers [Update: Public Beta Available]

Wednesday August 25, 2021 10:06 am PDT by
Apple today seeded the seventh betas of iOS and iPadOS 15 to developers for testing purposes, with the updates coming one week after Apple released the sixth betas. Registered developers can download the profile for the iOS and iPadOS betas from the Apple Developer Center, and once the profile is installed, beta updates will be available over the air. iOS 15 is a major update that...