iOS 15's Live Text Feature Lets You Digitize Written Notes, Call a Number on a Sign, Translate a Menu, and Much More

In iOS 15, Apple is introducing a new feature called Live Text that can recognize text when it appears in your camera's viewfinder or in a photo you've taken and let you perform several actions with it.

Apple iPadPro iPadOS15 photos LiveText 060721 big
For example, Live Text allows you to capture a phone number from a storefront with the option to place a call, or look up a location name in Maps to get directions. It also incorporates optical character recognition, so you can search for a picture of a handwritten note in your photos and save it as text.

Live Text's content awareness extends to everything from QR codes to emails that appear in pictures, and this on-device intelligence feeds into Siri suggestions, too.

ios15 live text
For instance, if you take a picture that shows an email address and then open the Mail app and start composing a message, ‌Siri‌'s keyboard suggestions will offer up the option to add "Email from Camera" to the To field of your message.

Other Live Text options include the ability to copy text from the camera viewfinder or photos for pasting elsewhere, share it, look it up in the dictionary, and translate it for you into English, Chinese (both simplified and traditional), French, Italian, German, Spanish, or Portuguese.

live text translate
It can even sort your photos by location, people, scene, objects, and more, by recognizing the text in pictures. For example, searching for a word or phrase in Spotlight search will bring up pictures from your Camera Roll in which that text occurs.

Live Text works in Photos, Screenshot, Quick Look, and Safari and in live previews with Camera. In the Camera app, it's available whenever you point your iPhone's camera at anything that displays text, and is indicated by a small icon that appears in the bottom right corner whenever textual content is recognized in the viewfinder. Tapping the icon lets you tap recognized text and perform an action with it. A similar icon appears in the ‌Photos‌ app when you're viewing a shot image.

visual look up ios 15
In another neural engine feature, Apple is introducing something called Visual Look Up that lets you take photos of objects and scenes to get more information from them. Point your ‌iPhone‌'s camera at a piece of art, flora, fauna, landmarks, or books, and the Camera will indicate with an icon that it recognizes the content and has relevant ‌Siri‌ Knowledge that can add context.

Since Live Text relies heavily on Apple's neural engine, the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an ‌iPhone‌ X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.

The iOS 15 beta is currently in the hands of developers, with a public beta set to be released next month. The official launch of iOS 15 is scheduled for the fall.

Related Forum: iOS 15

Top Rated Comments

ruka.snow Avatar
35 months ago
Let's see them digitise doctors notes and prescriptions.
Score: 19 Votes (Like | Disagree)
Unggoy Murderer Avatar
35 months ago

They don’t want to. Hence why they aren’t enabling it for the intel macs. Just a cash grab as they hope everybody will sell their newish intel Mac, which they paid handsomely for, at a huge loss and then go and pay handsomely again for a new Apple silicon Mac.
I'm sure Apple could get it running on the A11, however people apparently forget there will be trade-offs to enable that to happen. What if enabling it on the A11 halved the battery life, would you want that, would that be a trade-off you'd be willing to make? Or perhaps the performance of the phone drops by 25%, how about that?

Apple can't (and shouldn't be expected to) support older hardware with every single new feature that's released. Apple do a good job of supporting older hardware far better than the competitors do (the iPhone 6s, a 2015 phone, gets iOS 15), they'll get the benefit of at least some new features and improvements.

I'm sure some people will want to upgrade their hardware to support the latest software additions, but I (and I'm sure like a lot of other folks like me with newish machines), won't be bothered enough. Nice features, but I won't lose sleep over not having them.


Apple isn’t the company from heaven as most seem to think.
Of course they're not - Apple is a money making enterprise.
Score: 14 Votes (Like | Disagree)
EmotionalSnow Avatar
35 months ago

Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
The later phones have an improved Neural Engine. I'm guessing that these are required in order to ensure good performance.
Score: 8 Votes (Like | Disagree)
Unggoy Murderer Avatar
35 months ago

The old World Lens app (and subsequent iterations after Google acquired it) were able to both capture and translate text in real-time using substantially slower hardware.

Live text would just be this, without the translation (whether on device or not). I can’t see why the neural engine would be an absolute requirement, even if you try to make a ‘performance’ argument.
Try using Word Lens for any sustained period - it absolutely hammered your battery. And just to be clear, that isn't a criticism of Word Lens - it was an exceptional app for its time, it just couldn't take advantage of optimised silicon.

The performance argument is *the* argument. Apple more than likely could implement it on all hardware, but what it it caused your battery life to be half of usual, or Safari and camera apps to be considerably slower while it runs CPU-bound neural networks on every image downloaded, or every camera frame?

Also worth noting, text recognition isn't a trivial task, especially with handwriting.
Score: 8 Votes (Like | Disagree)
farewelwilliams Avatar
35 months ago
implemented better than android
Score: 5 Votes (Like | Disagree)
LukeHarrison Avatar
35 months ago

Since Live Text relies heavily on Apple's neural engine, so the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an iPhone X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.
Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
Score: 5 Votes (Like | Disagree)

Popular Stories

General Apps Messages

Apple Announces 'Groundbreaking' New Security Protocol for iMessage

Wednesday February 21, 2024 6:00 am PST by
Apple today announced a new post-quantum cryptographic protocol for iMessage called PQ3. Apple says this "groundbreaking" and "state-of-the-art" protocol provides "extensive defenses against even highly sophisticated quantum attacks." Apple believes the PQ3 protocol's protections "surpass those in all other widely deployed messaging apps," according to its blog post:Today we are announcing...
iOS 17

iOS 17.4 Will Add These New Features to Your iPhone

Monday February 19, 2024 6:52 am PST by
Apple last month confirmed that iOS 17.4 will be released in March, and the update includes several new features and changes for the iPhone. Key new features in iOS 17.4 include major App Store changes in the EU and Apple Podcasts transcripts. The update also adds new emoji and includes preparations for the launch of next-generation CarPlay later this year. More details about the new...
Beyond iPhone 13 Better Blue Face ID Single Camera Hole

Six Reasons to Wait for Next Year's iPhone 17

Thursday February 22, 2024 4:20 am PST by
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models concurrently, which is why we sometimes get rumored feature leaks so far ahead of launch. The iPhone 17 series is no different, and already we have some idea of what to expect from Apple's 2025 smartphone lineup. If you plan to skip...
samsung galaxy ring

'Apple Ring' Allegedly in Development to Rival Samsung Galaxy Ring

Tuesday February 20, 2024 2:27 am PST by
Apple is speeding up development of a smart ring that can be worn on the finger to track users' health biometrics, claims a new report coming out of Korea. Teaser image of Samsung Galaxy Ring shown at Galaxy Unpacked in January Apple has toyed with the idea of a ring wearable for several years, as indicated by several patents, but with Samsung preparing to bring its own product to market, the ...
volvo s60 drivers apple maps

iOS 17.4 Beta Adds CarPlay Option to Show Upcoming Maneuvers in Instrument Cluster

Tuesday February 20, 2024 10:47 am PST by
The fourth beta of iOS 17.4 that Apple released today adds a new CarPlay feature, according to the notes that Apple provided to developers. In supported CarPlay vehicles, there's now an Apple Maps option to show information about upcoming maneuvers in the instrument cluster. CarPlay users will be able to swap the display type between the main and instrument cluster by tapping on the map...
Oled iPads and MackBook Pro Notch

Apple's OLED Roadmap: New iPad Mini, Foldable iPad Pro, and More

Wednesday February 21, 2024 5:29 am PST by
Apple is planning to launch at least nine new devices with OLED displays across the iPad and MacBook product lines, according to an updated forecast from research firm Omdia that sets out the company's plans in detail. As widely rumored, Apple's push to transition to OLED will apparently be jump-started by the release of new 11- and 13-inch iPad Pro models this year. The displays will...
iPad Pro OLED Feature 2

Apple's Upcoming OLED iPad Pro Models Rumored to Be Much Thinner

Tuesday February 20, 2024 1:39 pm PST by
Apple is planning to update the iPad Air and iPad Pro lineups as soon as March, and the new iPad Pro models will be significantly thinner according to dimensions shared by 9to5Mac. Citing sources with knowledge of Apple's plans, the site claims that the larger version will be more than 1mm thinner. The current 12.9-inch iPad Pro measures in at 6.4mm thick, but the new model is said to be 5mm ...
m3 macbook pro 14 16

Apple Now Selling Refurbished M3 Pro and M3 Max MacBook Pro Models

Monday February 19, 2024 5:04 pm PST by
Apple today began offering refurbished versions of the 14-inch and 16-inch MacBook Pro models with M3 Pro and M3 Max chip options, offering the machines at a discount for the first time since their October 2023 release. The release of M3 Pro and M3 Max models on Apple's refurbished store comes almost two weeks after the entry-level 14-inch M3 MacBook Pro first appeared on the store....