iOS 15's Live Text Feature Lets You Digitize Written Notes, Call a Number on a Sign, Translate a Menu, and Much More

In iOS 15, Apple is introducing a new feature called Live Text that can recognize text when it appears in your camera's viewfinder or in a photo you've taken and let you perform several actions with it.

Apple iPadPro iPadOS15 photos LiveText 060721 big
For example, Live Text allows you to capture a phone number from a storefront with the option to place a call, or look up a location name in Maps to get directions. It also incorporates optical character recognition, so you can search for a picture of a handwritten note in your photos and save it as text.

Live Text's content awareness extends to everything from QR codes to emails that appear in pictures, and this on-device intelligence feeds into Siri suggestions, too.

ios15 live text
For instance, if you take a picture that shows an email address and then open the Mail app and start composing a message, ‌Siri‌'s keyboard suggestions will offer up the option to add "Email from Camera" to the To field of your message.

Other Live Text options include the ability to copy text from the camera viewfinder or photos for pasting elsewhere, share it, look it up in the dictionary, and translate it for you into English, Chinese (both simplified and traditional), French, Italian, German, Spanish, or Portuguese.

live text translate
It can even sort your photos by location, people, scene, objects, and more, by recognizing the text in pictures. For example, searching for a word or phrase in Spotlight search will bring up pictures from your Camera Roll in which that text occurs.

Live Text works in Photos, Screenshot, Quick Look, and Safari and in live previews with Camera. In the Camera app, it's available whenever you point your iPhone's camera at anything that displays text, and is indicated by a small icon that appears in the bottom right corner whenever textual content is recognized in the viewfinder. Tapping the icon lets you tap recognized text and perform an action with it. A similar icon appears in the ‌Photos‌ app when you're viewing a shot image.

visual look up ios 15
In another neural engine feature, Apple is introducing something called Visual Look Up that lets you take photos of objects and scenes to get more information from them. Point your ‌iPhone‌'s camera at a piece of art, flora, fauna, landmarks, or books, and the Camera will indicate with an icon that it recognizes the content and has relevant ‌Siri‌ Knowledge that can add context.

Since Live Text relies heavily on Apple's neural engine, the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an ‌iPhone‌ X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.

The ‌iOS 15‌ beta is currently in the hands of developers, with a public beta set to be released next month. The official launch of ‌iOS 15‌ is scheduled for the fall.

Related Roundups: iOS 15, iPadOS 15
Related Forum: iOS 15

Top Rated Comments

ruka.snow Avatar
1 week ago
Let's see them digitise doctors notes and prescriptions.
Score: 19 Votes (Like | Disagree)
Unggoy Murderer Avatar
1 week ago

They don’t want to. Hence why they aren’t enabling it for the intel macs. Just a cash grab as they hope everybody will sell their newish intel Mac, which they paid handsomely for, at a huge loss and then go and pay handsomely again for a new Apple silicon Mac.
I'm sure Apple could get it running on the A11, however people apparently forget there will be trade-offs to enable that to happen. What if enabling it on the A11 halved the battery life, would you want that, would that be a trade-off you'd be willing to make? Or perhaps the performance of the phone drops by 25%, how about that?

Apple can't (and shouldn't be expected to) support older hardware with every single new feature that's released. Apple do a good job of supporting older hardware far better than the competitors do (the iPhone 6s, a 2015 phone, gets iOS 15), they'll get the benefit of at least some new features and improvements.

I'm sure some people will want to upgrade their hardware to support the latest software additions, but I (and I'm sure like a lot of other folks like me with newish machines), won't be bothered enough. Nice features, but I won't lose sleep over not having them.


Apple isn’t the company from heaven as most seem to think.
Of course they're not - Apple is a money making enterprise.
Score: 14 Votes (Like | Disagree)
EmotionalSnow Avatar
1 week ago

Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
The later phones have an improved Neural Engine. I'm guessing that these are required in order to ensure good performance.
Score: 8 Votes (Like | Disagree)
Unggoy Murderer Avatar
1 week ago

The old World Lens app (and subsequent iterations after Google acquired it) were able to both capture and translate text in real-time using substantially slower hardware.

Live text would just be this, without the translation (whether on device or not). I can’t see why the neural engine would be an absolute requirement, even if you try to make a ‘performance’ argument.
Try using Word Lens for any sustained period - it absolutely hammered your battery. And just to be clear, that isn't a criticism of Word Lens - it was an exceptional app for its time, it just couldn't take advantage of optimised silicon.

The performance argument is *the* argument. Apple more than likely could implement it on all hardware, but what it it caused your battery life to be half of usual, or Safari and camera apps to be considerably slower while it runs CPU-bound neural networks on every image downloaded, or every camera frame?

Also worth noting, text recognition isn't a trivial task, especially with handwriting.
Score: 8 Votes (Like | Disagree)
farewelwilliams Avatar
1 week ago
implemented better than android
Score: 5 Votes (Like | Disagree)
LukeHarrison Avatar
1 week ago

Since Live Text relies heavily on Apple's neural engine, so the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an iPhone X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.
Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
Score: 5 Votes (Like | Disagree)

Top Stories

16 inch macbook pro m2 render

When Can We Expect the Redesigned MacBook Pros Now?

Wednesday June 16, 2021 7:11 am PDT by
With no sign of redesigned MacBook Pro models at this year's WWDC, when can customers expect the much-anticipated new models to launch? A number of reports, including investor notes from Morgan Stanley and Wedbush analysts, claimed that new MacBook Pro models would be coming during this year's WWDC. This did not happen, much to the disappointment of MacBook Pro fans, who have been...
2021 back t0 school

Apple Launches 2021 Back to School Promotion: Free AirPods With Eligible Mac or iPad Purchase

Thursday June 17, 2021 4:56 am PDT by
Apple today launched its seasonal back-to-school sale for the upcoming school year in the United States and Canada, offering students free AirPods alongside purchases of select Macs and iPad models. Similar to last year's promotion, this year's offer includes free AirPods alongside the purchase of a MacBook Air, MacBook Pro, the new 24-inch iMac, the Mac Pro, Mac mini, and the new M1-powered ...
maxresdefault

Apple CEO Tim Cook: Sideloading Apps Would 'Destroy the Security' of the iPhone

Wednesday June 16, 2021 10:49 am PDT by
Apple CEO Tim Cook this morning participated in a virtual interview at the VivaTech conference, which is described as Europe's biggest startup and tech event. Cook was interviewed by Guillaume Lacroix, CEO and founder of Brut, a media company that creates short-form video content. Much of the discussion centered on privacy, as it often does in interviews that Cook participates in. He...
YouTube Picture in Picture Feature

YouTube Says iOS Picture-in-Picture Coming to All US Users

Friday June 18, 2021 9:41 am PDT by
After a long wait, YouTube for iOS is officially gaining picture-in-picture support, allowing all users, non-premium and premium subscribers, to close the YouTube app and continue watching their video in a small pop-up window. In a statement to MacRumors, YouTube says that picture-in-picture is currently rolling out to all premium subscribers on iOS and that a larger rollout to all US iOS...
applecare lower prices

Apple Lowers Prices of AppleCare+ Plans for M1 MacBook Air and MacBook Pro

Thursday June 17, 2021 7:33 am PDT by
Apple today lowered the prices of AppleCare+ plans for MacBook Air and 13-inch MacBook Pro models equipped with the M1 chip. Coverage offered by the plans, as well as accidental damage fees, appear to remain unchanged. In the United States, AppleCare+ for the MacBook Air now costs $199, down from $249. The new price applies to both M1 and Intel-based MacBook Air models, although Apple no...
m1 v intel thumb

Intel Processor Market Share May Fall to New Low Next Year Due to Apple Silicon

Friday June 18, 2021 2:06 am PDT by
Intel may see its market share fall to a new low next year, in large part thanks to Apple's decision to move away from using Intel processors in its Mac computers and instead use Apple silicon. Apple announced last year that it would embark on a two-year-long journey to transition all of its Mac computers, both desktops, and laptops, to use its own in-house processors. Apple is expected to...
apple watch edition series 5 ceramic black prototype

Apple Planned Black Ceramic Apple Watch Edition Series 5

Wednesday June 16, 2021 5:45 am PDT by
Apple considered offering a black version of the ceramic Apple Watch Edition Series 5, according to newly-shared images of the prototype casing. The images, shared on Twitter by the prototype collector and leaker known as "Mr. White," show a prototype black ceramic Apple Watch casing, alongside the white ceramic version. The ceramic Apple Watch Edition Series 5 was never available in a...
maxresdefault

Demo: Check Out AirPlay 2 on a Mac in macOS Monterey

Tuesday June 15, 2021 11:57 am PDT by
With macOS Monterey, Apple has introduced expanded AirPlay 2 support, so you can AirPlay content from an iPhone, iPad, or even another Mac to your main Mac. We thought we'd do a quick demo of this handy new feature in our latest YouTube video. Subscribe to the MacRumors YouTube channel for more videos. With AirPlay to Mac, you can extend or mirror an Apple device's display to a Mac, and since ...
app store blue banner

U.S. Antitrust Legislation Would Require Users to Be Able to Delete All Pre-Installed Apple Apps [Updated]

Wednesday June 16, 2021 11:26 am PDT by
Update: Bloomberg inaccurately interpreted the original comments and has since drastically changed the wording of its article. The original Bloomberg piece said that Apple would be prohibited from pre-installing its own apps on iPhones. The updated Bloomberg article has been rewritten to clarify that the antitrust legislation prohibits Apple from preventing users from removing Apple-created...
3nm apple silicon feature

Apple Supplier TSMC Readies 3nm Chip Production for Second Half of 2022

Friday June 18, 2021 6:59 am PDT by
Apple supplier TSMC is preparing to produce 3nm chips in the second half of 2022, and in the coming months, the supplier will begin production of 4nm chips, according to a new report from DigiTimes. Apple had previously booked the initial capacity of TSMC's 4nm chip production for future Macs and more recently ordered TSMC to begin production of the A15 chip for the upcoming iPhone 13,...