iOS 15's Live Text Feature Lets You Digitize Written Notes, Call a Number on a Sign, Translate a Menu, and Much More

In iOS 15, Apple is introducing a new feature called Live Text that can recognize text when it appears in your camera's viewfinder or in a photo you've taken and let you perform several actions with it.

Apple iPadPro iPadOS15 photos LiveText 060721 big
For example, Live Text allows you to capture a phone number from a storefront with the option to place a call, or look up a location name in Maps to get directions. It also incorporates optical character recognition, so you can search for a picture of a handwritten note in your photos and save it as text.

Live Text's content awareness extends to everything from QR codes to emails that appear in pictures, and this on-device intelligence feeds into Siri suggestions, too.

ios15 live text
For instance, if you take a picture that shows an email address and then open the Mail app and start composing a message, ‌Siri‌'s keyboard suggestions will offer up the option to add "Email from Camera" to the To field of your message.

Other Live Text options include the ability to copy text from the camera viewfinder or photos for pasting elsewhere, share it, look it up in the dictionary, and translate it for you into English, Chinese (both simplified and traditional), French, Italian, German, Spanish, or Portuguese.

live text translate
It can even sort your photos by location, people, scene, objects, and more, by recognizing the text in pictures. For example, searching for a word or phrase in Spotlight search will bring up pictures from your Camera Roll in which that text occurs.

Live Text works in Photos, Screenshot, Quick Look, and Safari and in live previews with Camera. In the Camera app, it's available whenever you point your iPhone's camera at anything that displays text, and is indicated by a small icon that appears in the bottom right corner whenever textual content is recognized in the viewfinder. Tapping the icon lets you tap recognized text and perform an action with it. A similar icon appears in the ‌Photos‌ app when you're viewing a shot image.

visual look up ios 15
In another neural engine feature, Apple is introducing something called Visual Look Up that lets you take photos of objects and scenes to get more information from them. Point your ‌iPhone‌'s camera at a piece of art, flora, fauna, landmarks, or books, and the Camera will indicate with an icon that it recognizes the content and has relevant ‌Siri‌ Knowledge that can add context.

Since Live Text relies heavily on Apple's neural engine, the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an ‌iPhone‌ X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.

The ‌iOS 15‌ beta is currently in the hands of developers, with a public beta set to be released next month. The official launch of ‌iOS 15‌ is scheduled for the fall.

Related Roundups: iOS 15, iPadOS 15
Related Forum: iOS 15

Top Rated Comments

ruka.snow Avatar
15 months ago
Let's see them digitise doctors notes and prescriptions.
Score: 19 Votes (Like | Disagree)
Unggoy Murderer Avatar
15 months ago

They don’t want to. Hence why they aren’t enabling it for the intel macs. Just a cash grab as they hope everybody will sell their newish intel Mac, which they paid handsomely for, at a huge loss and then go and pay handsomely again for a new Apple silicon Mac.
I'm sure Apple could get it running on the A11, however people apparently forget there will be trade-offs to enable that to happen. What if enabling it on the A11 halved the battery life, would you want that, would that be a trade-off you'd be willing to make? Or perhaps the performance of the phone drops by 25%, how about that?

Apple can't (and shouldn't be expected to) support older hardware with every single new feature that's released. Apple do a good job of supporting older hardware far better than the competitors do (the iPhone 6s, a 2015 phone, gets iOS 15), they'll get the benefit of at least some new features and improvements.

I'm sure some people will want to upgrade their hardware to support the latest software additions, but I (and I'm sure like a lot of other folks like me with newish machines), won't be bothered enough. Nice features, but I won't lose sleep over not having them.


Apple isn’t the company from heaven as most seem to think.
Of course they're not - Apple is a money making enterprise.
Score: 14 Votes (Like | Disagree)
EmotionalSnow Avatar
15 months ago

Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
The later phones have an improved Neural Engine. I'm guessing that these are required in order to ensure good performance.
Score: 8 Votes (Like | Disagree)
Unggoy Murderer Avatar
15 months ago

The old World Lens app (and subsequent iterations after Google acquired it) were able to both capture and translate text in real-time using substantially slower hardware.

Live text would just be this, without the translation (whether on device or not). I can’t see why the neural engine would be an absolute requirement, even if you try to make a ‘performance’ argument.
Try using Word Lens for any sustained period - it absolutely hammered your battery. And just to be clear, that isn't a criticism of Word Lens - it was an exceptional app for its time, it just couldn't take advantage of optimised silicon.

The performance argument is *the* argument. Apple more than likely could implement it on all hardware, but what it it caused your battery life to be half of usual, or Safari and camera apps to be considerably slower while it runs CPU-bound neural networks on every image downloaded, or every camera frame?

Also worth noting, text recognition isn't a trivial task, especially with handwriting.
Score: 8 Votes (Like | Disagree)
farewelwilliams Avatar
15 months ago
implemented better than android
Score: 5 Votes (Like | Disagree)
LukeHarrison Avatar
15 months ago

Since Live Text relies heavily on Apple's neural engine, so the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an iPhone X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.
Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
Score: 5 Votes (Like | Disagree)

Related Stories

22 mic camera access

AV Artists Complain Monterey 'Orange Dot' Security Feature Makes Macs Unsuitable for Live Performances

Wednesday December 22, 2021 4:45 am PST by
Audiovisual artists who use Macs to output to large stage screens are calling on Apple to provide a way to disable a new security feature in macOS Monterey that is hindering their live performances. For those unfamiliar with the feature, when an app is using the camera or the microphone in Monterey‌, a colored dot appears in the menu bar just next to the Control Center icon. Like in iOS, a ...
ipad mini jelly scrolling ifixit

Apple Hit With Class Action Lawsuit Over 'Jelly Scrolling' on iPad Mini

Thursday February 10, 2022 7:56 am PST by
Colorado resident Christopher Bryan on Wednesday filed a class action lawsuit against Apple that accuses the company of fraudulently marketing and selling the sixth-generation iPad mini despite being aware of a so-called "jelly scrolling" defect with the device, according to court documents reviewed by MacRumors. A few days after the sixth-generation iPad mini was released in September, some ...
telegram

Telegram Messenger Gains Download Manager, New Attachment Menu, and More

Friday March 11, 2022 3:24 am PST by
Telegram Messenger has updated its iPhone and iPad app with several new features, including a new download manager, redesigned attachment menu, support for live streaming with third-party apps, and more. Telegram users are able to send files of any type up to 2GB each and access them from any device, with no limit on cloud storage, which has made downloading files more popular on the...
telegram reactions

Telegram Updated With Message Reactions and Several Other Features

Thursday December 30, 2021 6:52 am PST by
Telegram today announced it has updated its iPhone and iPad app with several new features, including message reactions, translation, hidden text, additional interactive emoji options, themed QR codes for users and channels, and more. With the addition of reactions, users can now double tap on a message bubble and a small thumbs-up emoji will appear at the bottom of that message. You can...
iPad Air Feature 2 green

Buyer's Guide: Don't Buy an iPad Air Right Now – New Model Imminent

Monday February 21, 2022 2:16 am PST by
The fifth-generation iPad Air is expected to arrive in less than a week with a number of notable features and improvements, so prospective customers should now hold off on buying an iPad Air until the new model arrives. According to Japanese blog Mac Otakara, Apple is planning to give the fifth-generation iPad Air features from the sixth-generation iPad mini. This seems plausible since the...
Twitter Feature

Twitter's DM Search Bar Now Lets You Search the Content of Messages

Thursday March 24, 2022 3:01 am PDT by
Twitter has bolstered its search feature for direct messages so that users can now search their inbox using keywords and names to find the specific message they are looking for. The DM search bar isn't new, but previously it only allowed you to find the name of a person or a group chat in your inbox. The latest development means you can now refine searches to find particular content in your...
osbscura 3

Obscura 3 Camera App Gains Overhauled Interface and New Capture Modes

Friday February 18, 2022 2:16 am PST by
Alternative iPhone camera app Obscura 3 has launched with several new camera features and an overhauled design that aims to make its various photography tools easier to access. In the newly designed interface, the control wheel of the previous version has been replaced by an entirely gesture-based control system with haptic feedback, which the developer says improves single-thumb control and ...
iOS 16 mock for article

With WWDC Five Months Away, What Do You Want to See in iOS 16?

Thursday January 6, 2022 1:22 pm PST by
The Worldwide Developers Conference, Apple's annual five-day-long conference to announce updates to its major operating systems, services, and sometimes hardware, is five months away. At this year's conference, we expect Apple to announce iOS 16, iPadOS 16, macOS 13, watchOS 9, and tvOS 16. iOS is Apple's most important operating system as it runs on the iPhone and shares many features with...

Popular Stories

cook sept 2020 event

Gurman: Apple Preparing Pre-Recorded iPhone 14 and Apple Watch Series 8 Event

Sunday August 7, 2022 6:13 am PDT by
Apple has "started to record" its virtual September event, where it's expected to announce the upcoming iPhone 14 lineup, the Apple Watch Series 8, and a new "rugged" Apple Watch model, according to Bloomberg's Mark Gurman. Writing in his latest Power On newsletter, Gurman says the event, which is expected to take place in the early part of September, is already under production, implying...
iPhone 14 Lineup Feature Purple

Color Options for All iPhone 14 Models: Everything We Know

Monday August 8, 2022 3:59 am PDT by
The iPhone 14 and iPhone 14 Pro models are rumored to be available in a refreshed range of color options, including an all-new purple color. Most expectations about the iPhone 14 lineup's color options come from an unverified post on Chinese social media site Weibo earlier this year. Overall, the iPhone 14 and iPhone 14 Pro's selection of color options could look fairly similar to those of the ...
ios 16 beta 5 battery percent

iOS 16 Beta 5: Battery Percentage Now Displayed in iPhone Status Bar

Monday August 8, 2022 10:43 am PDT by
With the fifth beta of iOS 16, Apple has updated the battery icon on iPhones with Face ID to display the specific battery percentage rather than just a visual representation of battery level. The new battery indicator is available on iPhone 12 and iPhone 13 models, with the exception of the 5.4-inch iPhone 12/13 mini. It is also available on the iPhone 11 Pro and Pro Max, XS and XS Max, and...
iOS 16 battery percentage

Apple Limiting iOS 16 Beta 5 Battery Percentage Display to Select iPhones: Here Are the Supported Devices

Tuesday August 9, 2022 2:51 am PDT by
Apple this week brought back one of the most highly requested features from iOS users since the launch of the iPhone X in 2017: the ability to see your battery percentage directly in the status bar. Ever since the launch of the iPhone X with the notch, Apple has not allowed users to show their battery percentage directly in the status bar, forcing them to swipe down into Control Center to...
ios 16 battery indicator 2

Everything New in iOS 16 Beta 5: Battery Percentage in Status Bar, Find My Changes and More

Monday August 8, 2022 12:53 pm PDT by
Apple today seeded the fifth beta of iOS 16 to developers for testing purposes, introducing some small but notable changes to the iOS operating system. Subscribe to the MacRumors YouTube channel for more videos. We've rounded up everything new in the fifth beta below. Battery Percentage in Status Bar The battery icon in the status bar now displays the exact battery percent, a feature that ...
iphone 14 pro max camera bump compared lipilipsi 16 9

Bigger iPhone 14 Pro Max Camera Bump Shown Alongside iPhone 13 Pro Max

Monday August 8, 2022 4:33 am PDT by
The camera bump on the upcoming iPhone 14 Pro Max is expected to be the largest rear lens housing Apple has ever installed on its flagship smartphones, and a new photo offers a rare glimpse at just how prominent it is compared to Apple's predecessor device. iPhone 14 Pro Max dummy (left) vs iPhone 13 Pro Max All iPhone 14 models are expected to see upgrades to the Ultra Wide camera on the...
airpods pro black background

Beyond iPhone 14: Five Apple Products Expected to Launch Later This Year

Monday August 8, 2022 9:43 am PDT by
While the iPhone 14 and Apple Watch Series 8 are expected to be announced in September as usual, there are several more Apple products rumored to launch later this year, including new iPad and Mac models and more. Beyond the iPhone and Apple Watch, we've put together a list of five Apple products that are most likely to be unveiled by the end of 2022. Second-Generation AirPods Pro Apple...