iOS 15's Live Text Feature Lets You Digitize Written Notes, Call a Number on a Sign, Translate a Menu, and Much More

In iOS 15, Apple is introducing a new feature called Live Text that can recognize text when it appears in your camera's viewfinder or in a photo you've taken and let you perform several actions with it.

Apple iPadPro iPadOS15 photos LiveText 060721 big
For example, Live Text allows you to capture a phone number from a storefront with the option to place a call, or look up a location name in Maps to get directions. It also incorporates optical character recognition, so you can search for a picture of a handwritten note in your photos and save it as text.

Live Text's content awareness extends to everything from QR codes to emails that appear in pictures, and this on-device intelligence feeds into Siri suggestions, too.

ios15 live text
For instance, if you take a picture that shows an email address and then open the Mail app and start composing a message, ‌Siri‌'s keyboard suggestions will offer up the option to add "Email from Camera" to the To field of your message.

Other Live Text options include the ability to copy text from the camera viewfinder or photos for pasting elsewhere, share it, look it up in the dictionary, and translate it for you into English, Chinese (both simplified and traditional), French, Italian, German, Spanish, or Portuguese.

live text translate
It can even sort your photos by location, people, scene, objects, and more, by recognizing the text in pictures. For example, searching for a word or phrase in Spotlight search will bring up pictures from your Camera Roll in which that text occurs.

Live Text works in Photos, Screenshot, Quick Look, and Safari and in live previews with Camera. In the Camera app, it's available whenever you point your iPhone's camera at anything that displays text, and is indicated by a small icon that appears in the bottom right corner whenever textual content is recognized in the viewfinder. Tapping the icon lets you tap recognized text and perform an action with it. A similar icon appears in the ‌Photos‌ app when you're viewing a shot image.

visual look up ios 15
In another neural engine feature, Apple is introducing something called Visual Look Up that lets you take photos of objects and scenes to get more information from them. Point your ‌iPhone‌'s camera at a piece of art, flora, fauna, landmarks, or books, and the Camera will indicate with an icon that it recognizes the content and has relevant ‌Siri‌ Knowledge that can add context.

Since Live Text relies heavily on Apple's neural engine, the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an ‌iPhone‌ X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.

The ‌iOS 15‌ beta is currently in the hands of developers, with a public beta set to be released next month. The official launch of ‌iOS 15‌ is scheduled for the fall.

Related Roundups: iOS 15, iPadOS 15
Related Forum: iOS 15

Top Rated Comments

ruka.snow Avatar
8 months ago
Let's see them digitise doctors notes and prescriptions.
Score: 19 Votes (Like | Disagree)
Unggoy Murderer Avatar
8 months ago

They don’t want to. Hence why they aren’t enabling it for the intel macs. Just a cash grab as they hope everybody will sell their newish intel Mac, which they paid handsomely for, at a huge loss and then go and pay handsomely again for a new Apple silicon Mac.
I'm sure Apple could get it running on the A11, however people apparently forget there will be trade-offs to enable that to happen. What if enabling it on the A11 halved the battery life, would you want that, would that be a trade-off you'd be willing to make? Or perhaps the performance of the phone drops by 25%, how about that?

Apple can't (and shouldn't be expected to) support older hardware with every single new feature that's released. Apple do a good job of supporting older hardware far better than the competitors do (the iPhone 6s, a 2015 phone, gets iOS 15), they'll get the benefit of at least some new features and improvements.

I'm sure some people will want to upgrade their hardware to support the latest software additions, but I (and I'm sure like a lot of other folks like me with newish machines), won't be bothered enough. Nice features, but I won't lose sleep over not having them.


Apple isn’t the company from heaven as most seem to think.
Of course they're not - Apple is a money making enterprise.
Score: 14 Votes (Like | Disagree)
EmotionalSnow Avatar
8 months ago

Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
The later phones have an improved Neural Engine. I'm guessing that these are required in order to ensure good performance.
Score: 8 Votes (Like | Disagree)
Unggoy Murderer Avatar
8 months ago

The old World Lens app (and subsequent iterations after Google acquired it) were able to both capture and translate text in real-time using substantially slower hardware.

Live text would just be this, without the translation (whether on device or not). I can’t see why the neural engine would be an absolute requirement, even if you try to make a ‘performance’ argument.
Try using Word Lens for any sustained period - it absolutely hammered your battery. And just to be clear, that isn't a criticism of Word Lens - it was an exceptional app for its time, it just couldn't take advantage of optimised silicon.

The performance argument is *the* argument. Apple more than likely could implement it on all hardware, but what it it caused your battery life to be half of usual, or Safari and camera apps to be considerably slower while it runs CPU-bound neural networks on every image downloaded, or every camera frame?

Also worth noting, text recognition isn't a trivial task, especially with handwriting.
Score: 8 Votes (Like | Disagree)
farewelwilliams Avatar
8 months ago
implemented better than android
Score: 5 Votes (Like | Disagree)
LukeHarrison Avatar
8 months ago

Since Live Text relies heavily on Apple's neural engine, so the feature is only available on iPhones and iPads with at least an A12 Bionic or better chip, which means if you have an iPhone X or earlier model or anything less than an iPad mini (5th generation), iPad Air (2019, 3rd generation), or iPad (2020, 8th generation), then unfortunately you won't have access to it.
Forgive me if I'm wrong, but wasn't the A11 the first chip with a Neural Engine? So surely they could make this work on the X (and even the 8 and 8 Plus) if they REALLY wanted to.
Score: 5 Votes (Like | Disagree)

Related Stories

maxresdefault

Apple Shares 10 Helpful iPhone Tips and Tricks

Wednesday December 22, 2021 12:38 pm PST by
Apple today shared a new video on YouTube demonstrating 10 helpful iPhone tips and tricks that some users may not have known about. An overview of each of the 10 tips and tricks:If you accidentally enter a wrong digit in the Calculator app, you can swipe left or right at the top of the screen to erase the digit. Starting with iOS 15, you can pin content like web links in a Messages...
telegram reactions

Telegram Updated With Message Reactions and Several Other Features

Thursday December 30, 2021 6:52 am PST by
Telegram today announced it has updated its iPhone and iPad app with several new features, including message reactions, translation, hidden text, additional interactive emoji options, themed QR codes for users and channels, and more. With the addition of reactions, users can now double tap on a message bubble and a small thumbs-up emoji will appear at the bottom of that message. You can...
General iOS 15

Apple Releases iOS 15.1 and iPadOS 15.1 With SharePlay, ProRes, Auto Macro Toggle, Vaccine Cards in Wallet and More

Monday October 25, 2021 10:03 am PDT by
Apple today released iOS and iPadOS 15.1, the first major updates to the iOS 15 operating system that was released in September 2021. iOS 15.1 comes one month after the launch of iOS 15. The iOS and iPadOS 15.1 updates can be downloaded for free and the software is available on all eligible devices over-the-air in the Settings app. To access the new software, go to Settings > General >...
iOS 15 icon on phone

Apple Releases iOS 15 and iPadOS 15 With Safari Updates, Focus Mode, Live Text, iCloud+, System-Wide Translate, On-Device Siri and More

Monday September 20, 2021 10:04 am PDT by
Apple today released iOS 15 and iPadOS 15, the newest operating system updates designed for the iPhone, iPad, and iPod touch. As with all of Apple's software updates, iOS and iPadOS 15 can be downloaded at no cost. iOS 15 is available on the iPhone 6s and later while iPadOS 15 is available on the iPad Air 2 and later. The new software can be downloaded on eligible devices over-the-air by...
General iOS 15

Apple Seeds Second Release Candidate Version of iOS 15.2 for iPhone 13 Models to Developers and Public Beta Testers

Friday December 10, 2021 10:12 am PST by
Apple today seeded the second RC version of an upcoming iOS 15.2 update to developers and public beta testers, less than a week after seeding the first RC and six weeks after the launch of iOS 15.1. iOS 15.2 beta can be downloaded through the Apple Developer Center or over the air after the proper profile has been installed on an iPhone or an iPad. Public beta testers can also download the...
iPhone 15 years

15 Features the Original iPhone Didn't Have When First Announced

Friday January 7, 2022 4:34 pm PST by
Sunday will mark the 15th anniversary of Steve Jobs introducing the original iPhone at the Macworld Expo in San Francisco, and while the device revolutionized the smartphone industry, it's easy to forget how much the iPhone has improved since then. To celebrate the upcoming occasion, we've put together a list of 15 features the original iPhone didn't have when it was first announced. App...
iOS 16 mock for article

With WWDC Six Months Away, What Do You Want to See in iOS 16?

Thursday January 6, 2022 1:22 pm PST by
The Worldwide Developers Conference, Apple's annual five-day-long conference to announce updates to its major operating systems, services, and sometimes hardware, is six months away. At this year's conference, we expect Apple to announce iOS 16, iPadOS 16, macOS 13, watchOS 9, and tvOS 16. iOS is Apple's most important operating system as it runs on the iPhone and shares many features with...
General iOS 15

Apple Seeds Release Candidate Versions of iOS 15.2 and iPadOS 15.2 to Developers

Tuesday December 7, 2021 10:05 am PST by
Apple today seeded the RC versions of upcoming iOS and iPadOS 15.2 updates to developers for testing purposes, less than a week after seeding the fourth betas and six weeks after the launch of iOS 15.1 and iPadOS 15.1. iOS and iPadOS 15.2 can be downloaded through the Apple Developer Center or over the air after the proper profile has been installed on an iPhone or an iPad. iOS and iPadOS ...

Popular Stories

Upcoming Products 2022 Feature

Gurman: Apple Preparing 'Widest Array of New Hardware Products in Its History' for Fall

Sunday January 23, 2022 10:32 am PST by
Apple is working on a number of new products that are set to launch this fall, and Bloomberg's Mark Gurman says that it will be "the widest array" of new devices that Apple has introduced in its history. In his latest "Power On" newsletter, Gurman explains that Apple is working on four new flagship iPhones (iPhone 14, iPhone 14 Max, iPhone 14 Pro, and iPhone 14 Pro Max), an updated low-end Ma...
macbook pro 14 16 2021

Three Months After Launch, Apple Still Struggling to Meet Demand for Redesigned 14-Inch and 16-Inch MacBook Pro

Monday January 24, 2022 7:12 am PST by
Three months after their launch, the 14-inch and 16-inch MacBook Pros continue to experience high demand and seemingly short supply, with shipping dates for both models stretching into multiple weeks in several of Apple's key markets. In the United States, the baseline 14-inch MacBook Pro with the M1 Pro chip is estimated to ship in three to four weeks, promising an arrival by at least...
General Dropbox Feature

macOS 12.3 Will Include Cloud Storage Changes Affecting Dropbox and OneDrive

Tuesday January 25, 2022 3:31 pm PST by
Dropbox today announced that users who update to macOS 12.3 once that software version becomes available may temporarily encounter issues with opening online-only files in some third-party apps on their Mac. In a support document and an email to customers, Dropbox said it is actively working on full support for online-only files on macOS 12.3 and will begin rolling out an updated version of...
Questionable Design Decisions

Apple's Most Questionable Design Decisions in Recent Memory

Sunday January 23, 2022 2:59 am PST by
Apple has always emphasized the depth of thought that goes into the design of its products. In the foreword to Designed by Apple in California, a photo book released by the company in 2016, Jony Ive explains how Apple strives "to define objects that appear effortless" and "so simple, coherent and inevitable that there could be no rational alternative." But every once in a while even Apple...
Apple Watch Red Yellow Green Feature 1

Apple Launches Black Unity Braided Solo Loop With 'Unity Lights' Watch Face

Wednesday January 26, 2022 6:05 am PST by
Apple today announced the Black Unity Braided Solo Loop for the Apple Watch, as well as a new downloadable watch face, to celebrate Black History Month. Following the launch of the limited edition Black Unity Apple Watch Series 6 and Sport Band in 2021, Apple today launched the Black Unity Braided Solo Loop as part of its celebrations for Black History Month this year.Apple is launching a...
att gigabit internet

AT&T Bringing $180/Month 5-Gigabit Internet to 70 Cities

Monday January 24, 2022 9:20 am PST by
AT&T today announced the launch of upgraded AT&T Fiber plans, which support speeds of up to 5 Gigabits for some customers. There are two separate plans, one "2 GIG" plan and one "5 GIG" plan, available to new and existing AT&T Fiber subscribers. According to AT&T, the new plans are available to nearly 5.2 million customers across 70 metro areas including Los Angeles, Atlanta, Chicago, San...
intel vs m1 max chip purple

Benchmarks Confirm Intel's Latest Core i9 Chip Outperforms Apple's M1 Max With Several Caveats

Wednesday January 26, 2022 8:56 am PST by
Benchmark results have started to surface for MSI's new GE76 Raider, one of the first laptops to be powered by Intel's new 12th-generation Core i9 processor. Intel previously said that its new high-end Core i9 processor is faster than Apple's M1 Max chip in the 16-inch MacBook Pro and, as noted by Macworld, early Geekbench 5 results do appear to confirm this claim, but there are several...
ios 15

Apple Releases iOS 15.3 and iPadOS 15.3 With Fix for Safari Bug That Leaks Browsing Activity

Wednesday January 26, 2022 10:00 am PST by
Apple today released iOS 15.3 and iPadOS 15.3, the third major updates to the iOS and iPadOS 15 operating systems that were released in September 2021. iOS and iPadOS 15.3 come almost two weeks after the release of iOS and iPadOS 15.2.1, minor bug fix updates. The iOS 15.3 and iPadOS 15.3 updates can be downloaded for free and the software is available on all eligible devices over-the-air in ...