Apple Vision Pro 'Visual Search' Feature Can Identify Items, Copy Printed Text, Translate and More
The Apple Vision Pro headset's visionOS operating system includes a feature called "Visual Search," which sounds like it is similar to the Visual Look Up feature on the iPhone and the iPad.
With Visual Search, users can use the Vision Pro headset to get information about an item, detect and interact with text in the world around them, copy and paste printed text from the real world into apps, translate text between 17 different languages, and more.
Real world text that includes contact information, webpages, and unit conversions and similar information can be acted upon in visionOS. So, for example, if a printed handout has a website link in it, you can scan the link with the Vision Pro, opening up a Safari window to view the website. Or, if a recipe calls for grams and you need ounces, you can convert using the headset.
Real-time text translation will also be useful for traveling and other instances when you might want to quickly translate what you're seeing in the real world. The Apple Vision Pro headset will be able to automatically detect text and documents, similar to how the iPhone can detect text in photos and allow it to be interacted with.
The Visual Search function was found in visionOS by Steve Moser. visionOS can be accessed through the latest Xcode beta at the moment, as Apple released the first version of the software earlier today.
Popular Stories
Apple is planning some of the "biggest iOS and macOS redesigns in its history," according to Bloomberg's Mark Gurman.
In his Power On newsletter today, Gurman reiterated that iOS 19 will have a visionOS-like design with more transparent interfaces:The new interfaces will adopt the design principles introduced in visionOS, the software for Apple's Vision Pro headset. That includes greater...
If you've been following iPhone rumors over the last few years, you may remember reading reports that Apple flirted with the idea of introducing a super high-end "Ultra" model that would either replace its Pro Max device or sit above it in Apple's smartphone hirearchy. These reports appeared in the pre-launch iPhone 15 and iPhone 16 rumor cycles, but ultimately came to nothing. Now though, the...
While the so-called "iPhone 17 Air" is not expected to launch until September, there are already plenty of rumors about the ultra-thin device.
Overall, the "iPhone 17 Air" sounds like a mixed bag. While the device is expected to have an impressively thin and light design, rumors indicate it will have some compromises compared to iPhone 17 Pro models, including only a single rear camera, a...
Apple prototyped a larger ultra-slim iPhone 17 Air with a 6.9-inch display, but ultimately decided not to go ahead with the device because of fears that it could be susceptible to bending, according to a new report.
Bloomberg reporter Mark Gurman, writing in his latest Power On newsletter:
When it first started work on the phone, it prototyped a device with a 6.9-inch screen — matching...
In an investor research note today with British bank Barclays, analyst Tim Long said Apple's first foldable iPhone could have a starting price in the $2,300 range in the United States, which would make it by far the most expensive iPhone model ever.
If the first foldable iPhone starts at $2,299, that means it would cost nearly twice as much as the iPhone 16 Pro Max, which starts at $1,199.
...
Bloomberg's Mark Gurman today shared some new details about the rumored iPhone 17 Air.
In his Power On newsletter, Gurman said he was told that the device may start at roughly $899 in the U.S., which means that it would occupy the same price point as the iPhone 16 Plus. This would make sense, as it has been widely rumored that the Air model will take over the Plus model's spot in the iPhone...
Apple today launched a new app called Surveyor, which is designed to allow users to collect data like images of street signs and roadside details to improve Apple Maps.
The app is not public facing and appears to be for use with companies that Apple partners with to assign mapping tasks. Downloading the app and opening it up directs users to "Open Partner App" to choose a task. Tapping on...
Apple considered launching the iPhone 17 Air without a USB-C charging port, according to Bloomberg's Mark Gurman.
In his Power On newsletter today, Gurman said that while Apple ultimately decided against making the iPhone 17 Air its first iPhone model without a charging port, the idea is still on the table for future iPhone models.
He said the iPhone 17 Air will "foreshadow a move to...
