Apple was granted a pair of patents on Tuesday that depict a mobile augmented reality system that can detect objects in the surrounding environment and overlay them with virtual information.

Picked up by AppleInsider, the first patent published by the U.S. Patent and Trademark Office is titled "Wearable information system having at least one camera" and describes an AR device with intelligent object recognition capabilities.

AR patent
Combining cameras, a screen, and a user interface, the device is described as being ideal for a head-mounted display, but the patent also suggests a use for the system in future smartphones, owing to the power efficient way in which it monitors the environment.

The device is described as having a default low-power scanning mode for normal operation, with high-power modes activated for short periods – when downloading and displaying AR content, for example. The rest of the patent gets quite technical as it details methods of optical tracking and determining camera orientation, two areas of AR that will prove particularly difficult to perfect.

Detecting, matching, and describing environmental features quickly and accurately are two challenges that Apple suggests could be overcome using a combination of dedicated hardware and pre-learned data. One aspect of such a system would use depth detection, achieved via dual-lens cameras similar to those found in the iPhone 7 Plus, which also utilizes depth mapping algorithms to produce Portrait Mode photos.

Apple AR patent
The second patent, titled "Method for representing virtual information in a real environment", details a way of labeling points in a topological real-world space while taking in environmental features such as object occlusion as the user moves around.

In one example, a user views an AR city map on a transparent display – or on a smartphone screen feeding a live image from a rear-facing camera – and pulls up information relating to nearby buildings or landmarks.

The patent goes on to suggest using geometry models, depth-sensing, positioning data and other advanced technologies to compensate for shortcomings in current visual modeling systems. The second patent also relies on a dual-camera system for creating a depth map of the immediate environment onto which the virtual data is superimposed.

The described technologies may or may not appear in future Apple products, but it's worth noting that both patents were originally filed for by German AR specialist Metaio just before Apple acquired the firm in 2015. Existing staff would have likely joined Apple's team dedicated to AR/VR research, to help explore how the technologies can be incorporated into future products.

Several times over the last year, Tim Cook has spoken of Apple's deep interest in augmented reality. "AR can be really great. We have been and continue to invest a lot in this. We're high on AR in the long run," he said in July.

Related Roundup: Apple Glasses
Tag: patent

Top Rated Comments

Abazigal Avatar
53 months ago
That's right, one more device to charge every night. :p
Well, my 6-port USB hub still has 2 ports available...
Score: 2 Votes (Like | Disagree)
gtg465x Avatar
53 months ago
Why is the dude in the first graphic touching himself.
Score: 1 Votes (Like | Disagree)
macTW Avatar
53 months ago
I'm excited for the future.
Score: 1 Votes (Like | Disagree)

Top Stories

jon prosser imac 2021colors

Prosser: 2021 iMac to Come in Five Colors, Apple Silicon Mac Pro to Resemble 'Stacked' Mac Minis

Wednesday February 24, 2021 7:26 am PST by
Hit-and-miss leaker Jon Prosser has today alleged that the upcoming 2021 iMac models will offer five color options, mirroring the colors of the fourth-generation iPad Air, and revealed a number of additional details about the Mac Pro with Apple Silicon. In a new video on YouTube channel FrontPageTech, Prosser explained that the redesigned iMacs will come featuring options for Silver, Space ...
2021 mbp sd slot feature2

Kuo: New MacBook Pro Models With HDMI Port and SD Card Reader to Launch Later This Year

Monday February 22, 2021 8:52 pm PST by
Apple plans to release two new MacBook Pro models equipped with an HDMI port and SD card reader in the second half of 2021, according to analyst Ming-Chi Kuo, who outlined his expectations in a research note obtained by MacRumors. The return of an SD card reader was first reported by Bloomberg's Mark Gurman last month. "We predict that Apple's two new MacBook Pro models in 2H21 will have...
m1 mac mini

M1 Mac Users Report Excessive SSD Wear

Tuesday February 23, 2021 7:07 am PST by
Over the past week, some M1 Mac users have been reporting alarming SSD health readings, suggesting that these devices are writing extraordinary amounts of data to their drives (via iMore). Across Twitter and the MacRumors forums, users are reporting that M1 Macs are experiencing extremely high drive writes over a short space of time. In what appear to be the most severe cases, M1 Macs are sai...
iphone 12 pro display video

BOE Rumored to Supply iPhone 13 Display Panels After iPhone 12 Failures

Monday February 22, 2021 9:54 am PST by
Display manufacturer BOE will be one of the main suppliers of OLED panels for iPhone 13 models, according to a new report today from Taiwan's Economic Daily News. BOE is said to be working with touch panel manufacturer General Interface Solution (GIS), part of the Hon Hai Group to develop OLED panels. Multiple iPhone 12 rumors suggested that BOE would supply some panels for the devices,...
flat mbp 14 inch feature yellow

Redesigned 14-Inch MacBook Pro Expected to Feature Brighter Mini-LED Display With Slimmer Bezels and More

Thursday February 25, 2021 7:48 am PST by
Apple plans to unveil new 14-inch and 16-inch MacBook Pro models with Mini-LED-backlit displays in the second half of this year, according to industry sources cited by Taiwanese supply chain publication DigiTimes. The report claims that Radiant Opto-Electronics will be the exclusive supplier of the Mini-LED backlight units, while Quanta Computer is said to be tasked with final assembly of the...
anker magsafe powercore battery pack

Anker Releases MagSafe-Compatible Battery Pack for iPhone 12 Lineup

Tuesday February 23, 2021 7:49 am PST by
Following rumors that Apple is working on a MagSafe battery pack for iPhone 12 models, popular accessory maker Anker has beaten Apple to the punch with the release of its PowerCore Magnetic 5K Wireless Power Bank. First previewed at CES 2021, the PowerCore battery pack magnetically attaches to the back of any iPhone 12 model and provides 5W of wireless charging. With a 5,000 mAh capacity,...
iPad Pro Mini LED

New iPad Pro and MacBook Models With Mini-LED Displays Again Rumored to Launch This Year

Monday February 22, 2021 9:32 pm PST by
Taiwanese company Ennostar will begin production of Mini-LED backlight units for an upcoming 12.9-inch iPad Pro in the late first quarter or second quarter of this year, according to industry sources cited by DigiTimes. Ennostar is a holding company that was jointly established in January 2021 by LED-related manufacturers Epistar and Lextar Electronics. Apple is expected to unveil the new ...
First Look Big Sur Feature2

Apple Releases macOS Big Sur 11.2.2 to Prevent MacBooks From Being Damaged by Third-Party Non-Compliant Docks

Thursday February 25, 2021 10:07 am PST by
Apple today released macOS Big Sur 11.2.2, the fourth update to the macOS Big Sur operating system that launched in November. macOS Big Sur 11.2.2 comes two weeks after the release of macOS Big Sur 11.2.1, a bug fix update. The new ‌‌‌‌macOS Big Sur‌‌‌ 11.2.2‌ update can be downloaded for free on all eligible Macs using the Software Update section of System Preferences....
steam apple logo

Valve Ordered to Give Apple Information on 436 Steam Games As Part of Epic Games Legal Case

Thursday February 25, 2021 1:50 am PST by
Valve, the makers behind popular game distribution platform Steam, will be forced to hand over aggregate historical sales, price, and other information on 436 games hosted on the store to Apple, as part of the Apple vs. Epic Games antitrust case. As reported in a paywalled report by Law360, during a virtual discovery hearing on Wednesday, U.S. Magistrate Judge Thomas S. Hixson ordered that...
scott forstall original iphone

Epic Games 'Attempting to Locate' Former iOS Chief Scott Forstall to Testify in Ongoing Legal Battle with Apple

Tuesday February 23, 2021 4:22 am PST by
As part of its ongoing legal battle, Epic Games is now seeking testimony from Apple's former iOS software chief, Scott Forstall. Forstall left Apple in 2013, in what was widely reported at the time to have been a forced exit, largely because of the botched early launch of Apple Maps. Since then, Forstall has kept a very low profile and has only made a few media appearances in the past few...