Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

Multiple reports suggest that Apple is developing an augmented reality headset as well as smart glasses, and a newly granted Apple patent may provide a clue about one of the potential features that could be included in the latter device.

patent
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent describing "systems and methods for gaze-driven recording of video."

Apple's patent notes that head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users, and envisions video from such a device being recorded for later playback or analysis. Specifically, it envisions such a system in smart glasses, where the use of built-in gaze-tracking sensors could provide an indication of where a person is currently looking, which could direct a built-in camera to record the scene where the user's eyes are trained, instead of simply recording what's in front of the user.

The patent envisions a system with one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to access the gaze data captured. This apparatus would apply a temporal filter to the gaze data to obtain a "smoothed gaze estimate," which would allow it to determine a region of interest based on the sensors. The system would then apply signal processing to the video based on the region of interest to obtain an enhanced recording that could mitigate the impact of rapid-movement eye saccades.

According to the patent, the system would consist of a pair of smart glasses and a separate connected device for power and storage, which could be interpreted to mean an iPhone, iPad or Mac.

record gaze tracking patent 2
The invention harks back to another patent filed last month in which an eye tracking system is used for detecting the position and movements of a user's eyes in smart glasses and other head-mounted displays.

In the latter patent, Apple envisages the eye tracking system being used to analyze the position and movements of the user's eyes, or to detect other information such as pupil dilation. In one example, identifying the point of the user's gaze allows for interaction with content shown on the near-eye display of the head-mounted display, while another application includes the creation of eye image animations that could be used for digital avatars visible to other users in a mixed-reality communal environment.

Of course, Apple files numerous patent applications every week, and many of these inventions do not see the light of day. But they do provide a glimpse into the kinds of technologies Apple is exploring for potential use in its upcoming AR/VR products, and based on the rumored launch roadmap for its mixed-reality headset and smart glasses, the company still has plenty of time to implement them.

The Information and Bloomberg have both said that Apple is working on smart glasses AND an AR/VR headset, with the headset to come out in first followed by the glasses. Apple analyst Ming-Chi Kuo believes the "mixed reality" headset will come out in 2022, with the Apple Glasses to follow in 2025. The headset is AR/VR, while Apple's glasses are augmented reality.

The headset is rumored to be similar to Facebook's Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable. JP Morgan believes the headset will look similar to other brands of VR headsets, featuring six lenses and an optical LiDAR scanner to map the wearer's environment. The headset will be targeted at the top end of the consumer market and will be more expensive than others on the market.

The glasses, meanwhile, which are reportedly in an early stage of development, are said to resemble high-end sunglasses with thick frames that house the battery and chips. Kuo expects the AR glasses to be marketed as an ‌iPhone‌ accessory and will primarily take a display role offloading computing, networking, and positioning to the ‌iPhone‌, with the glasses providing a mobile-first "optical see-through AR experience."

Patently Apple was first to report on the patent being granted today. Apple originally applied for the patent in December 2019.

Related Roundup: AR/VR Headset
Tag: Patent

Top Rated Comments

Salty Pirate Avatar
26 months ago
such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Score: 4 Votes (Like | Disagree)
Corsig Avatar
26 months ago

such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Is there another reason to use them?
Score: 4 Votes (Like | Disagree)
CarpalMac Avatar
26 months ago
Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.
Score: 2 Votes (Like | Disagree)
jjudson Avatar
26 months ago
A whole new bunch of glassholes coming soon...
Score: 2 Votes (Like | Disagree)
CJ Dorschel Avatar
26 months ago
Apple: ”Let’s discontinue the HomePod instead of updating it cause we’re losing money”

also Apple: “Pour more money into the Apple Car and Glasses!”

Over 1000 posts on the HomePod in a day or two and barely anything on the Apple Car and Glasses. Just sayin??‍♂️
Score: 1 Votes (Like | Disagree)
Corsig Avatar
26 months ago

Dem tatas?
I stand corrected
Score: 1 Votes (Like | Disagree)

Popular Stories

iOS 16

iOS 16.4 for iPhone Nearing Launch With These 5 New Features

Monday March 20, 2023 11:50 am PDT by
Apple says iOS 16.4 is coming in the spring, which began this week. In his Sunday newsletter, Bloomberg's Mark Gurman said the update should be released "in the next three weeks or so," meaning a public release is likely in late March or early April. iOS 16.4 remains in beta testing and introduces a handful of new features and changes for the iPhone. Below, we have recapped five new features ...
iPhone 15 Pro Buttons CAD Leak

iPhone 15 Pro Leak Reveals Unified Volume Button and Mute Button

Monday March 20, 2023 8:33 am PDT by
As previously rumored, the next-generation iPhone 15 Pro and iPhone 15 Pro Max will feature a unified volume button and a mute button, according to leaked CAD images shared in a video on the Chinese version of TikTok and posted to Twitter by ShrimpApplePro. Instead of separate buttons for volume up and volume down, the iPhone 15 Pro models are expected to have a single elongated button for...
voice isolation

iOS 16.4 Adds Voice Isolation for Cellular Phone Calls

Tuesday March 21, 2023 11:01 am PDT by
The iOS 16.4 update that is set to be released to the public in the near future includes voice isolation for cellular calls, according to notes that Apple shared today. Apple says that Voice Isolation will prioritize your voice and block out the ambient noise around you, making for clearer phone calls where you can better hear the person you're chatting with and vice versa. Voice...
airpodsd 3 purple 4

iOS 16.4 Seemingly References New AirPods and AirPods Case

Tuesday March 21, 2023 11:43 am PDT by
The iOS 16.4 release candidate version that was provided to developers today appears to hint at a new set of AirPods that could be coming in the near future. According to @aaronp613, the beta features references to AirPods that have a model number of A3048 and an AirPods case with a model number of A2968. There have been no rumors that new AirPods are on the horizon, and it is early for...
maxresdefault

Nothing Launches $149 Ear (2) Wireless Earbuds to Compete With AirPods Pro 2

Wednesday March 22, 2023 9:48 am PDT by
Nothing today announced the launch of its second-generation wireless earbuds, the Nothing Ear (2), which offer many of the same features as Apple's AirPods Pro 2 at a lower price point. We went hands-on with the Ear (2) earbuds to see whether they're a viable alternative to the AirPods Pro 2 for those who want to save some cash. The Ear (2) earbuds are the successor to the Nothing Ear (1),...
iOS 16

Apple Seeds Release Candidate Versions of iOS 16.4 and iPadOS 16.4 to Developers [Update: Public RC Available]

Tuesday March 21, 2023 10:25 am PDT by
Apple today seeded the release candidate versions of upcoming iOS 16.4 and iPadOS 16.4 updates to developers for testing purposes, with the software coming a week after the launch of the fourth betas. The RCs mark the final version of the software that will be provided to the public in the near future. Registered developers are able to download the iOS 16.4 and iPadOS 16.4 updates...
apple park at night 1

Apple 'Tracking Employee Attendance' in Crackdown on Remote Working

Thursday March 23, 2023 3:41 am PDT by
Apple is tracking the attendance of its employees at offices using badge records in order to ensure they are coming in at least three times a week, according to Platformer's Zoë Schiffer. Since April 2022, Apple employees have been operating on a hybrid home/office work policy as part of a gradual return strategy following the pandemic, with staff required to work from the office at least...
iphone 14 pro max deep purple feature purple

iPhone 15 Pro Rumor Recap: 10 New Features and Changes to Expect

Thursday March 23, 2023 6:42 am PDT by
While the iPhone 15 series is still around six months away from launching, there have already been plenty of rumors about the devices. Many new features and changes have been rumored for the iPhone 15 Pro and iPhone 15 Pro Max in particular. Below, we have recapped 10 changes rumored for iPhone 15 Pro models that are not expected to be available on the standard iPhone 15 and iPhone 15 Plus:A1...