Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

Multiple reports suggest that Apple is developing an augmented reality headset as well as smart glasses, and a newly granted Apple patent may provide a clue about one of the potential features that could be included in the latter device.

patent
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent describing "systems and methods for gaze-driven recording of video."

Apple's patent notes that head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users, and envisions video from such a device being recorded for later playback or analysis. Specifically, it envisions such a system in smart glasses, where the use of built-in gaze-tracking sensors could provide an indication of where a person is currently looking, which could direct a built-in camera to record the scene where the user's eyes are trained, instead of simply recording what's in front of the user.

The patent envisions a system with one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to access the gaze data captured. This apparatus would apply a temporal filter to the gaze data to obtain a "smoothed gaze estimate," which would allow it to determine a region of interest based on the sensors. The system would then apply signal processing to the video based on the region of interest to obtain an enhanced recording that could mitigate the impact of rapid-movement eye saccades.

According to the patent, the system would consist of a pair of smart glasses and a separate connected device for power and storage, which could be interpreted to mean an iPhone, iPad or Mac.

record gaze tracking patent 2
The invention harks back to another patent filed last month in which an eye tracking system is used for detecting the position and movements of a user's eyes in smart glasses and other head-mounted displays.

In the latter patent, Apple envisages the eye tracking system being used to analyze the position and movements of the user's eyes, or to detect other information such as pupil dilation. In one example, identifying the point of the user's gaze allows for interaction with content shown on the near-eye display of the head-mounted display, while another application includes the creation of eye image animations that could be used for digital avatars visible to other users in a mixed-reality communal environment.

Of course, Apple files numerous patent applications every week, and many of these inventions do not see the light of day. But they do provide a glimpse into the kinds of technologies Apple is exploring for potential use in its upcoming AR/VR products, and based on the rumored launch roadmap for its mixed-reality headset and smart glasses, the company still has plenty of time to implement them.

The Information and Bloomberg have both said that Apple is working on smart glasses AND an AR/VR headset, with the headset to come out in first followed by the glasses. Apple analyst Ming-Chi Kuo believes the "mixed reality" headset will come out in 2022, with the Apple Glasses to follow in 2025. The headset is AR/VR, while Apple's glasses are augmented reality.

The headset is rumored to be similar to Facebook's Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable. JP Morgan believes the headset will look similar to other brands of VR headsets, featuring six lenses and an optical LiDAR scanner to map the wearer's environment. The headset will be targeted at the top end of the consumer market and will be more expensive than others on the market.

The glasses, meanwhile, which are reportedly in an early stage of development, are said to resemble high-end sunglasses with thick frames that house the battery and chips. Kuo expects the AR glasses to be marketed as an ‌iPhone‌ accessory and will primarily take a display role offloading computing, networking, and positioning to the ‌iPhone‌, with the glasses providing a mobile-first "optical see-through AR experience."

Patently Apple was first to report on the patent being granted today. Apple originally applied for the patent in December 2019.

Related Roundup: Apple Glasses
Tag: patent

Top Rated Comments

Salty Pirate Avatar
4 weeks ago
such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Score: 4 Votes (Like | Disagree)
Corsig Avatar
4 weeks ago

such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Is there another reason to use them?
Score: 4 Votes (Like | Disagree)
WBRacing Avatar
4 weeks ago
Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.
Score: 2 Votes (Like | Disagree)
jjudson Avatar
4 weeks ago
A whole new bunch of glassholes coming soon...
Score: 2 Votes (Like | Disagree)
CJ Dorschel Avatar
4 weeks ago
Apple: ”Let’s discontinue the HomePod instead of updating it cause we’re losing money”

also Apple: “Pour more money into the Apple Car and Glasses!”

Over 1000 posts on the HomePod in a day or two and barely anything on the Apple Car and Glasses. Just sayin??‍♂️
Score: 1 Votes (Like | Disagree)
Corsig Avatar
4 weeks ago

Dem tatas?
I stand corrected
Score: 1 Votes (Like | Disagree)

Top Stories

siir apple event april 20

Siri Reveals Apple Event Planned for Tuesday, April 20

Tuesday April 13, 2021 12:04 am PDT by
Siri has apparently prematurely revealed that Apple plans to hold an event on Tuesday, April 20, where the company is expected to reveal brand new iPad Pro models and possibly its long-awaited AirTags trackers. Subscribe to the MacRumors YouTube channel for more videos. Upon being asked "When is the next Apple Event," Siri is currently responding with, "The special event is on Tuesday, April...
apple event spring loaded

Apple's 'Spring Loaded' Event Officially Announced for Tuesday, April 20

Tuesday April 13, 2021 9:04 am PDT by
Following an overnight leak by Siri, Apple today officially announced that it will be holding a special "Spring Loaded" event on Tuesday, April 20 at 10:00 a.m. Pacific Time at the Steve Jobs Theater on the Apple Park campus in Cupertino, California. As with all of Apple's 2020 events, the April 2021 event will be a digital-only gathering with no members of the media invited to attend in...
apple event hashflag

Twitter Hashflag for April 20 Apple Event Goes Live

Tuesday April 13, 2021 2:21 pm PDT by
Following the overnight Siri leak and subsequent announcement that Apple will hold a media event on Tuesday, April 20, a new Twitter hashflag has appeared to help provide visibility for the event on the platform. For the last several recent events, Apple has utilized hashflags, which are little icons next to hashtags on Twitter, as a way to market its events. The company first started the...
iphone12cameras

Kuo: 2022 iPhones to Feature 48-Megapixel Camera, 8K Video, and 6.1 and 6.7" Sizes With No 5.4" Mini Option

Tuesday April 13, 2021 10:45 pm PDT by
The upcoming 2022 iPhone lineup will feature two 6.1-inch devices and two 6.7-inch devices, with no mini-sized 5.4-inch iPhone, well-respected Apple analyst Ming-Chi Kuo said in a note to investors that was seen by MacRumors. Two of the iPhones will be high-end models and two of the iPhones will be lower-end models, similar to the current iPhone 12 lineup. Apple introduced the 5.4-inch...
macos catalina serial number

Apple Preparing Rollout of New Randomized Product Serial Numbers Ahead of 'Spring Loaded' Event

Wednesday April 14, 2021 2:08 am PDT by
Apple is advising its authorized premium resellers and dealers to prepare for new products with 10 and 12 digital serial numbers, days ahead of when it's expected to reveal a slew of new products. MacRumors previously reported that Apple plans to switch to randomized serial numbers for future products starting in early 2021. The company now seems to be preparing for that roll-out, telling...
duanrui iphone13 notch samples

More Leaked iPhone 13 Samples Show Smaller Notch, Repositioned Earpiece and Front Camera

Wednesday April 14, 2021 1:06 am PDT by
Leaker known as "DuanRui" has today shared an image of two iPhone 13 "film samples," which show the same rumored smaller notch design coming to the iPhone 13 series that we've seen from other sources. In past tweets, DuanRui has accurately leaked the correct names of the iPhone 12 models and an iPad Air 4 manual revealing its new design, so there's good reason to think this leak is credible, ...
Google maps feaure green

Google Maps App for iOS Finally Updated After Four Months

Monday April 12, 2021 10:03 am PDT by
Following the completed rollout of App Privacy labels for its App Store apps, Google today updated the Google Maps app for the first time in four months. Apple in December began requiring all new app submissions and app updates to include App Privacy labels, detailing the data that is collected by the app so consumers know what they're sharing. Google didn't begin implementing App Privacy ...
iPad Pro Mini LED

Mini-LED iPad Pro Expected at Apple Event on April 20 as Production Ramps Up

Tuesday April 13, 2021 9:53 am PDT by
Apple today announced that it will be hosting a virtual event on Tuesday, April 20 at 10 a.m. Pacific Time, and one of the new products expected to be unveiled at the event is a new iPad Pro with a Mini-LED display. In a brief snippet shared with paid subscribers, Taiwanese supply chain news website DigiTimes today reported that Apple supplier Ennostar has substantially improved its yield...
apple event particularly innovative article

Gurman: Apple's 'Spring Loaded' Event Won't Feature Anything 'Particularly Innovative'

Thursday April 15, 2021 1:30 am PDT by
Bloomberg's highly-respected Mark Gurman says that he expects nothing "particularly innovative" or "extraordinary" to launch at Apple's "Spring Loaded" event next week, Tuesday, April 20. Gurman made the remarks during an interview for Bloomberg Technology, in which he reaffirmed that Apple will launch a new 11-inch and 12.9-inch iPad Pro, with the higher-end model featuring a brand new...
spotify car thing 1

Spotify Announces the 'Car Thing' as its First Hardware Device

Tuesday April 13, 2021 7:03 am PDT by
Spotify has today announced its first hardware device, the "Car Thing," which is an in-car dash-mounted music and podcast player (via TechCrunch). Spotify is looking to provide a product for customers who want a "more seamless" and personalized in-car listening experience, especially in the large number of cars that do not support modern in-car infotainment systems. The Car Thing is aimed ...