Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

Multiple reports suggest that Apple is developing an augmented reality headset as well as smart glasses, and a newly granted Apple patent may provide a clue about one of the potential features that could be included in the latter device.

patent
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent describing "systems and methods for gaze-driven recording of video."

Apple's patent notes that head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users, and envisions video from such a device being recorded for later playback or analysis. Specifically, it envisions such a system in smart glasses, where the use of built-in gaze-tracking sensors could provide an indication of where a person is currently looking, which could direct a built-in camera to record the scene where the user's eyes are trained, instead of simply recording what's in front of the user.

The patent envisions a system with one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to access the gaze data captured. This apparatus would apply a temporal filter to the gaze data to obtain a "smoothed gaze estimate," which would allow it to determine a region of interest based on the sensors. The system would then apply signal processing to the video based on the region of interest to obtain an enhanced recording that could mitigate the impact of rapid-movement eye saccades.

According to the patent, the system would consist of a pair of smart glasses and a separate connected device for power and storage, which could be interpreted to mean an iPhone, iPad or Mac.

record gaze tracking patent 2
The invention harks back to another patent filed last month in which an eye tracking system is used for detecting the position and movements of a user's eyes in smart glasses and other head-mounted displays.

In the latter patent, Apple envisages the eye tracking system being used to analyze the position and movements of the user's eyes, or to detect other information such as pupil dilation. In one example, identifying the point of the user's gaze allows for interaction with content shown on the near-eye display of the head-mounted display, while another application includes the creation of eye image animations that could be used for digital avatars visible to other users in a mixed-reality communal environment.

Of course, Apple files numerous patent applications every week, and many of these inventions do not see the light of day. But they do provide a glimpse into the kinds of technologies Apple is exploring for potential use in its upcoming AR/VR products, and based on the rumored launch roadmap for its mixed-reality headset and smart glasses, the company still has plenty of time to implement them.

The Information and Bloomberg have both said that Apple is working on smart glasses AND an AR/VR headset, with the headset to come out in first followed by the glasses. Apple analyst Ming-Chi Kuo believes the "mixed reality" headset will come out in 2022, with the Apple Glasses to follow in 2025. The headset is AR/VR, while Apple's glasses are augmented reality.

The headset is rumored to be similar to Facebook's Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable. JP Morgan believes the headset will look similar to other brands of VR headsets, featuring six lenses and an optical LiDAR scanner to map the wearer's environment. The headset will be targeted at the top end of the consumer market and will be more expensive than others on the market.

The glasses, meanwhile, which are reportedly in an early stage of development, are said to resemble high-end sunglasses with thick frames that house the battery and chips. Kuo expects the AR glasses to be marketed as an ‌iPhone‌ accessory and will primarily take a display role offloading computing, networking, and positioning to the ‌iPhone‌, with the glasses providing a mobile-first "optical see-through AR experience."

Patently Apple was first to report on the patent being granted today. Apple originally applied for the patent in December 2019.

Related Roundup: Apple Glasses
Tag: Patent

Top Rated Comments

Salty Pirate Avatar
15 months ago
such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Score: 4 Votes (Like | Disagree)
Corsig Avatar
15 months ago

such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Is there another reason to use them?
Score: 4 Votes (Like | Disagree)
CarpalMac Avatar
15 months ago
Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.
Score: 2 Votes (Like | Disagree)
jjudson Avatar
15 months ago
A whole new bunch of glassholes coming soon...
Score: 2 Votes (Like | Disagree)
CJ Dorschel Avatar
15 months ago
Apple: ”Let’s discontinue the HomePod instead of updating it cause we’re losing money”

also Apple: “Pour more money into the Apple Car and Glasses!”

Over 1000 posts on the HomePod in a day or two and barely anything on the Apple Car and Glasses. Just sayin??‍♂️
Score: 1 Votes (Like | Disagree)
Corsig Avatar
15 months ago

Dem tatas?
I stand corrected
Score: 1 Votes (Like | Disagree)

Related Stories

apple ar headset concept 2

Apple's 'realityOS' for Rumored AR/VR Headset Referenced in Source Code

Wednesday February 9, 2022 2:12 am PST by
New references to "realityOS," the AR/VR operating system that will run Apple's rumored mixed reality headset, have been found in App Store upload logs and Apple open source code this morning. Concept render based on purported leaked information by Ian Zelbo Apple is working on at least two AR projects that include an augmented reality headset set to be released in late 2022 or 2023, followed ...
apple ar headset concept 1

Apple's AR/VR Headset Could Come With a Monthly Subscription Service

Friday February 18, 2022 2:27 am PST by
Apple's long-rumored augmented reality (AR) and virtual reality (VR) headset could come with a monthly subscription service, according to a report from TrendForce. Concept render based on purported leaked information by Ian Zelbo In a report forecasting the growing sales of AR and VR devices from major tech companies in 2022, TrendForce said that Apple's headset will likely be...
ipad augmented reality

Apple Looking to Create an 'Entirely New Application Paradigm' for AR and VR

Wednesday December 8, 2021 1:33 am PST by
Apple is looking to create an "entirely new application paradigm" for augmented and virtual reality according to a new job listing, highlighting the company's growing ambitions in the AR and VR space as it ramps up the development of its next-generation products. The job posting initially shared by Hayden Lee, an AR/VR software engineering manager at Apple, describes Apple as looking for an ...
apple ar headset concept 1

Apple AR/VR Headset Potentially Delayed Until 2023 Amid Development Problems

Friday January 14, 2022 8:10 am PST by
Apple's long-rumored AR/VR headset could be delayed until 2023 after being beset with issues during the development process, according to Bloomberg. Concept render based on purported leaked information by Ian Zelbo Citing people familiar with Apple's plans, the report claims that Apple originally planned to launch the headset in 2021 and ship it this year. The company then set its sights on...
apple mixed reality headset mockup feature orange

Apple's Upcoming AR/VR Headset to Feature Three Displays

Tuesday January 4, 2022 10:29 am PST by
Apple's mixed reality headset that's set to launch in 2022 will be equipped with three displays, according to a research report shared today by display analyst Ross Young. The display configuration will include two micro OLED displays along with one AMOLED panel, with Sony set to supply the micro OLED displays that Apple will use. The micro OLED displays will be the main displays for the...
apple ar headset concept 2

Apple AR/VR Headset Again Rumored to Feature Micro OLED Displays While Samsung Readies 'Hologram' AR Device Competitor

Thursday February 24, 2022 6:28 am PST by
Apple's mixed-reality headset has again been rumored to feature advanced micro OLED displays, Korea's ET News reports. Concept render based on purported leaked information by Ian Zelbo Micro-OLED displays are built directly on to chip wafers rather than a glass substrate, which results in displays that are thinner, smaller, and more power efficient. They allow for pixel sizes in the range of...
14 16 inch 2021 mbps back to back feature orange

Apple Using MacBook Pro Parts to Achieve Mac-Like Experience on New Headset

Friday January 28, 2022 3:08 am PST by
Apple appears to be bringing aspects of the redesigned MacBook Pro to its long-rumored mixed-reality headset to facilitate a Mac-level experience, according to recent reports. Last year, TF International Securities analyst Ming-Chi Kuo said that Apple's mixed-reality headset will feature two independent chips, including one for Mac-level computing and one to manage the device's array of...
General Apps FaceTime

Gurman: FaceTime for Apple AR/VR Headset Could Rely on Memojis and SharePlay

Sunday February 13, 2022 7:34 am PST by
Memojis and SharePlay could be central to the FaceTime experience on Apple's long-rumored mixed-reality headset, according to Bloomberg's Mark Gurman. Gurman, who often reveals accurate insights into Apple's plans, has previously said that Apple's mixed-reality headset will focus on gaming, media consumption, and communication. The headset itself is rumored to run "rOS" or "realityOS,"...

Popular Stories

RIP iPod Feature

RIP iPod: A Look Back at Apple's Iconic Music Player Over the Years

Friday May 13, 2022 2:25 pm PDT by
Apple earlier this week announced the discontinuation of the iPod touch, and because it was the last iPod still available for purchase, its sunsetting effectively marks the end of the entire iPod lineup. To send the iPod on its way, we thought it would be fun to take a look back at some of the most notable iPod releases over the last 21 years. Original iPod (2001) Introduced in October...
iOS 16 mock for article

Gurman: iOS 16 to Include New Ways of System Interaction and 'Fresh Apple Apps'

Sunday May 15, 2022 6:14 am PDT by
iOS 16 will include new ways of interacting with the system and some "fresh Apple apps," Bloomberg's Mark Gurman has said, offering some more detail on what Apple has in store for the upcoming release of iOS and iPadOS set to be announced in a few weeks at WWDC. In the latest edition of his Power On newsletter, Gurman wrote that while iOS 16 is not likely to introduce a major face-lift to...
maxresdefault

Unbox Therapy Shares Hands-On Look at iPhone 14 Pro Max Replica

Monday May 16, 2022 4:40 am PDT by
YouTuber Unbox Therapy has shared a hands-on look at the iPhone 14 Pro Max using what he claims is a one-to-one replica created by third-party case makers with access to detailed schematics and dimensions for Apple's new upcoming flagship smartphone. As with the iPhone 13 Pro lineup, in 2022, we are expecting a 6.1-inch iPhone 14 Pro and a 6.7-inch iPhone 14 Pro Max, but this time the Pro...
iOS 15

Apple Releases iOS 15.5 and iPadOS 15.5 With Wallet and Podcast Updates

Monday May 16, 2022 10:00 am PDT by
Apple today released iOS 15.5 and iPadOS 15.5, the fifth major updates to the iOS and iPadOS 15 operating systems that were initially released in September 2021. iOS and iPadOS 15.5 come a little over two months after the launch of iOS 15.4 and iPadOS 15.4. The iOS 15.5 and iPadOS 15.5 updates can be downloaded for free and the software is available on all eligible devices over-the-air in...
14 16 inch 2021 mbps back to back feature orange

Five Things You Still Can't Do With a MacBook Pro

Wednesday May 11, 2022 11:16 am PDT by
It's been over 200 days since Apple debuted its redesigned MacBook Pro lineup. Offered in 14-inch and 16-inch display sizes, the new-look MacBooks wowed Apple fans and creative pros alike with their powerful custom Apple silicon, mini-LED screen, and multiple connectivity options. But there are still some things you can't do with a MacBook Pro. Here are five features some Mac users are still...
apple mac ipad watch trade in

Apple Launches Limited-Time Bonus Trade-In Credit for iPhone, iPad, Mac, and Apple Watch in Many Countries

Wednesday May 11, 2022 5:14 am PDT by
Apple has launched a special limited-time offer for iPhone, Apple Watch, Mac, and iPad trade-in that offers customers additional credit when trading in their only device for a new one. The offer is being run in several countries including the US, UK, Germany, Spain, Italy, South Korea, Japan, Taiwan, China, India, and France. In the UK, Apple is offering up to £50 of extra trade-in credit...
airpodsprodesign

Kuo: AirPods, MagSafe Battery Pack, and Other Apple Accessories Also to Switch to USB-C in Future

Sunday May 15, 2022 5:59 am PDT by
Earlier this week, well-known Apple analyst Ming-Chi Kuo claimed that Apple plans to release at least one iPhone 15 model with a USB-C port in 2023. Now, in a follow-up tweet, he has claimed that accessories like AirPods, the MagSafe Battery Pack, and the Magic Keyboard/Mouse/Trackpad trio would also switch to USB-C in the "foreseeable future." Both the iPhone and all of the aforementioned...