Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

Multiple reports suggest that Apple is developing an augmented reality headset as well as smart glasses, and a newly granted Apple patent may provide a clue about one of the potential features that could be included in the latter device.

patent
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent describing "systems and methods for gaze-driven recording of video."

Apple's patent notes that head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users, and envisions video from such a device being recorded for later playback or analysis. Specifically, it envisions such a system in smart glasses, where the use of built-in gaze-tracking sensors could provide an indication of where a person is currently looking, which could direct a built-in camera to record the scene where the user's eyes are trained, instead of simply recording what's in front of the user.

The patent envisions a system with one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to access the gaze data captured. This apparatus would apply a temporal filter to the gaze data to obtain a "smoothed gaze estimate," which would allow it to determine a region of interest based on the sensors. The system would then apply signal processing to the video based on the region of interest to obtain an enhanced recording that could mitigate the impact of rapid-movement eye saccades.

According to the patent, the system would consist of a pair of smart glasses and a separate connected device for power and storage, which could be interpreted to mean an iPhone, iPad or Mac.

record gaze tracking patent 2
The invention harks back to another patent filed last month in which an eye tracking system is used for detecting the position and movements of a user's eyes in smart glasses and other head-mounted displays.

In the latter patent, Apple envisages the eye tracking system being used to analyze the position and movements of the user's eyes, or to detect other information such as pupil dilation. In one example, identifying the point of the user's gaze allows for interaction with content shown on the near-eye display of the head-mounted display, while another application includes the creation of eye image animations that could be used for digital avatars visible to other users in a mixed-reality communal environment.

Of course, Apple files numerous patent applications every week, and many of these inventions do not see the light of day. But they do provide a glimpse into the kinds of technologies Apple is exploring for potential use in its upcoming AR/VR products, and based on the rumored launch roadmap for its mixed-reality headset and smart glasses, the company still has plenty of time to implement them.

The Information and Bloomberg have both said that Apple is working on smart glasses AND an AR/VR headset, with the headset to come out in first followed by the glasses. Apple analyst Ming-Chi Kuo believes the "mixed reality" headset will come out in 2022, with the Apple Glasses to follow in 2025. The headset is AR/VR, while Apple's glasses are augmented reality.

The headset is rumored to be similar to Facebook's Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable. JP Morgan believes the headset will look similar to other brands of VR headsets, featuring six lenses and an optical LiDAR scanner to map the wearer's environment. The headset will be targeted at the top end of the consumer market and will be more expensive than others on the market.

The glasses, meanwhile, which are reportedly in an early stage of development, are said to resemble high-end sunglasses with thick frames that house the battery and chips. Kuo expects the AR glasses to be marketed as an ‌iPhone‌ accessory and will primarily take a display role offloading computing, networking, and positioning to the ‌iPhone‌, with the glasses providing a mobile-first "optical see-through AR experience."

Patently Apple was first to report on the patent being granted today. Apple originally applied for the patent in December 2019.

Related Roundup: Apple Glasses
Tag: patent

Top Rated Comments

Salty Pirate Avatar
11 months ago
such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Score: 4 Votes (Like | Disagree)
Corsig Avatar
11 months ago

such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Is there another reason to use them?
Score: 4 Votes (Like | Disagree)
CarpalMac Avatar
11 months ago
Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.
Score: 2 Votes (Like | Disagree)
jjudson Avatar
11 months ago
A whole new bunch of glassholes coming soon...
Score: 2 Votes (Like | Disagree)
CJ Dorschel Avatar
11 months ago
Apple: ”Let’s discontinue the HomePod instead of updating it cause we’re losing money”

also Apple: “Pour more money into the Apple Car and Glasses!”

Over 1000 posts on the HomePod in a day or two and barely anything on the Apple Car and Glasses. Just sayin??‍♂️
Score: 1 Votes (Like | Disagree)
Corsig Avatar
11 months ago

Dem tatas?
I stand corrected
Score: 1 Votes (Like | Disagree)

Related Stories

arglassesyay

Apple Glasses Prototype Reportedly Falls Behind 2021 Testing Schedule

Monday April 26, 2021 2:54 am PDT by
Apple has yet to start testing the second prototype of its rumored AR glasses, bucking the expectations of its supply partners who expected testing to commence in the first quarter of this year, according to a new report. Apple is known to be working on a set of augmented reality glasses that look like regular glasses, but which feature lenses with built-in displays that can be interacted...
apple view concept right corner

Apple Mixed Reality Headset Renders Depict Rumored Design

Wednesday February 10, 2021 9:11 am PST by
Designer Antonio De Rosa has created 3D renders of Apple's long-rumored mixed-reality headset based on recent reports about the device, giving the first photorealistic look at the mysterious Apple device. De Rosa tentatively titles Apple's mixed-reality headset device the "Apple View," but there has been no evidence that this is the company's chosen moniker. What Apple may call the device...
apple mixed reality headset mockup feature orange

Kuo: Apple Still on Track to Release Mixed Reality Headset Next Year

Friday April 23, 2021 5:53 am PDT by
Apple analyst Ming-Chi Kuo had previously stated that Apple's mixed reality headset would debut in the middle of next year, and in a new note today obtained by MacRumors, the highly-respected analyst is once again doubling down that Apple plans to showcase its mixed reality headset in 2022. The information we know on Apple's headset is somewhat scarce, but according to one report, the...
samsung ar glasses

Leaked Videos Reveal Samsung's Plans for AR Glasses to Compete With Apple

Monday February 22, 2021 5:43 am PST by
Amid a spate of rumors in recent months about Apple's plans for a mixed reality headset and AR glasses, videos of Samsung's plans for its own AR glasses have purportedly leaked online (via The Verge). A leaker known as "WalkingCat" has shared seemingly official Samsung concept videos that depict "Samsung Glasses Lite" and "Samsung AR Glasses." Samsung Glasses Lite give the illusion of a 2D ...
apple mixed reality headset mockup feature

Kuo: Apple's Upcoming Mixed Reality Headset to Feature 15 Camera Modules

Monday March 8, 2021 9:25 pm PST by
Apple's upcoming mixed reality headset will feature a total of 15 optical camera modules, Apple analyst Ming-Chi Kuo said this evening in a research note obtained by MacRumors. Eight of the 15 camera modules will be used for see-through augmented reality experiences, six modules will be used for "innovative biometrics," and one camera module will be used for environmental detection. Largan...
snap spectacles

Snap Working on Advanced Smart Glasses With Augmented Reality Capabilities

Tuesday March 30, 2021 2:33 pm PDT by
Snap is designing a new set of "Spectacles" smart glasses that have augmented reality capabilities, reports The Information. Snap has sold several versions of its Spectacles, but current options do not have AR features and are instead equipped with a camera for uploading content to social network Snapchat. The augmented reality Spectacles will be aimed at developers and creators rather than...
Apple VR Feature

Apple VR Headset May Launch in Q1 2022 With LiDAR and $500+ Price Tag [Updated]

Wednesday February 3, 2021 6:09 am PST by
Apple's rumored virtual reality (VR) headset may be released as soon as the first quarter of 2022, according to JP Morgan analysis seen by China Times. The report, spotted by iMore, explains that the "industrial design of Apple's headset will be close to that of other brands of VR headsets." The headset is said to feature six lenses and an optical LiDAR scanner to map the wearer's...
smart contact lens

Augmented Reality Apple Contact Lenses Are Coming

Friday March 12, 2021 2:13 am PST by
Reliable Apple analyst Ming-Chi Kuo has set out an unusually ambitious prediction for an Apple product set to be released in the 2030s: augmented reality contact lenses. Mojo Vision smart contact lens In a recent research note seen by MacRumors, Kuo explained that Apple will release augmented reality contact lenses at some point in the 2030s. The lenses are said to bring electronics from...

Popular Stories

airpodsinear 1

AirPods Save Woman's Life With Feature Everyone Should Know

Friday January 21, 2022 2:13 am PST by
Apple's AirPods have been credited with saving a woman's life after a potentially fatal fall, People reports. When a 60-year-old florist in New Jersey tripped and hit her head in her studio, she lost consciousness and awoke heavily bleeding. With nobody around to call for help, she realized she had her AirPods in, and used a "Hey Siri" command to call 911. An operator was able to stay on the ...
iphone 13 earpods

Apple to Stop Including EarPods With Every iPhone Sold in France From Next Week

Friday January 21, 2022 3:21 am PST by
Apple will no longer include EarPods with every iPhone sold in France, starting on January 24, according to a notice posted by a French carrier (via iGeneration). Apple was previously required to include EarPods in the box with the iPhone due to a French law that required every smartphone sold in the country to come with a "handsfree kit," but the law has now been changed in favor of reducing the ...
peloton tv workout cardio

Apple Floated as Potential Buyer of Peloton

Friday January 21, 2022 6:11 am PST by
Following months of bleak news about Peloton's "precarious state," including the revelation that it has halted production of its bikes and treadmills, Apple is being floated as a potential buyer of Peloton's troubled fitness business. Yesterday, CNBC reported that Peloton will temporarily stop production of its connected fitness products due to a "significant reduction" in consumer demand, a ...
Questionable Design Decisions

Apple's Most Questionable Design Decisions in Recent Memory

Sunday January 23, 2022 2:59 am PST by
Apple has always emphasized the depth of thought that goes into the design of its products. In the foreword to Designed by Apple in California, a photo book released by the company in 2016, Jony Ive explains how Apple strives "to define objects that appear effortless" and "so simple, coherent and inevitable that there could be no rational alternative." But every once in a while even Apple...
Spring 2022 Apple Products Feature

New iPad Air, Macs, and iPhone SE With 5G Likely to Be Announced at Apple Event This Spring

Thursday January 20, 2022 8:32 am PST by
Earlier this week, Bloomberg's Mark Gurman tweeted that Apple "will be holding a spring event" to announce a new iPhone SE and other hardware. In a recent edition of his newsletter, Gurman said the event is likely to occur in March or April. Gurman did not elaborate on what "other hardware" will be announced at Apple's purported spring event, but rumors suggest at least four products are...
Upcoming Products 2022 Feature

Gurman: Apple Preparing 'Widest Array of New Hardware Products in Its History' for Fall

Sunday January 23, 2022 10:32 am PST by
Apple is working on a number of new products that are set to launch this fall, and Bloomberg's Mark Gurman says that it will be "the widest array" of new devices that Apple has introduced in its history. In his latest "Power On" newsletter, Gurman explains that Apple is working on four new iPhones (5G iPhone SE, iPhone 14, iPhone 14 Pro, and iPhone 14 Pro Max), an updated low-end MacBook Pro,...
apple watch series 7 aluminum colors yellowbg

Apple Watch Charging Bug Fixed in watchOS 8.4 Release Candidate

Thursday January 20, 2022 4:01 pm PST by
The watchOS 8.4 release candidate that was seeded to developers and beta testers this morning addresses an ongoing bug that could cause some Apple Watch chargers not to work properly with the Apple Watch. Back in December, we reported on a growing number of charging issues that Apple Watch Series 7 owners were facing. Since watchOS 8.3, there have been a number of complaints about...
safari icon blue banner

macOS Monterey 12.2 and iOS 15.3 Release Candidates Fix Safari Bug That Leaks Browsing Activity

Thursday January 20, 2022 1:30 pm PST by
The macOS Monterey 12.2 and iOS 15.3 release candidates that came out today appear to address a Safari bug that could cause your recent browsing history and details about your identity to be leaked to malicious entities. As shared last week by browser fingerprinting service FingerprintJS, there is an issue with the WebKit implementation of the IndexedDB JavaScript API. Any website that uses...
apple college discounts

Apple Walks Back UNiDAYS Verification Requirement for U.S. Education Store

Friday January 21, 2022 12:43 pm PST by
Earlier this week, Apple began requiring that customers taking advantage of educational discounts in the United States verify their status as a teacher, student, or school staff member through UNiDAYS. The requirement was a major change as Apple had never asked customers to go through a verification process in the United States before, and now, just three days after verification was added,...