eBay Builds Tech That Uses iPhone X TrueDepth Camera and ARKit to Navigate Apps With Head Motion

by

eBay has created a new technology it's calling "HeadGaze," which tracks the user's head movement through the iPhone X's TrueDepth camera and ARKit so that they can navigate around eBay without touching the screen. The technology was created by Muratcan Cicek, an eBay intern with motor impairments who was looking for a way to shop online independently.


The eBay team built a model that tracks the user's head using 3D information from ARKit, creating a virtual cursor that follows the head's motion in every direction. Then, designated buttons on the screen can be activated depending on how long the cursor has been in one spot. These buttons can perform actions like scrolling down, moving to another page, selecting a product to purchase, and more, all without touching the screen.

Cicek says that the technology's modular code design will let developers "easily integrate" HeadGaze's features into existing or future apps:

This year as part of my internship project at eBay, my team and I developed HeadGaze, a reusable technology library that tracks head movement on your iPhone X and starting today, the technology is available via open source on GitHub.com. The first of its kind, this technology uses Apple ARKit and the iPhone X camera to track your head motion so you can navigate your phone easily without using your hands.

HeadGaze enables you to scroll and interact on your phone with only subtle head movements. Think of all the ways that this could be brought to life. Tired of trying to scroll through a recipe on your phone screen with greasy fingers while cooking? Too messy to follow the how-to manual on your cell phone while you’re tinkering with the car engine under the hood? Too cold to remove your gloves to use your phone?

eBay has developed an app called "HeadSwipe" as a way to test the HeadGaze technology. HeadSwipe is focused on browsing and buying items in eBay's deals section, and can be navigated entirely through head motions on iPhone X. Both HeadGaze and HeadSwipe are available open source on GitHub.


Next, the team is also looking at technology that tracks eye movements, and potentially fusing the two experiences together in future apps.

Tag: eBay

Top Rated Comments

(View all)
Avatar
22 months ago
For anyone with mobility problems, this is a really useful tool. I would have expected it to come from Leap or Apple itself, not eBay. But I guess that shows how Apple's "Kit" frameworks are enabling some interesting innovations.
Score: 13 Votes (Like | Disagree)
Avatar
22 months ago
Imagine if they did this to Tinder? The neck ache some people would get
Score: 6 Votes (Like | Disagree)
Avatar
22 months ago
Now that's using your head!
Score: 3 Votes (Like | Disagree)
Avatar
22 months ago

You do realize the iPhone SE & 5S work perfectly , right?

I would love to hear your definition of "work perfectly"...
Score: 3 Votes (Like | Disagree)
Avatar
22 months ago
New ads: "HEY YOU! LOOK HERE! NOW LOOK HERE! Thank you for your purchase"
Score: 3 Votes (Like | Disagree)
Avatar
22 months ago
Although I can’t see the average user utilizing this, it will be great for accessibility uses and those who really need it!
Score: 2 Votes (Like | Disagree)

Top Stories

Leaker: Apple to Stick With Lightning Over USB-C for 'iPhone 12' Before Going Port-Less Next Year

Tuesday May 26, 2020 2:31 am PDT by
Apple will use a Lightning port instead of USB-C in the upcoming "iPhone 12," but it will be the last major series of Apple's flagship phones to do so, with models set to combine wireless charging and a port-less Smart Connector system for data transfer and syncing in the iPhone "13 series" next year. The above claim comes from occasional Apple leaker and Twitter user "Fudge" (@choco_bit),...

Apple Releases macOS Catalina 10.15.5 With Battery Health Management Features, Fix for Finder Freezing

Tuesday May 26, 2020 1:59 pm PDT by
Apple today released macOS Catalina 10.15.5, the fifth update to the macOS Catalina operating system that was released in October 2019. macOS Catalina 10.15.5 comes two months after the launch of macOS Catalina 10.15.4, which introduced Screen Time Communication Limits. macOS Catalina 10.15.5 is a free update that can be downloaded from the Mac App Store using the Update feature in the...

Apple Reissuing Numerous iOS App Updates, Potentially Related to Recent 'This App is No Longer Shared' Bug

Sunday May 24, 2020 9:13 pm PDT by
Over the past few hours, a number of MacRumors readers have reported seeing dozens or even hundreds of pending app updates showing in the App Store on their iOS devices, including for many apps that were already recently updated by the users. In many cases, the dates listed on these new app updates extend back as far as ten days. Apple has not shared any information as to why updates for...

16-Inch MacBook Pro, iPad Pro, and iMac Pro With Mini-LED Displays Again Rumored to Launch in 2021

Tuesday May 26, 2020 5:30 am PDT by
Apple plans to release several higher-end devices with Mini-LED displays in 2021, including a new 12.9-inch iPad Pro in the first quarter, a new 16-inch MacBook Pro in the second quarter, and a new 27-inch iMac in the second half of the year, according to Jeff Pu, an analyst at Chinese research firm GF Securities. This timeframe lines up with one shared by analyst Ming-Chi Kuo, who recently...

Jailbreak Tool 'unc0ver' 5.0 Released With iOS 13.5 Compatibility

Sunday May 24, 2020 3:06 pm PDT by
The team behind the "unc0ver" jailbreaking tool for iOS has released version 5.0.0 of its software that claims to have the ability to jailbreak "every signed iOS version on every device" using a zero-day kernel vulnerability by Pwn20wnd, a renowned iOS hacker. The announcement comes just days after it was announced that the tool would soon launch. The unc0ver website highlights how the tool...

Future AirPods to Include 'Ambient Light Sensors' Possibly Related to Rumored Health Features

Monday May 25, 2020 2:53 am PDT by
Apple is reportedly looking to integrate light sensors in a new model of AirPods in the next couple of years, according to a new report today, suggesting their use could be part of rumored upcoming health monitoring features in the true wireless earbuds. In a paywalled article, DigiTimes reports that ASE Technology could be involved in manufacturing the sensors: Apple is expected to...

First App Using Apple and Google's Exposure Notification API Launches in Switzerland

Tuesday May 26, 2020 3:02 pm PDT by
The first app that takes advantage of the Exposure Notification API developed by Apple and Google has launched in Switzerland, according to a report from the BBC. A team of app developers working on contact tracing app called SwissCovid have rolled out the app in a beta capacity for members of the Swiss army, hospital workers, and civil servants. After the app is tested and approved by MPs,...

'This App is No Longer Shared' iOS Bug Preventing Some Apps From Opening

Friday May 22, 2020 3:58 pm PDT by
An app bug is causing some iOS users to be unable to open their apps, with affected iPhone and iPad users seeing the message "This app is no longer shared with you" when attempting to access an app. There are multiple complaints about the issue on the MacRumors forums and on Twitter from users who are running into problems. A MacRumors reader describes the issue:Is anyone else experiencing...

HBO Max Now Available on Apple TV and iOS Devices

Wednesday May 27, 2020 2:42 am PDT by
HBO Max launched today, and is now available on Apple TV, iPhone, and iPad. WarnerMedia's new streaming service, which replaces HBO Now, combines HBO content with shows and films from Warner Bros and Turner TV. The service is available as a native app on the ‌Apple TV‌ HD and ‌Apple TV‌ 4K, but second and third-generation ‌Apple TV‌ owners will need to AirPlay HBO Max content...

Apple's Pro Display XDR Wins 'Displays of the Year' Award

Tuesday May 26, 2020 7:45 am PDT by
The Society for Information Display today announced its selections for the 26th Annual Display Industry Awards, and Apple has once again taken one of the top spots, this time with its Pro Display XDR that debuted last year alongside the revamped Mac Pro. The Pro Display XDR was one of three display products named "Displays of the Year," alongside Samsung's foldable display and BOE's...