eBay Builds Tech That Uses iPhone X TrueDepth Camera and ARKit to Navigate Apps With Head Motion

by

eBay has created a new technology it's calling "HeadGaze," which tracks the user's head movement through the iPhone X's TrueDepth camera and ARKit so that they can navigate around eBay without touching the screen. The technology was created by Muratcan Cicek, an eBay intern with motor impairments who was looking for a way to shop online independently.


The eBay team built a model that tracks the user's head using 3D information from ARKit, creating a virtual cursor that follows the head's motion in every direction. Then, designated buttons on the screen can be activated depending on how long the cursor has been in one spot. These buttons can perform actions like scrolling down, moving to another page, selecting a product to purchase, and more, all without touching the screen.

Cicek says that the technology's modular code design will let developers "easily integrate" HeadGaze's features into existing or future apps:

This year as part of my internship project at eBay, my team and I developed HeadGaze, a reusable technology library that tracks head movement on your iPhone X and starting today, the technology is available via open source on GitHub.com. The first of its kind, this technology uses Apple ARKit and the iPhone X camera to track your head motion so you can navigate your phone easily without using your hands.

HeadGaze enables you to scroll and interact on your phone with only subtle head movements. Think of all the ways that this could be brought to life. Tired of trying to scroll through a recipe on your phone screen with greasy fingers while cooking? Too messy to follow the how-to manual on your cell phone while you’re tinkering with the car engine under the hood? Too cold to remove your gloves to use your phone?

eBay has developed an app called "HeadSwipe" as a way to test the HeadGaze technology. HeadSwipe is focused on browsing and buying items in eBay's deals section, and can be navigated entirely through head motions on iPhone X. Both HeadGaze and HeadSwipe are available open source on GitHub.


Next, the team is also looking at technology that tracks eye movements, and potentially fusing the two experiences together in future apps.

Tag: eBay

Top Rated Comments

(View all)
Avatar
28 months ago
For anyone with mobility problems, this is a really useful tool. I would have expected it to come from Leap or Apple itself, not eBay. But I guess that shows how Apple's "Kit" frameworks are enabling some interesting innovations.
Score: 13 Votes (Like | Disagree)
Avatar
28 months ago
Imagine if they did this to Tinder? The neck ache some people would get
Score: 6 Votes (Like | Disagree)
Avatar
28 months ago
Now that's using your head!
Score: 3 Votes (Like | Disagree)
Avatar
28 months ago

You do realize the iPhone SE & 5S work perfectly , right?

I would love to hear your definition of "work perfectly"...
Score: 3 Votes (Like | Disagree)
Avatar
28 months ago
New ads: "HEY YOU! LOOK HERE! NOW LOOK HERE! Thank you for your purchase"
Score: 3 Votes (Like | Disagree)
Avatar
28 months ago
Although I can’t see the average user utilizing this, it will be great for accessibility uses and those who really need it!
Score: 2 Votes (Like | Disagree)

Top Stories

Early iPhone 12 Tests Show Ceramic Shield is Stronger and More Scratch Resistant Than iPhone 11 Glass

Friday October 23, 2020 1:21 pm PDT by
Apple's new iPhone 12 models are protected by a Ceramic Shield cover glass that has nano-ceramic crystals infused right into the glass to improve durability. According to Apple, Ceramic Shield offers four times better drop protection than the glass used for the iPhone 11 models. YouTube channel MobileReviewsEh conducted some tests on the iPhone 12 using a force meter to compare its performance ...

First Impressions From New iPhone 12 and 12 Pro Owners

Thursday October 22, 2020 4:20 pm PDT by
It's already Friday, October 23, in Australia and New Zealand, which means some customers who purchased an iPhone 12 or 12 Pro already have their new devices in hand. We've seen dozens of reviews of the iPhone 12 and iPhone 12 Pro from media sites, but now first impressions from regular Apple customers are available. Image via MacRumors reader Boardiesboi New iPhone 12 and 12 Pro owners are...

iPhone 12 Pro Allows You to Measure Someone's Height Instantly Using LiDAR Scanner

Saturday October 24, 2020 11:12 am PDT by
iPhone 12 Pro models feature a new LiDAR Scanner for enhanced augmented reality experiences, but the sensor also enables another unique feature: the ability to measure a person's height instantly using the Measure app. You can even measure the seated height of a person in a chair, according to Apple. When the Measure app detects a person in the viewfinder, it automatically measures their...

Apple VP Kaiann Drance Interview Addresses Battery Life, MagSafe, and Power Adapter Concerns

Friday October 23, 2020 3:37 am PDT by
Apple's Vice President of iPhone Marketing, Kaiann Drance, has provided a new interview to Rich DeMuro on the Rich on Tech Podcast, to discuss the iPhone 12 and iPhone 12 Pro. Although much of the interview repeated points from Apple's "Hi, Speed" event, there were a number of interesting tidbits regarding the affect of 5G on battery life, MagSafe concerns, and the lack of a power adapter in...

iPhone 11 Pro Outlasts iPhone 12 and 12 Pro in Extensive Battery Life Test

Friday October 23, 2020 8:36 am PDT by
Arun Maini today shared a new side-by-side iPhone battery life video test on his YouTube channel Mrwhosetheboss, timing how long the new iPhone 12 and iPhone 12 Pro models last on a single charge compared to older models, with equal brightness, settings, battery health, and usage. All of the devices are running iOS 14 without a SIM card inserted. In the test, the iPhone 11 Pro outlasted both ...

Apple Distributing New Heated Display Removal Machine for iPhone 12 Repairs

Thursday October 22, 2020 6:20 pm PDT by
Apple is providing Genius Bars and Apple Authorized Service Providers with a new heated display removal fixture for iPhone 12 and iPhone 12 Pro repairs, according to information obtained by MacRumors from a reliable source. To open iPhone 12 models, technicians will be required to slide the device into a specialized tray, and then place the tray into the high-temperature fixture for two...

Apple Warns MagSafe Charger Can Leave Circular Imprints on Leather Cases

Friday October 23, 2020 3:23 pm PDT by
If you keep your iPhone in a leather case while charging with Apple's new MagSafe Charger, the case might show circular imprints from contact with the accessory, according to a new Apple support document published today. Apple's leather cases for the iPhone 12 and iPhone 12 Pro are not available until November 6, but a MacRumors reader has already shared a photo of a circular imprint on...

MagSafe Charger Teardown Reveals Simple Design With Magnets and Charging Coil Encircling a Small Circuit Board

Friday October 23, 2020 7:50 am PDT by
iFixit has today shared a teardown of Apple's new MagSafe charger for the iPhone 12 and iPhone 12 Pro. An X-ray of the MagSafe charger courtesy of Creative Electron reveals the internal charging coil surrounded by a circular arrangement of magnets within the puck. The only seam that iFixit was able to leverage to open the device was where the white rubber circle meets the metal rim,...

PSA: Non-iPhone 12 Models Charge Super Slowly With MagSafe Charger

Friday October 23, 2020 4:11 pm PDT by
Alongside the iPhone 12 models, Apple introduced a new $39 MagSafe Charger that's meant to work with the magnets in the iPhone 12 Pro models to charge them up at a maximum of 15W. The MagSafe Charger is technically able to be used with older iPhones, but it's not a good idea because the charging with non-iPhone 12 devices is so slow. We did two tests with the iPhone XS Max, draining the...

New Photos Offer Better Look at iPhone 12 Color Options

Tuesday October 20, 2020 2:34 am PDT by
As we wait for the iPhone 12 review embargo to lift later today, more pictures are circulating of the devices in real-world lighting conditions, providing a better look at the different colors available. Leaker DuanRui has shared images on Twitter of the iPhone 12 in white, black, blue, green, and (PRODUCT)RED. The black and white colors are similar to the iPhone 11 colors, but the other...