eBay’s HeadGaze brings hands-free input to the iPhone X using ARKit

Today, eBay revealed a nifty new piece of augmented reality software that makes use of both the front-facing iPhone X camera and Apple’s ARKit framework. By tracking head motions using the 3D sensors in the TrueDepth camera, which is the same hardware that enables Apple’s Face ID unlock feature, eBay developed a way to transmit input commands by just moving your head.

The system, which eBay is calling HeadGaze, seems to work similarly to eye tracking, although it would understandably be less precise than a fully featured eye-tracking kit. Given its primary business of helping people sell products online, eBay’s demo video illustrates how HeadGaze can be used to do hands-free online shopping, and just glancing at certain parts of the iPhone screen can progress through the checkout flow.

The technology was created by a team of eBay interns led by Muratcan Cicek, a software engineer and PhD student who uses assistive technology as an aid for his motor impairment disability. In that sense, HeadGaze — and the shopping app Cicek created, called HeadSwipe — could be a very useful piece of technology for people with disabilities who are looking for a more accessible way to use mobile devices.

“In addition to this head gazing experience, we’re exploring an experience that tracks eye movements,” Cicek writes in a blog post about HeadGaze. “The fusion of these gazing experiences open up a broader possibility on defining various hands-free gestures, enabling much more interesting applications.” Given its utility to those with disabilities, eBay has decided to release both HeadGaze and HeadSwipe as an open-source project on GitHub so that other app makers can implement Cicek’s technology into their own products.