Beyond Television: How Apple Could Incorporate PrimeSense Technology Into Its Products

While PrimeSense, Apple's latest acquisition, is best known for developing the chip powering Microsoft's original Kinect motion-sensing gaming system, its 3D sensors have also been used in several industries beyond television and gaming.

It has been widely rumored that Apple will incorporate motion sensing technology into a potential upgraded set top box or television set, a logical potential use for PrimeSense's assets, but other uses of PrimeSense's sensors indicate Apple could also incorporate the technology into Macs and mobile devices, 3D scanners, or interactive retail displays.

Multiple companies have already taken advantage of PrimeSense technology via its existing Carmine sensors and NiTE natural interaction middleware.

The Technology

PrimeSense uses "Light Coding" for its 3D depth sensing, which involves a near-IR light source that projects an invisible light into a room or a scene. A separate CMOS image sensor then reads the IR-light and captures it along with a series of synchronized images.

The infrared patterns cast by the IR-light, which enable depth acquisition, are then deciphered by the company's chips to create a virtual image of a scene or object.

Mobile Devices, Gaming, and Mapping

PrimeSense's newest 3D sensor and most recent technology, the Capri, is a system-on-a-chip small enough to be incorporated into mobile devices and laptop computers. According to PrimeSense, the Capri enables a mobile device to "see the world in 3D," letting users interact with their surroundings, "blurring between the real and virtual worlds."

A demo of the Capri at Google I/O earlier this year focused heavily on the augmented reality applications of the technology rather than gesture-based uses. The Capri is able to scan a full environment in 3D, including furniture and people, providing an on-screen representation. It is also able to measure virtual objects and provide relative distances and sizes, which could have a number of uses in interactive gaming, indoor mapping, and other applications.

Apple has filed several different patents that pertain to similar technology, including a 2009 patent that referenced "hyper-reality" displays to allow users to view 3D objects on electronic devices and a 2012 patent describing the use of motion sensors to create a virtual three-dimensional interface for iOS devices.

At Google I/O, PrimeSense suggested that its Capri chip was ready for market and could be embedded into various mobile devices by manufacturers. An embedded Capri chip could then be used in "hundreds of thousands" of applications created by app developers using a companion SDK. Apple could potentially include the Capri or a similar chip in its future iOS or Mac devices as well as a potential set top box or television, opening up a new avenue of app development.

3D Scanning and Printing

PrimeSense's chips can be used to create highly accurate 360 degree scans of real world objects, which can then be sent to 3D printers to create an exact replica of an object. The technology can also be used to scan real world objects and incorporate them into applications, which PrimeSense has demoed as useful in home decorating and catalog-style apps.

PrimeSense's 3D scanning technology has already been used by 3D Systems to create a 3D peripheral scanner called "Sense," which is aimed at consumers. Apple has expressed some interest in 3D printing to create prototype products and PrimeSense’s sub millimeter accuracy 3D scans could possibly be useful in the creation process.

Watch on YouTube

On its website, PrimeSense describes interactive digital displays that incorporate touch, almost touch, object pointing, and gesture interfaces that can be used to attract and engage users as well as incorporating analytics information such as people counting, tracking, and demographics classification.

With PrimeSense sensors, digital signs and displays can respond to people's presence and movement via touch or "almost touch," which allows 3D sensors to "see the space between the user and the display."

Apple is reportedly working on increasing customer engagement at its Apple Stores, through the implementation of Bluetooth LE iBeacons that would give customers additional information on products via their own mobile devices. PrimeSense’s technology could also be used to improve customer engagement, through the use of immersive, interactive displays and 3D technology incorporated into its Apple Store apps.

Currently, companies like Bodymetrics use the technology to allow customers to get their bodies scanned in-store and at home, which then facilitates virtual try-ons.

Competition

Though Microsoft originally relied on PrimeSense technology for the Kinect, it purchased PrimeSense rival Canesta in 2010 and has since developed its own in-house solutions for the Kinect motion sensing system that shipped with the recently released Xbox One.

At the time of Microsoft's purchase, PrimeSense rival Canesta was believed to have more potential as it utilized a specialized chip geared specifically towards 3D recognition, while PrimeSense was more affordable. It is unclear how the two technologies compare at the current point in time, but PrimeSense will undoubtedly flourish under Apple’s wing.

Future Use

While news of Apple’s PrimeSense acquisition first surfaced in July, confirmation of the purchase didn't come until November, suggesting a long period of negotiations. Apple is reported to have paid $345 million for PrimeSense, which employs approximately 150 employees in Tel Aviv and several other locations around the world.

It is not known how Apple plans to incorporate PrimeSense's sensors and software into its products, but there are a wide range of possibilities for the technology. According to PrimeSense, its Capri chip is immediately ready for deployment, but as the technology is already several months old and rather large, it is likely that Apple will work with PrimeSense on developing new technology custom built for Apple products.