Beyond Television: How Apple Could Incorporate PrimeSense Technology Into Its Products

While PrimeSense, Apple's latest acquisition, is best known for developing the chip powering Microsoft's original Kinect motion-sensing gaming system, its 3D sensors have also been used in several industries beyond television and gaming.

It has been widely rumored that Apple will incorporate motion sensing technology into a potential upgraded set top box or television set, a logical potential use for PrimeSense's assets, but other uses of PrimeSense's sensors indicate Apple could also incorporate the technology into Macs and mobile devices, 3D scanners, or interactive retail displays.

Multiple companies have already taken advantage of PrimeSense technology via its existing Carmine sensors and NiTE natural interaction middleware.

The Technology

PrimeSense uses "Light Coding" for its 3D depth sensing, which involves a near-IR light source that projects an invisible light into a room or a scene. A separate CMOS image sensor then reads the IR-light and captures it along with a series of synchronized images.

The infrared patterns cast by the IR-light, which enable depth acquisition, are then deciphered by the company's chips to create a virtual image of a scene or object.

primesense
Mobile Devices, Gaming, and Mapping

PrimeSense's newest 3D sensor and most recent technology, the Capri, is a system-on-a-chip small enough to be incorporated into mobile devices and laptop computers. According to PrimeSense, the Capri enables a mobile device to "see the world in 3D," letting users interact with their surroundings, "blurring between the real and virtual worlds."

A demo of the Capri at Google I/O earlier this year focused heavily on the augmented reality applications of the technology rather than gesture-based uses. The Capri is able to scan a full environment in 3D, including furniture and people, providing an on-screen representation. It is also able to measure virtual objects and provide relative distances and sizes, which could have a number of uses in interactive gaming, indoor mapping, and other applications.

Apple has filed several different patents that pertain to similar technology, including a 2009 patent that referenced "hyper-reality" displays to allow users to view 3D objects on electronic devices and a 2012 patent describing the use of motion sensors to create a virtual three-dimensional interface for iOS devices.

At Google I/O, PrimeSense suggested that its Capri chip was ready for market and could be embedded into various mobile devices by manufacturers. An embedded Capri chip could then be used in "hundreds of thousands" of applications created by app developers using a companion SDK. Apple could potentially include the Capri or a similar chip in its future iOS or Mac devices as well as a potential set top box or television, opening up a new avenue of app development.

3D Scanning and Printing

PrimeSense's chips can be used to create highly accurate 360 degree scans of real world objects, which can then be sent to 3D printers to create an exact replica of an object. The technology can also be used to scan real world objects and incorporate them into applications, which PrimeSense has demoed as useful in home decorating and catalog-style apps.

PrimeSense's 3D scanning technology has already been used by 3D Systems to create a 3D peripheral scanner called "Sense," which is aimed at consumers. Apple has expressed some interest in 3D printing to create prototype products and PrimeSense’s sub millimeter accuracy 3D scans could possibly be useful in the creation process.


Interactive Displays and Retail

On its website, PrimeSense describes interactive digital displays that incorporate touch, almost touch, object pointing, and gesture interfaces that can be used to attract and engage users as well as incorporating analytics information such as people counting, tracking, and demographics classification.

With PrimeSense sensors, digital signs and displays can respond to people's presence and movement via touch or "almost touch," which allows 3D sensors to "see the space between the user and the display."

Apple is reportedly working on increasing customer engagement at its Apple Stores, through the implementation of Bluetooth LE iBeacons that would give customers additional information on products via their own mobile devices. PrimeSense’s technology could also be used to improve customer engagement, through the use of immersive, interactive displays and 3D technology incorporated into its Apple Store apps.

Currently, companies like Bodymetrics use the technology to allow customers to get their bodies scanned in-store and at home, which then facilitates virtual try-ons.

Competition

Though Microsoft originally relied on PrimeSense technology for the Kinect, it purchased PrimeSense rival Canesta in 2010 and has since developed its own in-house solutions for the Kinect motion sensing system that shipped with the recently released Xbox One.

At the time of Microsoft's purchase, PrimeSense rival Canesta was believed to have more potential as it utilized a specialized chip geared specifically towards 3D recognition, while PrimeSense was more affordable. It is unclear how the two technologies compare at the current point in time, but PrimeSense will undoubtedly flourish under Apple’s wing.

Future Use

While news of Apple’s PrimeSense acquisition first surfaced in July, confirmation of the purchase didn't come until November, suggesting a long period of negotiations. Apple is reported to have paid $345 million for PrimeSense, which employs approximately 150 employees in Tel Aviv and several other locations around the world.

It is not known how Apple plans to incorporate PrimeSense's sensors and software into its products, but there are a wide range of possibilities for the technology. According to PrimeSense, its Capri chip is immediately ready for deployment, but as the technology is already several months old and rather large, it is likely that Apple will work with PrimeSense on developing new technology custom built for Apple products.

Related Roundup: Apple Glasses

Top Rated Comments

LostSoul80 Avatar
91 months ago

Take that, make it smaller and add internals putting it on par with a PS4/Xbox one.

The result a beautiful Apple TV / gaming console !

Could it work ? I don't know, maybe ?


No.
Score: 8 Votes (Like | Disagree)
wovel Avatar
91 months ago

Niche product for a niche market.

If people truly want this type of experience, there are other ways to get it now.


That's what tablets were a few years ago.
Score: 4 Votes (Like | Disagree)
Komrad808 Avatar
91 months ago
Apple will start to implant technology just like "Minority Report". Walk into your home and the "future Apple TV" will automatically turn onto your favorite channel. Download data from your iWatch or iPhone and gather your productivity in the day to make your home into a relaxing environment....:D
Score: 4 Votes (Like | Disagree)
macduke Avatar
91 months ago
A thoughtful, well written and researched article. More of this please! While I love my speculative rumor tabloids, I also appreciate these in-depth pieces.

iPhones are finally to a point where you don't really need a faster one. Maybe if you're into high-end games. Otherwise Apple needs to come up with new ways to leverage the speed of these newer devices. Real-time 3D tracking? Sure, but I hope it's not a gimmick. If it can assist in navigating a user interface or somehow quickly provide me contextual information in my environment, then do it. The added benefit for Apple is there is yet another reason to upgrade. Now that we have the M7 motion coprocessor, I wonder if they could roll these new features into that for the M9 in the iPhone 6S in late 2015. Now that's just the type of speculative rumor tabloid story that I would read!
Score: 3 Votes (Like | Disagree)
FirstNTenderbit Avatar
91 months ago

Niche product for a niche market.

If people truly want this type of experience, there are other ways to get it now.


So if there are other ways, how is it a niche market? I don't think you understand the meaning of the word, really.


Sorry bud, TWSS37 is right. Unfortunately it's you who may not understand the meaning of niche market.

Niche markets have limited customers, the product makers could be many.
Supercars are a niche market yet there are several manufacturers. Right now 4K television is a niche market with several vendors. On and on...
Score: 3 Votes (Like | Disagree)
AppleMark Avatar
91 months ago

Seems kinda lame. But if that's where technology is heading...then so be it, I guess.

Agreed, if we are to assume that this tech is for a TV set, control, recognition etc.

But am I alone in just wanting a TV that works and is a revolution in function, design and not motion detection gimmicks?

Am I the only person who feels uncomfortable with what is basically a camera array pointed at you the whole time, in order to potentially meet the seemingly obligatory gimmick requirements these days?

I would be happy if they just gave me the best ever viewing experience merged with my ATV, App's, iOS and a revolutionary remote control. Most definitely control without anything that can potentially be used, or be hijacked to spy, or be used for targeted for advertising etc.
Score: 2 Votes (Like | Disagree)

Top Stories

0 Deals Hero

Black Friday 2020: Best Apple Deals to Plan For

Saturday November 21, 2020 10:00 am PST by
In the lead-up to Black Friday next week, we've been putting a spotlight on the best deals coming from various retailers like Best Buy and Walmart. In an effort to further prepare our readers for the best Black Friday deals, we're breaking down what we think should be on your radar for Black Friday in 2020. Note: MacRumors is an affiliate partner with some of these vendors. When you click a...
2020 apple shopping event

Apple Offering Up to $150 Gift Card With Select Products on Black Friday Through Cyber Monday

Monday November 23, 2020 2:53 am PST by
Apple has announced its annual four-day shopping event, offering customers up to a $150 Apple Store gift card with the purchase of select products between Black Friday and Cyber Monday in the United States. The gift card values in the United States are as follows: $150 for 16-inch MacBook Pro $150 for 21.5-inch iMac $50 for 13-inch MacBook Pro $50 for MacBook Air $50 for iPhone SE,...
m1 mac mini vignette

Apple Lists M1-Based Mac Mini Logic Boards With 10 Gigabit Ethernet in Internal Parts Ordering System

Friday November 20, 2020 9:32 am PST by
While the new Mac mini with the M1 chip is only available with Gigabit Ethernet, Apple has listed multiple M1-based Mac mini logic boards with 10 Gigabit Ethernet in an internal parts list for Apple Authorized Service Providers. For every Mac mini logic board with Gigabit Ethernet in the parts list, obtained by MacRumors, there is a corresponding logic board with 10 Gigabit Ethernet:...
ipad pro 2020 display

Black Friday Week Kicks Off With Up to $150 Savings on 2020 iPad Pro

Sunday November 22, 2020 2:37 pm PST by
As we head into Black Friday week, we're seeing some of the best deals of the season so far, with Amazon and Best Buy today discounting the latest iPad Pro models by up to $150 at the lowest prices we've ever tracked on these models. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep ...
macos big sur m1 macs restore issue

Apple Provides Instructions to Fix macOS Reinstallation Errors on M1 Macs

Sunday November 22, 2020 3:30 pm PST by
Shortly after the launch of Apple's new M1 Macs, we saw reports that attempts to restore and reinstall macOS on those machines right away could result in an installation error that would leave your Mac non-functional. Specifically, the error message would read: "An error occurred preparing the update. Failed to personalize the software update. Please try again." Over the weekend, Apple p...
iPhone 6s main

Rumor Claims iOS 15 to Drop Support for iPhone 6s and Original iPhone SE

Sunday November 22, 2020 9:25 am PST by
Apple will drop support for the iPhone SE, iPhone 6s, and iPhone 6s Plus in next year's release of iOS 15, according to a rumor shared today by Israeli site The Verifier. If the rumor is accurate, that would mean iOS 15 will be compatible with the following Apple devices: 2021 iPhone series iPhone 12 Pro Max iPhone 12 Pro iPhone 12 mini iPhone 12 iPhone 11 iPhone 11 Pro iPhone 11 Pro ...
new m1 chip

Craig Federighi: Native Windows on M1 Macs is 'Really up to Microsoft'

Friday November 20, 2020 11:57 am PST by
Following the release of the M1 Macs Apple executives have been doing interviews with a range of publications, and today, Ars Technica published another interview with software engineering chief Craig Federighi, hardware technologies lead Johny Srouji, and marketing VP Greg Joswiak. Much of the interview focuses on topics that the three have already covered in prior discussions, but there is ...
Walmart November Deals Hero

Black Friday Spotlight: Walmart Will Have AirPods Pro Down to Lowest Price of $169, and More Apple Deals

Thursday November 19, 2020 8:05 am PST by
We've been tracking early Black Friday deals in our dedicated Black Friday Roundup, and in an effort to prepare our readers for the big shopping event we're highlighting sales store-by-store in the lead-up to November 27. Note: MacRumors is an affiliate partner with Walmart. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. Next ...
xlK6MS4MLqVFaskl

iFixit Shares iPhone 12 Pro Max Teardown Revealing L-Shaped Battery and Bigger Camera Module

Sunday November 22, 2020 2:53 am PST by
In the last of its iPhone 12 series teardowns, iFixit has published its iPhone 12 Pro Max disassembly, which reveals some unique differences in the design of the internals, including the full extent of the larger camera system responsible for the improved low-light performance on Apple's largest iPhone to date. On opening the iPhone 12 Pro Max, the first obvious difference compared to the...
apple leather sleeve

Leather Sleeve for iPhone 12 Models Now Available From Apple

Friday November 20, 2020 12:16 pm PST by
Apple today began selling the Leather Sleeve for the new iPhone 12 models, with the accessory having first been announced alongside the updated iPhones in October. Priced at $129, the Leather Sleeve is not a case and is designed to be removed when the iPhone is in use. It features a cutout at the front that displays the time, and it comes with a matching leather strap. According to Apple, it ...