Starting in iOS 14 and macOS Big Sur, developers will be able to add the capability to detect human body and hand poses in photos and videos to their apps using Apple's updated Vision framework, as explained in this WWDC 2020 session.
This functionality will allow apps to analyze the poses, movements, and gestures of people, enabling a wide variety of potential features. Apple provides some examples, including a fitness app that could automatically track the exercise a user performs, a safety-training app that could help employees use correct ergonomics, and a media-editing app that could find photos or videos based on pose similarity.
Hand pose detection in particular promises to deliver a new form of interaction with apps. Apple's demonstration showed a person holding their thumb and index finger together and then being able to draw in an iPhone app without touching the display.
Additionally, apps could use the framework to overlay emoji or graphics on a user's hands that mirror the specific gesture, such as a peace sign.
Another example is a camera app that automatically triggers photo capture when it detects the user making a specific hand gesture in the air.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing. The algorithm can also experience difficulties if a person is close to edge of the screen or partially obstructed.
Similar functionality is already available through ARKit, but it is limited to augmented reality sessions and only works with the rear-facing camera on compatible iPhone and iPad models. With the updated Vision framework, developers have many more possibilities.
Thursday July 10, 2025 4:54 am PDT by Tim Hardwick
Apple will launch its new iPhone 17 series in two months, and the iPhone 17 Pro models are expected to get a new design for the rear casing and the camera area. But more significant changes to the lineup are not expected until next year, when the iPhone 18 models arrive.
If you're thinking of trading in your iPhone for this year's latest, consider the following features rumored to be coming...
A new Apple TV is expected to be released later this year, and a handful of new features and changes have been rumored for the device.
Below, we recap what to expect from the next Apple TV, according to rumors.
Rumors
Faster Wi-Fi Support
The next Apple TV will be equipped with Apple's own combined Wi-Fi and Bluetooth chip, according to Bloomberg's Mark Gurman. He said the chip supports ...
Apple's next-generation iPhone 17 Pro and iPhone 17 Pro Max are only two months away, and there are plenty of rumors about the devices.
Below, we recap key changes rumored for the iPhone 17 Pro models.
Latest Rumors
These rumors surfaced in June and July:A redesigned Dynamic Island: It has been rumored that all iPhone 17 models will have a redesigned Dynamic Island interface — it might ...
Thursday July 10, 2025 12:38 pm PDT by Juli Clover
Apple does not plan to refresh any Macs with updated M5 chips in 2025, according to Bloomberg's Mark Gurman. Updated MacBook Air and MacBook Pro models are now planned for the first half of 2026.
Gurman previously said that Apple would debut the M5 MacBook Pro models in late 2025, but his newest report suggests that Apple is "considering" pushing them back to 2026. Apple is now said to be...
iPhone 17 Pro and iPhone 17 Pro Max models with displays made by BOE will be sold exclusively in China, according to a new report.
Last week, it emerged that Chinese display manufacturer BOE was aggressively ramping up its OLED production capacity for future iPhone models as part of a plan to recapture a major role in Apple's supply chain.
Now, tech news aggregator Jukan Choi reports...
The long wait for an Apple Watch Ultra 3 is nearly over, and a handful of new features and changes have been rumored for the device.
Below, we recap what to expect from the Apple Watch Ultra 3:Satellite connectivity for sending and receiving text messages when Wi-Fi and cellular coverage is unavailable
5G support, up from LTE on the Apple Watch Ultra 2
Likely a wide-angle OLED display that ...
In select U.S. states, residents can add their driver's license or state ID to the Wallet app on the iPhone and Apple Watch, providing a convenient and contactless way to display proof of identity or age at select airports and businesses, and in select apps.
Unfortunately, this feature continues to roll out very slowly since it was announced in 2021, with only nine U.S. states, Puerto Rico,...
The iPhone 17 and iPhone 17 Air will be available in a total of nine color options, according to new information coming out of Asia.
The iPhone 17 Air's expected color options.
According to the leaker going by the account name "yeux1122" on the Korean blog Naver, accessory manufacturers are now producing camera protector rings for the iPhone 17 and iPhone 17 Air in colors to match their...
Honestly, this seems like the kinda stuff that'd make Apple AR compelling—being able to draw in midair means you’d also be able to navigate an interface in midair with just your hands.
Using AR/VR without bringing a controller everywhere seems analogous to what set the iPhone apart from other touchscreen phones in 2007; you didn’t need a stylus.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing.
I don't think the wizarding community is going to be too happy about this...
Biggest design overhaul since iOS 7 with Liquid Glass, plus new Apple Intelligence features and improvements to Messages, Phone, Safari, Shortcuts, and more. Developer beta available now ahead of public beta in July.
