Starting in iOS 14 and macOS Big Sur, developers will be able to add the capability to detect human body and hand poses in photos and videos to their apps using Apple's updated Vision framework, as explained in this WWDC 2020 session.
This functionality will allow apps to analyze the poses, movements, and gestures of people, enabling a wide variety of potential features. Apple provides some examples, including a fitness app that could automatically track the exercise a user performs, a safety-training app that could help employees use correct ergonomics, and a media-editing app that could find photos or videos based on pose similarity.
Hand pose detection in particular promises to deliver a new form of interaction with apps. Apple's demonstration showed a person holding their thumb and index finger together and then being able to draw in an iPhone app without touching the display.
Additionally, apps could use the framework to overlay emoji or graphics on a user's hands that mirror the specific gesture, such as a peace sign.
Another example is a camera app that automatically triggers photo capture when it detects the user making a specific hand gesture in the air.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing. The algorithm can also experience difficulties if a person is close to edge of the screen or partially obstructed.
Similar functionality is already available through ARKit, but it is limited to augmented reality sessions and only works with the rear-facing camera on compatible iPhone and iPad models. With the updated Vision framework, developers have many more possibilities.
Despite being more than two years old, Apple's AirPods Pro 2 still dominate the premium wireless‑earbud space, thanks to a potent mix of top‑tier audio, class‑leading noise cancellation, and Apple's habit of delivering major new features through software updates. With AirPods Pro 3 widely expected to arrive in 2025, prospective buyers now face a familiar dilemma: snap up the proven...
Tuesday April 22, 2025 10:22 am PDT by Juli Clover
Apple plans to release an all-new super thin iPhone this year, debuting it alongside the iPhone 17, iPhone 17 Pro, and iPhone 17 Pro Max. We've seen pictures of dummy models, cases, and renders with the design, but Lewis Hilsenteger of Unbox Therapy today showed off newer dummy models that give us a better idea of just how thin the "iPhone 17 Air" will be.
The iPhone 17 Air is expected to be ...
Thursday April 17, 2025 4:12 am PDT by Tim Hardwick
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models simultaneously, which is why we often get rumored features months ahead of launch. The iPhone 17 series is no different, and we already have a good idea of what to expect from Apple's 2025 smartphone lineup.
If you skipped the iPhone...
Tuesday April 22, 2025 5:01 am PDT by Tim Hardwick
A developer has demonstrated Windows 11 ARM running on an M2 iPad Air using emulation, which has become much easier since the EU's Digital Markets Act (DMA) regulations came into effect.
As spotted by Windows Latest, NTDev shared an instance of the emulation on social media and posted a video on YouTube (embedded below) demonstrating it in action. The achievement relies on new EU regulatory...
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices.
Subscribe to the MacRumors YouTube channel for more videos.
Below, we recap key changes rumored for the iPhone 17 Pro models as of April 2025:
Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and ...
Apple seeded the third beta of iOS 18.5 to developers today, and so far the software update includes only a few minor changes.
The changes are in the Mail and Settings apps.
In the Mail app, you can now easily turn off contact photos directly within the app, by tapping on the circle with three dots in the top-right corner.
In the Settings app, AppleCare+ coverage information is more...
Tuesday April 15, 2025 6:31 am PDT by Tim Hardwick
Apple is preparing a "bold" new iPhone Pro model for the iPhone's 20th anniversary in 2027, according to Bloomberg's Mark Gurman. As part of what's being described as a "major shake-up," Apple is said to be developing a design that makes more extensive use of glass – and this could point directly to the display itself.
Here's the case for Apple releasing a truly all-screen iPhone with no...
Wednesday April 16, 2025 11:28 am PDT by Joe Rossignol
If you have been experiencing issues with wireless CarPlay in your vehicle lately, it was likely due to a software bug that has now been fixed.
Apple released iOS 18.4.1 today, and the update's release notes say it "addresses a rare issue that prevents wireless CarPlay connection in certain vehicles."
If wireless CarPlay was acting up for you, updating your iPhone to iOS 18.4.1 should...
Honestly, this seems like the kinda stuff that'd make Apple AR compelling—being able to draw in midair means you’d also be able to navigate an interface in midair with just your hands.
Using AR/VR without bringing a controller everywhere seems analogous to what set the iPhone apart from other touchscreen phones in 2007; you didn’t need a stylus.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing.
I don't think the wizarding community is going to be too happy about this...
