Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE
Tag: Halide
Buyer's Guide: iPhone SE (Don't Buy)
Related Forum: iPhone

Popular Stories

Apple AI Command Center Concept Mock 3

Apple Expected to Launch This All-New Device Next Year

Wednesday November 27, 2024 1:05 pm PST by
Apple is expected to kick off 2025 by launching an all-new smart home hub, also referred to as a "command center," as early as March. The hub is expected to feature around a six-inch display that can be attached to a tabletop base with a speaker, or mounted on a wall. The device is said to run a new "homeOS" operating system with a customizable widget-focused home screen, and it is expected...
Whatsapp Feature

WhatsApp to Drop Support for These iPhones Starting May 2025

Monday December 2, 2024 2:57 am PST by
WhatsApp is set to end support for iOS versions older than iOS 15.1 from May next year, removing the chat platform's compatibility with several iPhone models in the process. From May 5, 2025, WhatsApp will no longer be compatible with iPhone 5s, iPhone 6, and iPhone 6 Plus models. Users with those devices won't be able to access the encrypted chat service after the specified date unless they ...
New Things Your iPhone Can Do in iOS 18

18 New Things Your iPhone Can Do in iOS 18.2

Wednesday November 27, 2024 5:05 am PST by
Apple is set to release iOS 18.2 in early December, bringing the second round of Apple Intelligence features to iPhone 15 Pro and iPhone 16 models. This update brings several major advancements to Apple's AI integration, including completely new image generation tools and a range of Visual Intelligence-based enhancements. There are a handful of new non-AI related feature controls incoming as...
iOS 18

Here Are Apple's Full Release Notes for iOS 18.2

Thursday December 5, 2024 11:48 am PST by
Apple seeded the release candidate version of iOS 18.2 today, which means it's going to see a public launch imminently. Release candidates represent the final version of new software that will be provided to the public should no last minute bugs be found, and Apple includes release notes with the RC launch. The iOS 18.2 release notes provide a look at all of the new features that are coming...
Flip iPhone Thumb 1

Apple's 2026 Foldable iPhone Could Reinvigorate Stalling Market

Monday December 2, 2024 4:04 pm PST by
The foldable smartphone market has stalled with customer interest in foldables waning, but that could change when Apple debuts a foldable iPhone, according to display analyst Ross Young. In a report on the current foldable smartphone market, Young says that Apple is expected to "enter the foldable market" in the second half of 2026. Apple's "dominant position in flagship smartphones" could...
airpods pro 2 gradient

AirPods Pro 3 Expected Next Year: Here's What We Know

Thursday November 28, 2024 3:30 am PST by
Despite being released over two years ago, Apple's AirPods Pro 2 continue to dominate the wireless earbud market. However, with the AirPods Pro 3 expected to launch sometime in 2025, anyone thinking of buying Apple's premium earbuds may be wondering if the next generation is worth holding out for. Apart from their audio and noise-canceling performance, which are generally regarded as...
Tim Cook WWDC 2024

Apple CEO Tim Cook Opens Up About AI Plans, Vision Pro Future, and More in New Interview

Wednesday December 4, 2024 5:40 am PST by
WIRED today shared in an in-depth interview with Apple CEO Tim Cook, asking questions about AI, Vision Pro sales, pre-recorded keynotes, and more. The wide-ranging interview covers Apple's pivot toward AI technology, including what Apple Intelligence features Cook finds most useful, Apple's partnership with OpenAI, and the environmental impact of AI. For example, WIRED asked Cook about...
iPhone 17 Pro Dual Tone Rectangle Feature 1

iPhone 17 Pro Already Rumored to Have These 8 New Features

Wednesday November 27, 2024 12:19 pm PST by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch for 10 more months, there are already plenty of rumors about the devices. An imaginative iPhone 17 Pro concept based on rumors Below, we recap key changes rumored for the iPhone 17 Pro models so far: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro ...

Top Rated Comments

luvbug Avatar
60 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
60 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
60 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
60 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
60 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
60 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)