Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE
Tag: Halide
Buyer's Guide: iPhone SE (Caution)
Related Forum: iPhone

Top Rated Comments

luvbug Avatar
50 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
50 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
50 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
50 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
50 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
50 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)

Popular Stories

Apple car wheel icon feature yellow

Apple Cancels Electric Car Project

Tuesday February 27, 2024 11:05 am PST by
Apple has canceled all plans to release an autonomous, electric vehicle, reports Bloomberg. Apple has been working on an Apple Car for more than a decade and invested millions of dollars into development before deciding it was not a viable project. Apple's Chief Operating Officer Jeff Williams today told approximately 2,000 employees working on the Apple Car that the project was canceled,...
iOS 18 Mock iPhone 16 Feature Gray

iOS 18 Rumored to Be Compatible With These iPhone Models

Tuesday February 27, 2024 6:31 am PST by
iOS 18 will be compatible with the iPhone XR, and thereby also the iPhone XS and iPhone XS Max models with the same A12 Bionic chip, according to a post on X today from a private account with a proven track record of sharing build numbers for upcoming iOS updates. The post was spotted by MacRumors contributor Aaron Perris, and it has since been deleted. However, this was likely because the...
Google maps feaure

Google Maps Finally Rolls Out Glanceable Directions

Wednesday February 28, 2024 2:07 am PST by
After more than a year since announcing the feature, Google Maps is finally rolling out glanceable directions on Android and iOS (via Android Police). The feature allows users to view turn-by-turn directions and a live ETA directly from their device's lock screen – information that was previously only visible when a phone was unlocked. Glanceable directions also work on the app's route...
iPad Air 5

iPadOS 18 Rumored to Drop Support for These iPad Models

Tuesday February 27, 2024 6:55 am PST by
iPadOS 18 will drop support for iPad models equipped with the A10X Fusion chip, according to a post on X today from a private account with a proven track record of sharing build numbers for upcoming iOS and iPadOS updates. This means that iPadOS 18 would not be compatible with the first-generation 10.5-inch iPad Pro or the second-generation 12.9-inch iPad Pro models released in 2017. It...
iOS 17

iOS 17.4 Coming Soon With These New Features for Your iPhone

Monday February 26, 2024 6:08 am PST by
In a press release last month, Apple confirmed that iOS 17.4 will be released in March, and the update includes several new features and changes for the iPhone. Key new features in iOS 17.4 include major App Store changes in the EU, Apple Podcasts transcripts, and an iMessage security upgrade. The update also adds new emoji and includes preparations for the launch of next-generation CarPlay...
M3 MacBook Air Feature

New MacBook Air Models Launching Next Month: 5 Features to Expect

Wednesday February 28, 2024 1:50 am PST by
The existing 15-inch MacBook Air arrived in June 2023, which is not that long ago in terms of Mac update cycles. However, Apple released the current 13-inch ‌MacBook Air back in June 2022. It is now the oldest Mac in Apple's current crop, having not been updated in 600 days. But rumors suggest that is unlikely to be the case for much longer. According to Bloomberg's Mark Gurman, Apple has...