Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE 2020
Tag: Halide
Buyer's Guide: iPhone SE (Don't Buy)
Related Forum: iPhone

Top Rated Comments

luvbug Avatar
23 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
23 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
23 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
23 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
23 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
23 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)

Popular Stories

maxresdefault

Review: M1 Max MacBook Pro After Three Months

Wednesday January 19, 2022 11:30 am PST by
It's now been a few months since the M1 Pro and M1 Max MacBook Pro models launched in October, and MacRumors video editor Dan Barbera has been using one of the new machines since they debuted. Over on the MacRumors YouTube channel, Dan has shared a three month review of his MacBook Pro to see how it has held up over time and how it's changed his workflow. Subscribe to the MacRumors YouTube ...
iphone se 2020 top

New iPhone SE Likely to Launch in April Based on Production Timeframe

Wednesday January 19, 2022 6:44 am PST by
Apple suppliers will begin producing display panels for the third-generation iPhone SE this month, with final assembly of the device likely to start in March, according to information shared by display industry consultant Ross Young. Based on this production timeframe, Young believes the third-generation iPhone SE is likely to launch in the second half of April, or perhaps in early May at...
AirPods 3 New Firmware Feature

Apple Updates AirPods 3 Firmware to Version 4C170

Tuesday January 18, 2022 11:46 am PST by
Apple today released a new 4C170 firmware update for the AirPods 3, an update from the prior 4C165 that was made available in December. Apple does not offer details on what's included in new firmware updates for the AirPods‌, so we don't know what improvements or bug fixes the new firmware brings. There is no standard way to upgrade the ‌AirPods‌‌ software, but firmware is...
iPad Air Feature 2 green

New Apple Products Filed in Regulatory Database, Likely Including New iPhone SE and iPad Air

Tuesday January 18, 2022 6:11 am PST by
Apple today filed unreleased iPhone and iPad models in the Eurasian Economic Commission database, as spotted by French blog Consomac. The filings likely represent the rumored third-generation iPhone SE, fifth-generation iPad Air, and potentially more. The unreleased iPhone models have the identifiers A2595, A2783, and A2784, while the unreleased iPad models have the identifiers A2588, A2589, ...
iphone 5g mmwave

U.S. Airlines Warn of 'Catastrophic' Crisis With Impending 5G Rollout, AT&T and Verizon Agree to Delay Around Airports

Tuesday January 18, 2022 10:35 am PST by
Verizon and AT&T's upcoming rollout of new C-Band 5G technology could cause chaos and lead to widespread delays of passenger and cargo flights, major U.S. airlines said on Monday in a letter sent to the White House National Economic Council, the FAA, and the FCC (via Reuters). "Unless our major hubs are cleared to fly, the vast majority of the traveling and shipping public will essentially...
microsoft office icons

Microsoft Releases Office for Mac Update With Full Apple Silicon Support in Excel

Tuesday January 18, 2022 4:15 am PST by
Microsoft has released a new version of its Office for Mac productivity suite that includes an updated Excel app with 100% native support for Apple silicon machines. According to the release notes accompanying version 16.57, Excel will now run natively on Macs powered by Apple's M1-series processors without having to use the Rosetta 2 translation layer, which means anyone using a Mac with an ...
appleeducation

Apple's US Education Store Now Requires Institution Verification to Buy Discounted Products

Wednesday January 19, 2022 2:22 am PST by
Apple is now requiring that customers in the United States verify that they're active students, teachers, or staff members at an educational institution in order to access education discounts on products. Previously, little verification was needed for customers to purchase products through Apple's education store in the United States. Apple's education stores offer models of the iPad and Mac ...
iPhone 14 Mock pill and hole thumb

ProMotion Now Expected to Remain Exclusive to iPhone 14 Pro Models, Not Expand to Entire Lineup

Sunday January 16, 2022 8:56 am PST by
Continuing the tradition set with the iPhone 13 Pro, only the highest-end iPhone 14 models will feature Apple's ProMotion display technology, according to a respected display analyst. Ross Young, who on multiple occasions has detailed accurate information about Apple's future products, said in a tweet that ProMotion will not be expanded to the entire iPhone 14 lineup and will remain...
Beyond iPhone 13 Better Blue Face ID

Rumored iPhone 14 Pro Feature Delayed Until Next Year

Wednesday January 19, 2022 3:00 am PST by
Apple's long-rumored under-display Face ID technology will not be coming to the iPhone 14 Pro, according to recent reports. Based on the latest rumors, it now looks unlikely that the 6.1-inch iPhone 14 Pro and 6.7-inch iPhone 14 Pro Max will feature Face ID technology under the display as several reports claimed over the past year. Display industry consultant Ross Young, who often reveals ...