Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE
Tag: Halide
Buyer's Guide: iPhone SE (Neutral)
Related Forum: iPhone

Top Rated Comments

luvbug Avatar
34 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
34 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
34 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
34 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
34 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
34 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)

Popular Stories

Emergency SOS via Satellite iPhone YT

Apple's iPhone 14 Emergency SOS via Satellite Feature Saves Stranded Man in Alaska

Thursday December 1, 2022 4:37 pm PST by
With the launch of iOS 16.1, Apple rolled out a Emergency SOS via Satellite, which is designed to allow iPhone 14 owners to contact emergency services using satellite connectivity when no cellular or WiFi connection is available. The feature was put to the test in Alaska today, when a man became stranded in a rural area. In the early hours of the morning on December 1, Alaska State Troopers ...
iPhone Measure Height

Newer iPhones Allow You to Measure Someone's Height Instantly — Here's How

Saturday December 3, 2022 10:23 am PST by
iPhone 12 Pro and Pro Max, iPhone 13 Pro and Pro Max, and iPhone 14 Pro and Pro Max models feature a LiDAR Scanner next to the rear camera that can be used to measure a person's height instantly in Apple's preinstalled Measure app. To measure a person's height, simply open the Measure app, point your iPhone at the person you want to measure, and make sure they are visible on the screen from...
General iOS 16 Feature Yellow

iOS 16.2 for iPhone Launching This Month With These 8 New Features

Thursday December 1, 2022 8:44 am PST by
Apple plans to publicly release iOS 16.2 for the iPhone in mid-December, according to Bloomberg's Mark Gurman. The update remains in beta testing for now, with at least eight new features and changes already uncovered so far. iOS 16.2 introduces a number of new features, including Apple's new whiteboard app Freeform, two new Lock Screen widgets for Sleep and Medications, the ability to hide...
iOS 16

When Will iOS 16.2 Be Released?

Friday December 2, 2022 2:13 pm PST by
Apple in late October began testing iOS 16.2 and iPadOS 16.2 updates, providing betas to both developers and public beta testers. As of now, we've had four total betas, with the fourth beta having been released earlier this week. iOS 16.2 and iPadOS 16.2 are expected before the end of the year, and we thought we'd try to narrow down the launch timeline. With only four betas released since...
14 vs 16 inch mbp m2 pro and max feature 1

Major RAM Upgrade Coming to Next-Generation MacBook Pro

Friday December 2, 2022 2:03 am PST by
The next-generation MacBook Pro models could feature faster RAM, according to a recent report from a reliable source. MacRumors Forums member "Amethyst," who accurately revealed details about the Mac Studio and Studio Display before those products were announced, recently provided information about Apple's upcoming 14- and 16-inch MacBook Pro models. The new machines are expected to feature...
iPad 10 Battery Pull Tabs

iPad 10 Teardown Reveals Why Device Isn't Compatible With Apple Pencil 2

Thursday December 1, 2022 10:48 am PST by
Do-it-yourself repair website iFixit today shared a video teardown of Apple's new 10th-generation iPad, providing a closer look inside the tablet and revealing why the device lacks support for the second-generation Apple Pencil. The teardown reveals the internal layout of the iPad, including its two-cell 7,606 mAh battery, logic board with the A14 Bionic chip, and more. As suspected, the...
android apple fix rcs

Google Again Criticizes Apple for Not Adopting RCS for Messages App: 'Their Texting is Stuck in the 1990s'

Friday December 2, 2022 10:54 am PST by
Google is continuing on with its attempt to convince Apple to adopt the RCS messaging standard, publishing a new "it's time for RCS" blog post. Promoted heavily by Google, RCS or Rich Communication Services is a messaging standard that is designed to replace the current SMS messaging standard. It provides support for higher resolution photos and videos, audio messages, and bigger file sizes, ...
ios 16 2 beta notifiation center

PSA: Older Notifications No Longer Hidden in Notification Center in iOS 16.2 Beta 4

Friday December 2, 2022 5:23 am PST by
In a small but significant change to the way the Notification Center works in the latest iOS 16.2 beta, older notifications are now shown by default without having to swipe up. In the current release as well as earlier versions of iOS 16, users do not automatically see older notifications in the Notification Center like they did in iOS 15, and instead must manually swipe up from the middle...
lastpass

LastPass Hacked for Second Time This Year

Friday December 2, 2022 4:04 am PST by
Password management app LastPass says it is investigating a security incident after an "unauthorized party" compromised its systems on Wednesday and gained access to some customer information. The information was stored in a third-party cloud service shared by LastPass and parent company GoTo, said LastPass CEO Karim Toubba in a blog post. Toubba said the hackers used information stolen from ...
Apple Card Savings

Apple Card Customer Agreement Updated for 'Upcoming' Savings Account Feature

Friday December 2, 2022 11:43 am PST by
Goldman Sachs this week updated its Apple Card customer agreement to reflect the credit card's upcoming Daily Cash savings account feature, which was expected to launch with iOS 16.1 but appears to have been delayed. "To enable new ways to use Daily Cash like the upcoming Savings account feature, we are updating the Daily Cash Program section of your Apple Card Customer Agreement," reads an...