Apple Makes New On-Screen Game Controller Available to Developers on iOS 15 and iPadOS 15

During the Platforms State of the Union at WWDC this week, Apple unveiled a new API for iOS 15 and iPadOS 15 that enables developers to implement an on-screen virtual game controller in their iPhone and iPad games with just a few lines of code.

apple virtual game controller ios 15
While many iPhone and iPad games already offer on-screen controls, Apple's new virtual game controller is available to all developers, easy to add, and can be customized on a per-game basis. Apple said the on-screen controller can be adjusted to a variety of layouts, with up to four buttons and a thumbstick, d-pad, or touchpad available per left and right side.

"These new on-screen controls for iPhone and iPad look amazing, and they're carefully tuned for grip locations across hand sizes and for a great responsiveness and feel," said Nat Brown, an Apple engineer working on Game Technologies.

Apple offers a WWDC session and developer documentation with more details.

Related Forum: iOS 15

Top Rated Comments

dannyyankou Avatar
24 months ago
On screen game controllers have never worked and will never work. It’s very difficult to keep your eyes on the gameplay and controls at the same time. You need physical buttons.
Score: 25 Votes (Like | Disagree)
TheDailyApple Avatar
24 months ago
I find the best on screen controllers are the ones designed from the ground up for touch screen instead on those trying to replicate standard game controls.
Score: 10 Votes (Like | Disagree)
TheDailyApple Avatar
24 months ago

On screen game controllers have never worked and will never work. It’s very difficult to keep your eyes on the gameplay and controls at the same time. You need physical buttons.
While I tend to agree with you, this could help with that. If multiple games share the same on screen controls then it will be easier to memorize button location and placement. Similar to how I can type on the 12.9” iPad’s software keyboard without looking just like I do on a real one (albeit more slowly) because the key placement is the same.

Obviously a heavy gamer should buy a real controller just as a coder or writer should buy a real keyboard. But as light/infrequent gamers are unlikely to invest in one, it’s worth optimizing the on screen interface.
Score: 9 Votes (Like | Disagree)
RadioHedgeFund Avatar
24 months ago

On screen game controllers have never worked and will never work. It’s very difficult to keep your eyes on the gameplay and controls at the same time. You need physical buttons.
I don’t know, Caster (http://www.elecorn.com/caster3d/) largely nailed the core aspects of it in 2009 by just using the position of your thumb from the first point it hits the screen. It is entirely possible to use the orientation of your character or vehicle as a visual cue for where you need to go to next. Controlling a camera using the same method is no different to using a mouse.

Meteor Blitz (https://toucharcade.com/2009/09/16/meteor-blitz-dual-stick-space-shooting-goodness/) also showed how you can use orientation and visual cues to make up for a lack of buttons. Geometry Wars 3 on iOS was and still is a fantastic port.

Where touchscreen controls do fall down is when devs tried to port over console titles to a phone and have to shoehorn touchscreen controls into games not designed for them. This does not work.

There are exceptions, for example games with a largely static camera like FFVII but this pales in comparison to proper ports like FFIX which have been redesigned with the host platform in mind. See also titles like Horizon Chase and Oceanhorn 1.
Score: 7 Votes (Like | Disagree)
LFC2020 Avatar
24 months ago
This is why they must of released a 16gb ram iPad ?? since the ram is useless for anything else on iPadOS at the moment.
Score: 5 Votes (Like | Disagree)
Joniz Avatar
24 months ago
Yeah, I can see someone making a game where these are a set of boobs or something.
Score: 5 Votes (Like | Disagree)

Popular Stories

iOS 16

Apple Releases iOS 16.4 With New Emoji, Safari Web Push Notifications, Beta Changes, Voice Isolation for Calls and More

Monday March 27, 2023 10:03 am PDT by
Apple today released iOS 16.4, the fourth major update to the iOS 16 operating system that initially came out last September. iOS 16.4 comes two months after the launch of iOS 16.3, an update that added Security Keys for Apple ID. iOS 16‌.4 and iPadOS 16.4 can be downloaded on eligible iPhones and iPads over-the-air by going to Settings > General > Software Update. It can take a few minutes...
iOS 16

iOS 16.4 Will Add These 8 New Features to Your iPhone

Sunday March 26, 2023 8:06 am PDT by
Following nearly six weeks of beta testing, iOS 16.4 is expected to be released to the public as soon as this week. The software update includes a handful of new features and changes for the iPhone 8 and newer. To install an iOS update, open the Settings app on the iPhone, tap General → Software Update, and follow the on-screen instructions. Below, we have recapped eight new features and...
apple mixed reality headset concept by david lewis and marcus kane

Some Apple Employees Seriously Concerned About Mixed-Reality Headset as Announcement Draws Closer

Sunday March 26, 2023 8:25 am PDT by
Some Apple employees are concerned about the usefulness and price point of the company's upcoming mixed-reality headset, The New York Times reports. Apple headset concept by David Lewis and Marcus Kane Initial enthusiasm around the device at the company has apparently become skepticism, according to eight current and former Apple employees speaking to The New York Times. The change of tone...
iOS 17 on Phone Feature

Gurman: iOS 17 to Provide Several 'Most Requested Features'

Sunday March 26, 2023 6:05 am PDT by
Apple changed the strategy for iOS 17 later in its development process to add several new features, suggesting that the update may be more significant than previously thought, Bloomberg's Mark Gurman reports. In January, Gurman said that iOS 17 could be a less significant update than iPhone updates in previous years due to the company's intense focus on its long-awaited mixed-reality...
apple tv 4k red image

Apple Releases tvOS 16.4 for Apple TV 4K and Apple TV HD

Monday March 27, 2023 10:00 am PDT by
Apple today released tvOS 16.4, the fourth major point update to the tvOS 16 operating system that came out last September. Available for the Apple TV 4K and Apple TV HD, tvOS 16.4 comes two months following the release of tvOS 16.3. The tvOS 16.4 update can be downloaded over the air through the Settings app on the ‌‌‌‌Apple TV‌‌‌‌ by going to System > Software Update....
Steve Jobs Theater dusk

Apple Reportedly Demoed Mixed-Reality Headset to Executives in the Steve Jobs Theater Last Week

Sunday March 26, 2023 5:53 am PDT by
Apple showcased its mixed-reality headset to the company's top 100 executives in the Steve Jobs Theater last week, according to Bloomberg's Mark Gurman. In the latest edition of his "Power On" newsletter, Gurman explained that the "momentous gathering" is a "key milestone" ahead of the headset's public announcement planned for June. The event was intended to rally Apple's top members of...
Apple Music Classical

Apple Explains Why It Launched an iPhone App Dedicated to Classical Music

Monday March 27, 2023 8:54 pm PDT by
Apple today published a support document explaining why it decided to release a standalone Apple Music Classical app for classical music. In short, Apple says the app was designed to support classical music's complex metadata:Classical music is different. It has longer and more detailed titles, multiple artists for each work, and hundreds of recordings of well-known pieces. The Apple Music...
iPhone 15 Pro Buttons CAD Leak

iPhone 15 Pro Low-Energy Chip to Allow Solid-State Buttons to Work When Device is Off or Out of Battery

Wednesday March 29, 2023 1:54 am PDT by
The iPhone 15 Pro and Pro Max will use a new ultra-low energy microprocessor allowing certain features like the new capacitive solid-state buttons to remain functional even when the handset is powered off or the battery has run out, according to a source that shared details on the MacRumors forums. CAD-based render of new solid-state buttons on iPhone 15 Pro models The source of this rumor is ...