Apple Makes New On-Screen Game Controller Available to Developers on iOS 15 and iPadOS 15

During the Platforms State of the Union at WWDC this week, Apple unveiled a new API for iOS 15 and iPadOS 15 that enables developers to implement an on-screen virtual game controller in their iPhone and iPad games with just a few lines of code.

apple virtual game controller ios 15
While many iPhone and iPad games already offer on-screen controls, Apple's new virtual game controller is available to all developers, easy to add, and can be customized on a per-game basis. Apple said the on-screen controller can be adjusted to a variety of layouts, with up to four buttons and a thumbstick, d-pad, or touchpad available per left and right side.

"These new on-screen controls for iPhone and iPad look amazing, and they're carefully tuned for grip locations across hand sizes and for a great responsiveness and feel," said Nat Brown, an Apple engineer working on Game Technologies.

Apple offers a WWDC session and developer documentation with more details.

Related Roundups: iOS 15, iPadOS 15
Related Forum: iOS 15

Top Rated Comments

dannyyankou Avatar
7 weeks ago
On screen game controllers have never worked and will never work. It’s very difficult to keep your eyes on the gameplay and controls at the same time. You need physical buttons.
Score: 25 Votes (Like | Disagree)
TheDailyApple Avatar
7 weeks ago
I find the best on screen controllers are the ones designed from the ground up for touch screen instead on those trying to replicate standard game controls.
Score: 10 Votes (Like | Disagree)
TheDailyApple Avatar
7 weeks ago

On screen game controllers have never worked and will never work. It’s very difficult to keep your eyes on the gameplay and controls at the same time. You need physical buttons.
While I tend to agree with you, this could help with that. If multiple games share the same on screen controls then it will be easier to memorize button location and placement. Similar to how I can type on the 12.9” iPad’s software keyboard without looking just like I do on a real one (albeit more slowly) because the key placement is the same.

Obviously a heavy gamer should buy a real controller just as a coder or writer should buy a real keyboard. But as light/infrequent gamers are unlikely to invest in one, it’s worth optimizing the on screen interface.
Score: 9 Votes (Like | Disagree)
RadioHedgeFund Avatar
7 weeks ago

On screen game controllers have never worked and will never work. It’s very difficult to keep your eyes on the gameplay and controls at the same time. You need physical buttons.
I don’t know, Caster (http://www.elecorn.com/caster3d/) largely nailed the core aspects of it in 2009 by just using the position of your thumb from the first point it hits the screen. It is entirely possible to use the orientation of your character or vehicle as a visual cue for where you need to go to next. Controlling a camera using the same method is no different to using a mouse.

Meteor Blitz (https://toucharcade.com/2009/09/16/meteor-blitz-dual-stick-space-shooting-goodness/) also showed how you can use orientation and visual cues to make up for a lack of buttons. Geometry Wars 3 on iOS was and still is a fantastic port.

Where touchscreen controls do fall down is when devs tried to port over console titles to a phone and have to shoehorn touchscreen controls into games not designed for them. This does not work.

There are exceptions, for example games with a largely static camera like FFVII but this pales in comparison to proper ports like FFIX which have been redesigned with the host platform in mind. See also titles like Horizon Chase and Oceanhorn 1.
Score: 7 Votes (Like | Disagree)
LFC2020 Avatar
7 weeks ago
This is why they must of released a 16gb ram iPad ?? since the ram is useless for anything else on iPadOS at the moment.
Score: 5 Votes (Like | Disagree)
Joniz Avatar
7 weeks ago
Yeah, I can see someone making a game where these are a set of boobs or something.
Score: 5 Votes (Like | Disagree)

Top Stories

apple virtual game controller ios 15

Apple Gaming Controller Arrives in Unexpected Way

Friday June 18, 2021 3:10 am PDT by
Apple has long been rumored to be developing on a gaming controller, and at WWDC this month, a form of Apple's gaming controller appears to have emerged, albeit not quite in the way that some users had hoped for. Apple has clearly expanded its interest in gaming controllers over the past year, integrating full support for even the latest Xbox and PlayStation controllers into iOS, iPadOS, macO...
ps5 controller apple online store

iOS 15 and macOS Monterey Introduce Ability to Capture 15-Second Gameplay Recordings Using a Game Controller

Tuesday June 8, 2021 5:23 pm PDT by
Apple's new iOS 15, iPadOS 15, and macOS Monterey software updates unveiled this week introduce the ability to capture and save the last 15 seconds of gameplay at any time by long pressing a designated button on a supported game controller. Instead of having to remember to start and stop a screen recording while playing a game, this optional new feature allows iPhone, iPad, and Mac users to...
playdate1

'Playdate' Handheld Game System Now Available for Pre-Order

Thursday July 29, 2021 10:17 am PDT by
Panic, a company known for its range of apps designed for Apple's iPhones, iPads, and Macs, today announced the launch of pre-orders for its Playdate handheld gaming system. Panic had 20,000 Playdate systems available for preorder for shipping in late 2021, but those quickly sold out and orders placed now will ship out in 2022. Priced at $179, the Playdate is a pocket-sized gaming system ...
apple arcade asphalt 8

Two New Games Coming to Apple Arcade, Including Gameloft's Racing Classic 'Asphalt 8: Airborne'

Tuesday July 27, 2021 10:00 am PDT by
Apple today announced two games that are coming soon to Apple Arcade, including the classic racing game "Asphalt 8: Airborne" from Gameloft and a new Zelda-inspired roleplaying game "Baldo" from Italian game studio NAPS team. Originally released on the App Store in 2013, "Asphalt 8: Airborne" is coming to Apple Arcade, but no specific release date has been announced. The popular racing game...
apple arcade

Eight New Games Came to Apple Arcade This Month, Including 'Solitaire Stories,' 'Ultimate Rivals: The Court,' and 'Detonation Racing'

Friday July 30, 2021 7:51 am PDT by
July was a busy month for Apple Arcade, with eight new titles joining the library of almost 200 games on Apple's growing mobile gaming subscription service, including some familiar, fan-favorite games. Last month, "Legends of Kingdom Rush," "Game Dev Story+," "Frenzic: Overtime," "INKS.+," and "Leo's Fortune+" arrived on Apple Arcade. In July, eight new games joined the platform in the card, ...
Netflix

Netflix to Take on Apple Arcade With Mobile Games for Subscribers

Tuesday July 20, 2021 1:32 pm PDT by
Netflix today confirmed that it is planning to expand on its TV movie offerings with a new gaming offering. During the company's second quarter earnings report (via CNET), Netflix said that gaming is "another new content category," similar to its expansion into original films, animation, and unscripted TV. Netflix plans to focus primarily on games that are designed for mobile devices, with...
applearcade

This Month on Apple Arcade: 'Legends of Kingdom Rush,' 'Game Dev Story+,' 'Frenzic: Overtime,' 'INKS.+,' and 'Leo's Fortune+'

Friday June 25, 2021 8:45 am PDT by
This month, "Legends of Kingdom Rush," "Game Dev Story+," "Frenzic: Overtime," "INKS.+," and "Leo's Fortune+" arrived on Apple Arcade, joining the expanding library of almost 200 games already on Apple's exclusive-based mobile gaming subscription service. While there were no new games in May, in April, Apple Arcade gained over 30 games, including a large number of classic titles such as "Cut ...
Facebook gaming web app

Facebook Gaming Now Available to iOS Users Through Web App Due to App Store Policy

Friday July 23, 2021 8:29 am PDT by
Due to limitations surrounding the distribution of cloud gaming services on the App Store, Facebook is bringing its gaming service to iOS users through a progressive web app starting today, reports The Verge. Apple's App Store guidelines don't permit cloud-game-based apps to appear on the platform unless developers submit each app offered in the service individually for review. The...
Netflix

Apple Arcade Facing More Competition as Netflix Plans Expansion Into Video Games

Wednesday July 14, 2021 3:39 pm PDT by
Netflix is planning to add to its TV and movie offerings by breaking into video game content, reports Bloomberg. Netflix has hired former Electronic Arts and Facebook executive Mike Verdu as vice president of game development, and he will report to Netflix chief operating officer Greg Peters. At Facebook, Verdu worked with developers to bring games and other content to the Oculus VR headset. ...
ps5 controller apple online store

Apple Begins Selling PS5 DualSense Controller After Compatibility With iPhone, iPad, Apple TV, and Mac Added

Friday May 21, 2021 6:26 am PDT by
Apple recently started selling the white PS5 DualSense wireless game controller through its online store, with pricing consistent with Sony at $69.95. Orders placed today are estimated to be delivered next week in the United States. iOS 14.5, iPadOS 14.5, tvOS 14.5, and macOS 11.3 software updates released last month added support for PS5 and Xbox Series X controllers on the iPhone, iPad,...