Apple Teams Up With NVIDIA to Speed Up AI Language Models

Apple has shared details on a collaboration with NVIDIA to greatly improve the performance of large language models (LLMs) by implementing a new text generation technique that offers substantial speed improvements for AI applications.

ml research apple
Apple earlier this year published and open-sourced Recurrent Drafter (ReDrafter), an approach that combines beam search and dynamic tree attention methods to accelerate text generation. Beam search explores multiple potential text sequences at once for better results, while tree attention organizes and removes redundant overlaps among these sequences to improve efficiency.

Apple has now integrated the technology into NVIDIA's TensorRT-LLM framework, which optimizes LLMs running on NVIDIA GPUs, where it achieved "state of the art performance," according to Apple. The integration saw the technique manage a 2.7x speed increase in tokens generated per second during testing with a production model containing tens of billions of parameters.

Apple says the improved performance not only reduces user-perceived latency but also leads to decreased GPU usage and power consumption. From Apple's Machine Learning Research blog:

"LLMs are increasingly being used to power production applications, and improving inference efficiency can both impact computational costs and reduce latency for users. With ReDrafter's novel approach to speculative decoding integrated into the NVIDIA TensorRT-LLM framework, developers can now benefit from faster token generation on NVIDIA GPUs for their production LLM applications."

Developers interested in implementing ReDrafter can find detailed information on both Apple's website and NVIDIA's developer blog.

Tag: Nvidia

Popular Stories

iOS 19 Mock WWDC25 Feature

iOS 19 Expected to Run on These iPhones

Monday March 31, 2025 5:28 pm PDT by
iOS 19 will not be available on the iPhone XR, iPhone XS, or the iPhone XS Max, according a private account on social media site X that has accurately provided information on device compatibility in the past. The iPhone XR, iPhone XS, and iPhone XS Max all have an A12 Bionic chip, so it looks like iOS 19 will discontinue support for that chip. All other iPhones that run iOS 18 are expected...
maxresdefault

Apple Releases iOS 18.4 With Priority Notifications, Ambient Music, New Emoji and More

Monday March 31, 2025 10:03 am PDT by
Apple today released iOS 18.4 and iPadOS 18.4, the fourth major updates to the iOS 18 and iPadOS 18 operating system updates that came out last year. iOS 18.4 and iPadOS 18.4 come two months after Apple released iOS 18.3 and iPadOS 18.3. Subscribe to the MacRumors YouTube channel for more videos. The new software can be downloaded on eligible iPhones and iPads over-the-air by going to...
watchOS 11 Thumb 2 1

Apple Releases watchOS 11.4 With Sleep Alarm Update

Tuesday April 1, 2025 10:34 am PDT by
Apple today released watchOS 11.4, the fourth major update to the operating system that runs on the Apple Watch. watchOS 11.4 is compatible with the Apple Watch Series 6 and later, all Apple Watch Ultra models, and the Apple Watch SE 2. watchOS 11.4 can be downloaded on a connected iPhone by opening up the Apple Watch app and going to General > Software Update. To install the new software,...
AirPods Pro Firmware Feature

Apple Releases New Firmware for AirPods Pro 2 and AirPods 4

Monday March 31, 2025 11:27 am PDT by
Apple today released new firmware updates for all AirPods 4 and AirPods Pro 2 models. The new firmware is version 7E93, up from the 7B21 firmware that was installed on the AirPods Pro 2 and the 7B20 firmware available on the AirPods 4 and AirPods 4 with ANC. It is not immediately clear what new features or changes are included in the new firmware, but we'll update this article should we find ...
macOS Sequoia Feature

Apple Releases macOS Sequoia 15.4 With Mail Categorization and More

Monday March 31, 2025 10:04 am PDT by
Apple today released macOS Sequoia 15.4, the fourth major update to the macOS Sequoia operating system that launched in September. macOS Sequoia 15.4 comes two months after the launch of macOS Sequoia 15.3. Mac users can download the ‌‌macOS Sequoia‌‌ update through the Software Update section of System Settings. It is available for free on all Macs able to run macOS 15. With...
iPhone 17 Pro 34ths Perspective

iPhone 17 Pro Launching Later This Year With These 10 New Features

Sunday March 23, 2025 10:00 am PDT by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of March 2025: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
Apple Card iPhone 16 Pro Feature

Visa and American Express Vying to Win Apple Card Deal in 'Fierce' Fight

Tuesday April 1, 2025 1:50 pm PDT by
Visa wants to pay Apple approximately $100 million to be the new payment network for the Apple Card, reports The Wall Street Journal. As of right now, the Apple Card is on the Mastercard payment network, but that is set to change because Apple is ending its partnership with Goldman Sachs. Both American Express and Visa are vying to replace Mastercard as Apple's card services provider, while...

Top Rated Comments

attohs Avatar
15 weeks ago
NVidia? Did hell freeze over again?
Score: 37 Votes (Like | Disagree)
vegetassj4 Avatar
15 weeks ago
NVIDIA and Apple??!!? Working together again?



Attachment Image
Score: 13 Votes (Like | Disagree)
Delgibbons Avatar
15 weeks ago
Can't wait to put a 5090 in my Ma....

oh.
Score: 12 Votes (Like | Disagree)
redbeard331 Avatar
15 weeks ago
Good we have to hurry this up.



Attachment Image
Score: 9 Votes (Like | Disagree)
lilkwarrior Avatar
15 weeks ago
What would be an even better collaboration would be Apple enabling Nvidia GPU options again—at least for the Mac Pro.

It would be AWESOME to be able to use Nvidia’s ray-tracing and tensor cores with my creative professional and AI problems with Titan-class/Prosumer/workstation GPUs (x90 and up) again without having to switch to my PC.

A Nvidia MPX GPU module as capable as a 5090 with no wires and Thunderbolt 5 support would be a nirvana-like outcome—especially if Microsoft, Apple, and/or Valve enables a way to dual boot to Windows on ARM and SteamOS.

While I love building a liquid-cooled PC, I and various prosumers would finally have a choice to stop buying PCs altogether
Score: 7 Votes (Like | Disagree)
Unregistered 4U Avatar
15 weeks ago

Since Apple now produces its own GPUs there is no need for hell to freeze over. Do you even remember the reason Apple and Nvidia parted ways? It was over Nvidia wanting complete access to macOS’s core. Apple said no way.
And, we’ve since had a REALLY good example (CrowdStrike) of why this would have been a baaaad idea.
Score: 4 Votes (Like | Disagree)