Apple Teams Up With NVIDIA to Speed Up AI Language Models
Apple has shared details on a collaboration with NVIDIA to greatly improve the performance of large language models (LLMs) by implementing a new text generation technique that offers substantial speed improvements for AI applications.

Apple earlier this year published and open-sourced Recurrent Drafter (ReDrafter), an approach that combines beam search and dynamic tree attention methods to accelerate text generation. Beam search explores multiple potential text sequences at once for better results, while tree attention organizes and removes redundant overlaps among these sequences to improve efficiency.
Apple has now integrated the technology into NVIDIA's TensorRT-LLM framework, which optimizes LLMs running on NVIDIA GPUs, where it achieved "state of the art performance," according to Apple. The integration saw the technique manage a 2.7x speed increase in tokens generated per second during testing with a production model containing tens of billions of parameters.
Apple says the improved performance not only reduces user-perceived latency but also leads to decreased GPU usage and power consumption. From Apple's Machine Learning Research blog:
"LLMs are increasingly being used to power production applications, and improving inference efficiency can both impact computational costs and reduce latency for users. With ReDrafter's novel approach to speculative decoding integrated into the NVIDIA TensorRT-LLM framework, developers can now benefit from faster token generation on NVIDIA GPUs for their production LLM applications."
Developers interested in implementing ReDrafter can find detailed information on both Apple's website and NVIDIA's developer blog.
Popular Stories
iOS 19 will not be available on the iPhone XR, iPhone XS, or the iPhone XS Max, according a private account on social media site X that has accurately provided information on device compatibility in the past.
The iPhone XR, iPhone XS, and iPhone XS Max all have an A12 Bionic chip, so it looks like iOS 19 will discontinue support for that chip. All other iPhones that run iOS 18 are expected...
Apple today released iOS 18.4 and iPadOS 18.4, the fourth major updates to the iOS 18 and iPadOS 18 operating system updates that came out last year. iOS 18.4 and iPadOS 18.4 come two months after Apple released iOS 18.3 and iPadOS 18.3.
Subscribe to the MacRumors YouTube channel for more videos.
The new software can be downloaded on eligible iPhones and iPads over-the-air by going to...
Apple today released watchOS 11.4, the fourth major update to the operating system that runs on the Apple Watch. watchOS 11.4 is compatible with the Apple Watch Series 6 and later, all Apple Watch Ultra models, and the Apple Watch SE 2.
watchOS 11.4 can be downloaded on a connected iPhone by opening up the Apple Watch app and going to General > Software Update. To install the new software,...
Apple today released new firmware updates for all AirPods 4 and AirPods Pro 2 models. The new firmware is version 7E93, up from the 7B21 firmware that was installed on the AirPods Pro 2 and the 7B20 firmware available on the AirPods 4 and AirPods 4 with ANC.
It is not immediately clear what new features or changes are included in the new firmware, but we'll update this article should we find ...
Apple today released macOS Sequoia 15.4, the fourth major update to the macOS Sequoia operating system that launched in September. macOS Sequoia 15.4 comes two months after the launch of macOS Sequoia 15.3.
Mac users can download the macOS Sequoia update through the Software Update section of System Settings. It is available for free on all Macs able to run macOS 15.
With...
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices.
Below, we recap key changes rumored for the iPhone 17 Pro models as of March 2025:
Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
Visa wants to pay Apple approximately $100 million to be the new payment network for the Apple Card, reports The Wall Street Journal. As of right now, the Apple Card is on the Mastercard payment network, but that is set to change because Apple is ending its partnership with Goldman Sachs.
Both American Express and Visa are vying to replace Mastercard as Apple's card services provider, while...