Apple Teams Up With NVIDIA to Speed Up AI Language Models
Apple has shared details on a collaboration with NVIDIA to greatly improve the performance of large language models (LLMs) by implementing a new text generation technique that offers substantial speed improvements for AI applications.

Apple earlier this year published and open-sourced Recurrent Drafter (ReDrafter), an approach that combines beam search and dynamic tree attention methods to accelerate text generation. Beam search explores multiple potential text sequences at once for better results, while tree attention organizes and removes redundant overlaps among these sequences to improve efficiency.
Apple has now integrated the technology into NVIDIA's TensorRT-LLM framework, which optimizes LLMs running on NVIDIA GPUs, where it achieved "state of the art performance," according to Apple. The integration saw the technique manage a 2.7x speed increase in tokens generated per second during testing with a production model containing tens of billions of parameters.
Apple says the improved performance not only reduces user-perceived latency but also leads to decreased GPU usage and power consumption. From Apple's Machine Learning Research blog:
"LLMs are increasingly being used to power production applications, and improving inference efficiency can both impact computational costs and reduce latency for users. With ReDrafter's novel approach to speculative decoding integrated into the NVIDIA TensorRT-LLM framework, developers can now benefit from faster token generation on NVIDIA GPUs for their production LLM applications."
Developers interested in implementing ReDrafter can find detailed information on both Apple's website and NVIDIA's developer blog.
Popular Stories
If you missed the video showing dummy models of Apple's all-new super thin iPhone 17 Air that's expected later this year, Sonny Dickson this morning shared some further images of the device in close alignment with the other dummy models in the iPhone 17 lineup, indicating just how thin it is likely to be in comparison.
The iPhone 17 Air is expected to be around 5.5mm thick – with a thicker ...
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices.
Below, we recap key changes rumored for the iPhone 17 Pro models as of April 2025:
Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
Apple plans to release an all-new super thin iPhone this year, debuting it alongside the iPhone 17, iPhone 17 Pro, and iPhone 17 Pro Max. We've seen pictures of dummy models, cases, and renders with the design, but Lewis Hilsenteger of Unbox Therapy today showed off newer dummy models that give us a better idea of just how thin the "iPhone 17 Air" will be.
The iPhone 17 Air is expected to be ...
Despite being more than two years old, Apple's AirPods Pro 2 still dominate the premium wireless‑earbud space, thanks to a potent mix of top‑tier audio, class‑leading noise cancellation, and Apple's habit of delivering major new features through software updates. With AirPods Pro 3 widely expected to arrive in 2025, prospective buyers now face a familiar dilemma: snap up the proven...
While the so-called "iPhone 17 Air" is not expected to launch until September, there are already plenty of rumors about the ultra-thin device.
Overall, the iPhone 17 Air sounds like a mixed bag. While the device is expected to have an impressively thin and light design, rumors indicate it will have some compromises compared to iPhone 17 Pro models, including only a single rear camera, a...
A developer has demonstrated Windows 11 ARM running on an M2 iPad Air using emulation, which has become much easier since the EU's Digital Markets Act (DMA) regulations came into effect.
As spotted by Windows Latest, NTDev shared an instance of the emulation on social media and posted a video on YouTube (embedded below) demonstrating it in action. The achievement relies on new EU regulatory...
Starting today, April 24, Apple Stores around the world are giving away a special pin for free to customers who request one, while supplies last.
Photo Credit: Filip Chudzinski
The enamel pin's design is inspired by the Global Close Your Rings Day award in the Activity app, which Apple Watch users can receive by closing all three Activity rings today. The limited-edition pin is the physical...
Apple's $570 million fine from the EU has triggered a sharp rebuke from the White House, which called the fine a form of economic extortion, Reuters reports.
The fine was announced on Wednesday by the European Commission, following a formal investigation into Apple's compliance with the bloc's Digital Markets Act (DMA), a landmark piece of legislation aimed at curbing the market dominance of ...