Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
There are concerning reports on Reddit that Apple's latest iOS 17.5 update has introduced a bug that causes old photos that were deleted – in some cases years ago – to reappear in users' photo libraries. After updating their iPhone, one user said they were shocked to find old NSFW photos that they deleted in 2021 suddenly showing up in photos marked as recently uploaded to iCloud. Other...
Apple today previewed new accessibility features coming with iOS 18 later this year, and this includes some new options for CarPlay. Apple highlighted three new features coming to CarPlay: Voice Control: This feature will allow users to navigate CarPlay and control apps with just their voice. Color Filters: This feature will make the CarPlay interface visually easier to use for...
The iMessage service that Apple users to send messages to one another appears to be down for some users, and messages are failing to go out or are taking an extra long time to send. There are numerous reports about the issue on social networks and a spike of outage reports on Down Detector, but Apple's System Status page is not yet reporting an outage. Update: Apple's status page says...
Apple today released tvOS 17.5, the fifth update update to the tvOS 17 operating system that came out last September. tvOS 17.5 comes two months after the release of tvOS 17.4. tvOS 17.5 can be downloaded using the Settings app on the Apple TV. Go to System > Software Update to get the new software. Apple TV owners who have automatic software updates activated will be upgraded to ...
At its Spring Update event, OpenAI announced that it will be releasing a desktop app for the Mac, as seen in the screenshot below. The app will be rolling out to ChatGPT Plus subscribers starting today, ahead of a wider launch "in the coming weeks." "With a simple keyboard shortcut (Option + Space), you can instantly ask ChatGPT a question," OpenAI's press release says. In addition, Voice...
Today is the official launch day of the new iPad Pro models, and these updated tablets mark the biggest feature and design refresh that we've seen for the iPad Pro in several years. We picked up one of the new 13-inch models to check out everything new. Subscribe to the MacRumors YouTube channel for more videos. When it comes to design, Apple is still offering 11-inch and 13-inch size options ...
Top Rated Comments
I'm going to stand on the sidelines to see what group A and group B says about this.
Siri: Tell me what you'd like to be reminded about.
Me: This.
Siri: Okay. I've added a reminder called 'this' to your reminders.