10/24/2024 | News release | Archived content
Our annual Snapdragon Summit wrapped up with exciting new announcements centered on the future of on-device artificial intelligence (AI). With decades of experience in research and product development, Qualcomm Technologies continues to set the standard for innovative system solutions.
Over the course of our three-day event, we welcomed technology media, analysts and industry-leading partners from around the globe to showcase our latest in mobile, automotive and Qualcomm AI Hub advancements. But we wanted to make this experience as accessible as possible, so here's a glimpse of what we are bringing to the table in the ever-changing world of on-device AI.
With an exciting kick-off to Snapdragon Summit, Alex Katouzian, group general manager of mobile, compute and XR for Qualcomm Technologies, introduced our new Snapdragon 8 Elite Mobile Platform. Designed to deliver world-class performance and experiences, the newly architected Qualcomm Hexagon NPU (neural processing unit) brings significant advancements in on-device AI and support for multi-modality. It has 45% faster AI performance and 45% better power efficiency compared to the Snapdragon 8 Gen 3.
For the first time, Katouzian demonstrated an on-device multi-modal AI assistant that doesn't just listen to you but also sees what you see, creating a more intuitive and immersive experience. This sample application was showcased with a live demonstration in which individuals can interact with objects on the camera live preview. The demo also showed someone pointing their camera at a receipt and asking the AI assistant to calculate a tip while splitting the bill between everyone at the table, saving time and hassle.
Continuing the momentum of the Snapdragon 8 Elite announcement was Siddhika Nevrekar, senior director of product management for Qualcomm Technologies, who took a deep dive into how the mobile platform works under the hood and maintains its leadership in on-device multi-modal generative AI. With our new and improved Qualcomm AI Engine, we're supporting the availability of advanced applications and multimodal AI assistants directly into the palm of your hands.
By introducing the Qualcomm Oyron CPU into our AI Engine, its amazing multi-tasking capabilities help the other compute cores focus on their respective AI tasks. The Qualcomm Oyron CPU is great at processing first-inference latency-critical tasks like your productivity apps, but also plays a role in initializing and distributing AI workloads to other compute cores, which is extremely important for on-device AI inferencing.
We also made serious improvements to the Hexagon NPU, the heart of the Qualcomm AI Engine:
With all the AI performance uplift, we're seeing up to 100% faster token rate performance on foundational LLMs and getting up to 70 tokens per second on specific LLMs.
The AI assistant example that Katouzian demonstrated is powered by a mix of advanced AI models:
As they all run on different cores of our AI Engine, this allows the assistant to understand your voice, interpret visual inputs and respond instantly. The Snapdragon 8 Elite supports a longer input window which is measured by how many tokens you can input.
Our improvements wouldn't be complete without advancements to the Qualcomm Sensing Hub. Being the gateway to support a personalized AI assistant, it's now 60% faster in AI performance with 34% more memory for better performance and efficiency.
Nakul Duggal, group general manager, automotive, industrial and cloud for Qualcomm Technologies, started off strong on Day 2 by announcing the Snapdragon Cockpit Elite for automotive platforms. Designed for power efficiency and performance, this platform is dedicated to transformer acceleration and E2E network architectures that support large foundational models.
Our latest NPU integrated in the Snapdragon Cockpit Elite system on a chip (SoC) is a dedicated AI engine designed to reach up to 12 times improvement over our previous flagship cockpit SoC. This powerful multimodal AI engine is designed to support applications built on large language models with billions of parameters like Llama, Gemini, Phi3, Bloom and more.
Utilizing one of these foundational models, use cases such as vehicle preventative maintenance can be done with retrieval augmented generation - where the model is trained on the vehicle manual, and an AI assistant can retrieve the answer regarding an unknown icon on the screen.
Lastly, Durga Malladi, senior vice president and general manager of technology planning and edge solutions for Qualcomm Technologies, welcomed our Qualcomm AI Hub advanced models from our collaborators and new community collaborators. Qualcomm AI Hub is excited to share many new models, specifically LLMs from our newly announced model maker collaborations. These collaborations also open the door for many new models coming soon to Qualcomm AI Hub. We encourage you to download and contact us for these models.
Another thrilling announcement is that we're making the Snapdragon 8 Elite available for developers on AI Hub via Qualcomm Device Cloud. This means developers can dive right in and start creating their incredible applications today
There's more to come in the world of on-device AI. Check back with the OnQ blog for the latest developments in this rapidly evolving area.