Putting AI quality first to enhance accessibility and collaboration

Technology can be a great equalizer and artificial intelligence is no exception. It can help us accomplish things that we weren't able to do before. As we continue to build out features and capabilities for Zoom AI Companion, we're extraordinarily mindful of how our products can help empower our customers and are committed to delivering the highest quality results so that they can be efficient and successful in their work.

Our customers use AI Companion across industries, including financial services, healthcare, education, government, and more. Today, in education and universities, students and teachers already use Zoom's AI capabilities to improve accessibility for lectures. In healthcare, people attend virtual appointments and use transcript and summarization capabilities to expand access to care.

Using AI to make technology more accessible isn't just reserved for closed captioning for the hearing impaired, but also to break down language barriers. When I was a student 35 years ago in Edinburgh, Scotland, and still learning English, not only did I benefit from closed captioning when watching BBC, but it also showed me what accurate captioning and translation can do for people around the world. The use of AI is already quite expansive, and we want to continue to improve the quality to open the possibilities for more people.

Our success in meeting transcriptions is one piece of this journey, and it touches on many aspects of how AI can augment a person's own skills and experience to help them go even further. That could be automated live captions in a meeting or webinar, translating live captions into another language, or using AI Companion for transcribing meetings for use with meeting summaries, smart recordings, action items, and more. Creating a more accessible and equitable meeting experience in a global business environment benefits everyone.

Accurate speech recognition is the foundation of Zoom AI Companion

Earlier this year, I shared how our federated approachto AI was matching or surpassing that of other AI models at far less expense. Today, I am excited to share findings from a recent evaluation of Zoom AI performance commissioned from TestDevLab, which further solidifies Zoom AI Companion 2.0as a leader in speech recognition and meeting intelligence in comparison to other tested AI tools.

Zoom AI Companion 2.0 seamlessly integrates work and web information to deliver an impressive upgrade in the Zoom Workplace experience. While there are many ways AI features are integrated into Zoom Workplace, there's one aspect of AI Companion that powers our most broadly used AI features, including meeting summaries, action items, and transcripts.

The effectiveness of these AI features hinges on the accuracy of the transcriptions they draw from. Many of our popular features, such as asking in-meeting questions, rely on high-quality speech recognition. A reliable transcript allows AI to capture names, topics, and intentions accurately-forming the backbone of summaries, highlights, and actionable insights.

The industry standard for measuring transcription accuracy is Word Error Rate (WER), which compares the percentage of differences between a human-generated transcript and an AI-generated one. Zoom has been working to modernize our AI architecture in the past years to minimize WER. By minimizing WER, we create precise, reliable transcripts to help drive effective, actionable outcomes for our customers.

In a WER analysis, the red text represents missed words, and the blue text represents words not found in the source transcript.

Zoom delivers more accurate transcripts to power AI features

The results were clear: Zoom AI Companion outperformed Microsoft Teams with significantly lower WER, delivering a higher level of transcription accuracy. In meetings where every word counts, even minor transcription errors can have a large impact on summaries, tasks, or answers to questions posed during meetings.

Here's how TestDevLab created and ran these tests:

• They used three recorded meetings, ranging from two to 16 participants.
• Files were played back synchronously on individual computers connected to the call.
• For consistency, each test was repeated five times for each platform.

For each test, they compared Zoom Workplace and Microsoft Teams and measured the Word Error Rate for each meeting.

When measuring WER (Word Error Rate), a low percentage signifies a higher quality transcription.

Meeting summaries and transcriptions are some of the most popular and most-used features across many AI platforms. Because they're so widely used, it's even more critical to make sure that errors are reduced as much as possible. They're popular because they save time and people find so much value in reading, summarizing, and better understanding important information.

TestDevLab also ranked Zoom Workplace and Microsoft Teams in meeting summary quality and conversational AI (such as answer stability). To assess the summaries, they created an LLM assistant with human-validated results.

Solid transcripts create a better foundation for downstream AI features

Transcription isn't the only AI Companion feature that TestDevLab measured. Unlike transcripts, the goals for other AI features like meeting summaries aren't to give a word-for-word copy, but to instead provide a tailored version of what happened that meets the users' expectations.

TestDevLab also ranked Zoom Workplace and Microsoft Teams in meeting summary quality and conversational AI (such as answer stability). To assess the summaries, they created an LLM assistant with human-validated results.

In this evaluation, we tested two summary capabilities within Microsoft Teams: the Intelligent Recap feature, available with a Teams Premium or Microsoft 365 Copilot license, and the prompt-based summary generation capability of Microsoft Copilot AI Assistant in Teams Meetings. Since Copilot AI Assistant can only generate summaries based on prompts, we prompted Copilot to create a summary and action items at the conclusion of the meeting.

In terms of conversational AI, TestDevLab measured how the AI platforms managed to answer questions while in a meeting, both for questions related to the meeting context and for unrelated web searches, like "What is the tallest building in the world?" In both response time and stability (the ability of all meeting participants to receive a similar answer), Zoom AI Companion outperformed Microsoft.

Higher stability indicates that Zoom's AI consistently delivers reliable responses to all participants, providing a smoother experience during meetings.

Zoom AI Companion provides a better, more equitable meeting experience

These results highlight our commitment to quality across Zoom Workplace and AI Companion. We strive to create a way for people to more easily connect with each other, be more efficient in their day, and have the opportunity to use artificial intelligence to support their work. AI-generated meeting transcriptions are just one way that people can use AI Companion to help create a more equitable experience for many people, including the Deaf, hard-of-hearing, neurodiverse, and those who may speak a different language than others in the meeting.

This evaluation conducted by TestDevLab highlights Zoom Workplace and AI Companion as a leader in AI performance, with superior transcription accuracy, faster in-meeting question response times, and more stable conversational AI capabilities compared to tested competitors.

What's more, Zoom Workplace offers AI Companion at no additional cost to paid Zoom accounts, unlike some alternative platforms, which charge extra for each additional user granted AI features. At Zoom, we think it's important to make these accurate and transformational tools available to as many customers as possible. We start at the foundation-creating high-quality transcripts-and continue to create exceptional AI experiences for all of our customers.

• To get a better understanding of everything TestDevLab measured, read the report.
• Learn more about how AI Companion can fit into your organization.

If you don't have an eligible paid Zoom plan, upgrade todayto access the benefits of AI Companion.