Nvidia’s Grace Hopper Runs at 700 W, Blackwell Will Be 1 KW. How Is the Power Supply Industry Enabling Data Centers to Run These Advanced AI Processors

Blackwell configurations will require 60 kW to 120 kW per rack but fewer than 5% of the world's data centers are capable of supporting even 50kW per rack

Generative AI's proliferation is far from complete, but its rise has been (without trying to hyperbolise) little short of meteoric.

This obviously needs context, so let us look at one of the industry's leading players (ChatGPT): Facebook took 10 months to reach a million registered users, Spotify took five, Instagram took 2.5 … and ChatGPT took 5 days. This site alone is the internet's (at time of writing) 18^th most visited site globally and has 2.4 billion visits per month (source SimilarWeb).

Fig 1: Growth of ChatGPT, and how this increase in adoption of AI affects power demand

Data sources: SimilarWeb, Goldman Sachs

As AI proliferates, the extreme power demanded by processors is increasing rapidly too. AI data centers are migrating to ever-more-advanced processors, like Nvidia's 'Grace Hopper' DGX H100, which demands 700 W per processor.

A cabinet running Nvidia's Grace Hopper H100 super-chips therefore requires c.100 kW or more to be delivered by the power supply unit (PSU), which is a two- or three-fold increase on most current servers that typically draw 30 to 40 kW per cabinet.

This is set to increase next year with Nvidia's introduction of the 'Blackwell' DGX B100 & B200 chips, which will run at 1 kW per processor.

And while it's true that these chips are becoming exceptionally more power efficient per operation (see fig 2), the sheer scale up in AI use means these the power required is scaling up significantly.

Fig 2: AI servers are becoming significantly more efficient, but the rise of AI usage requires greater power densities to be delivered

Note - figures quoted are per server, not per processor.

Data source: Goldman Sachs

The combination of these more power-hungry processors and the increased demand means significantly increased power densities need to be delivered by the data center power supplies that are limited to the standard CRPS and OCP form factors.

The need for this is particularly acute given a Blackwell architecture analysis by the AI infrastructure firm AMAX has highlighted a significant "readiness gap" between data centers.

As its summary states, Blackwell configurations present a major challenge for data centers and require multiple changes in their architecture if they are to be powered and cooled. In relation to supplies, it notes Blackwell configurations will require 60 kW to 120 kW per rack, but states that fewer than 5 percent of the world's data centers are capable of supporting even 50kW per rack​​.

This increased capacity can only be delivered through significantly improved power densities therefore.

Achieving increased power densities

Due to the physical properties of silicon, traditional power supplies have reached their performance limits in terms of efficiency and power density. Materials that possess a wider bandgap are needed to overcome these limitations.

Of note are silicon carbide (SiC) and gallium nitride (GaN), which have bandgaps of 2.26 eV and 3.39 eV respectively (2X and 3X that of silicon), and allow far higher switching frequencies, efficiencies and power densities in AI power supplies. The thermal properties of these wide bandgap supplies also help AI data centers to reduce some of the cost of cooling.

Navitas published its roadmap for AI PSUs based on GaN and SiC technologies, earlier this year, with its supplies setting all-new industry standards for power density and efficiency.

Under this roadmap, Navitas will release a 8.5 kW PSU for AI data centers in Q4 2024, with a > 10 kW solution following. This is a 3X increase in power densities in less than 18 months - versus its 3.2 kW data center power platform (launched August 2023). This will not only represent a huge step up in power delivery but, with a 98% power conversion efficiency, is also ahead of the 80 PLUS Titanium requirements.

Our team is well on track to meet this ambitious target, as evidenced by Navitas's July 2024 launch of 4.5 kW CRPS185 form-factor PSU. This has both a 97% power conversion efficiency and a power density of 137 w/in³ - the highest of any data center PSU currently available.

For further information on these power supplies visit here.

Public link

Please use the above public link if you want to share this noodl on another website.