Select text to annotate, Click play in YouTube to begin
- I'm gonna see the GPU guys. This is it. Welcome to an AI data center where you can hear. This is extremely loud right now. And feel. This can blow dry my hair. The power. The AI's in there. By 2028, data centers like this across the country could use 12% of all US electricity, enough to power more than 55 million homes for a year.
And yes, you are increasing that energy. Every time you hit enter on an AI prompt, it's routed through places like this. - You know, we need to do the AI. And we need more than double the energy, the electricity that we currently have. - But I wanted to know how much power I use when I do the AI. Is generating an image or a video the equivalent of making a steak on an electric grill?
Oh yeah. Okay, let's start with tracing this prompt, "Create a video of a dancing steak." The moment you hit enter, that request gets routed to a place like this massive building I visited in Ashburn, Virginia, AKA Data Center Alley. This building and the many around it are owned by Equinix, one of the largest companies powering the internet and now AI. When your prompt arrives at the data center,
it kicks off inference, the process where the model thinks and responds to your request. Rows of powerful GPUs generate your request. In this case, the dancing steak. This process is not the same as AI training. That happens earlier when the model learns from massive amounts of data. Back to my question, how much energy am I using every time I make that single image or video?
Google, Microsoft, and other generative AI companies don't share that information. We asked them. But researchers have done their own calculations. - The goal is to use the standardized method methodology. So same dataset, same hardware, kind of same setup, and compare different models in terms of energy usage. - [Joanna] Sasha Luccioni and other researchers test how much energy is used when generating text, images, and video
using Nvidia H100 GPUs. She found the energy required to generate content varies widely depending on the AI model and the GPU setup. That brings us back to this electric grill. So I plugged in this handy power meter to the grill, cranked up the heat and threw a pretty thin steak on. Oh yeah. If you're generating text that could burn anywhere from 0.17 watt hours to 2 watt hours,
equal to running this grill for about four seconds. Generating an image add 1.7 watt hours. All that, less than 10 seconds on the grill. But short videos can use far more power. In tests of various open source models, videos took anywhere between 20 watt hours and 110 watt hours. At 110 watt hours, one steamed electric grill steak,
about equal to one video generation. I wouldn't eat it, but my dog would. At 220 watt hours, it was looking much more edible. So two video generations equals one pretty good looking steak. That might not sound like much power until you consider the short film we made using Google VO and Runway. Luccioni's numbers come from open source models that generate six-second 480p clips,
far lower quality than tools like Google VO. We generated 1000 8-second 720p clips for our film. Going by these estimates, we might have used roughly 110,000 watt hours. That's enough to grill around 478 steaks, or power an average US home for 3.5 days. Then again, Google and others might have more efficient systems, so it could have taken less energy.
- Often I get asked, "Well, how does your research on open source compare to ChatGPT or Gemini or Sora?" Until we get access to these models, we can't actually give any information. And so all we can do is kind of do estimates. - [Joanna] Recently, OpenAI has shared something. In a blog post, CEO Sam Altman said that the average query uses about 0.34 watt hours of energy. OpenAI wouldn't provide details on image or video energy usage.
Based on Altman's numbers, 647 prompts equals one steak. But energy isn't the only concern. Water is used in big quantities to cool these hot GPUs. The Equinix data center we were in used a closed-loop system that recirculates water with minimal waste. But in other setups, water is cycled through and evaporated or dumped, leading to a net loss.
Okay, but why is this stuff so power intensive? The answer is the hottest product around, literally. They said it was gonna be too hot for TV, as in like they couldn't show it because it's sensitive, but they're gonna show us a GPU. (funky music) They look like computers, I don't know. Nvidia GPUs are some of the most powerful and expensive chips on the planet. This super pod held 31 Nvidia DGX H100s,
each packed with eight GPUs, totaling around $9 million in hardware alone. When people say the GPUs are screaming, it's for real. These chips generate so much heat, they'll shut down if they get too hot. So the fans work overtime to keep them cool. This is where the cold air comes in, and this is where the hot air goes out. And on top of that, the data center uses advanced air and liquid cooling
to manage the temperature. - Most equipment can run comfortably up to about 90, 92 degrees Fahrenheit. Beyond that, it'll power itself off or literally break. - [Joanna] And, obviously, all that uses a lot of energy. Sometimes a rack of GPUs like we saw can use 100 times as much as racks with other equipment like CPUs. But Nvidia says that the energy efficiency of its newer chips has improved
since the chips used in Luccioni's testing. - In the past year, year and a half or so, we've seen a 30 times improvement in energy efficiency. So we're using 1/30 of the energy for the same inference workload that we were just a year ago. - So what did we learn here today? Certainly that AI and GPUs draw huge amounts of power. And as AI demand grows, so will the size and number of these data centers.
We saw several new ones already under construction in Data Center Alley. But the big question, is it worth it? Is that one silly video worth the same energy it takes to make a meal or charge a laptop? - You know, if people saw how much energy was being used for each silly cat video that they generated, maybe they'd think twice, or, you know, they do it in a more conscious way. Or they'd make decisions,
"I'm gonna use this model and not that model." "I'm gonna use my phone for a calculator, not ChatGPT for a calculator." - [Joanna] But it's not all fun and viral videos. That big GPU cluster we saw, it belongs to Bristol Myers Squibb, and it's being used to discover new drugs. - So they're actually investigating which molecules might have application in treating disease in the future. So there's some workloads we really root for,
and those are some examples. - I'm not gonna eat any more of that. (chuckles)
End of transcript