Training a large AI model consumes roughly as much electricity as a small town uses in a year. That's not a metaphor or an exaggeration – it's a number that researchers have actually calculated, and it's climbing. The question of why that is, and whether it has to be, is one of the more important conversations happening in tech right now.
It's easy to picture AI training as something happening on a powerful laptop, or maybe a server rack. The reality is orders of magnitude larger – and understanding why reveals something genuinely interesting about how these systems learn, and what the industry's current trajectory actually means.
Before getting into the energy question, it helps to have a clear picture of what "training" means in this context, because it's not what most people imagine.
Training a large language model isn't like installing software or programming a set of rules. The model starts essentially empty – a massive mathematical structure with hundreds of billions of adjustable parameters, all set to random values. Training is the process of feeding it enormous quantities of text and, through countless iterations, nudging those parameters until the model gets better at predicting what word should come next in a sequence. Do that enough times, at enough scale, and something that looks a lot like understanding of language, reasoning, and knowledge emerges from what is, at its core, a very sophisticated pattern-matching system.
The key phrase is "enough times, at enough scale." Training a frontier model involves running forward passes and backward passes through a neural network with hundreds of billions of parameters, across datasets containing trillions of tokens, for weeks or months at a time. Each one of those passes involves an enormous number of floating-point math operations. Multiply the operations per pass by the number of passes required, and you arrive at the numbers that make energy researchers raise their eyebrows.
The specific hardware used for AI training is a significant part of the energy story. Graphics Processing Units – GPUs – are the workhorses of model training, though increasingly custom chips like Google's TPUs (Tensor Processing Units) and purpose-built AI accelerators are taking on more of the load.
GPUs were originally designed for rendering video game graphics, a task that requires doing massive numbers of parallel mathematical operations simultaneously. That same characteristic – parallelism at enormous scale – turns out to be exactly what neural network training needs. Modern AI training clusters contain tens of thousands of these chips running simultaneously, coordinated so that different parts of the network's computations can proceed at the same time.
The problem is that doing all this computation generates heat, and heat requires cooling. Data centers running large AI training jobs don't just consume electricity for the chips themselves – they consume additional electricity to keep those chips from overheating. Power Usage Effectiveness (PUE) is the metric that captures this: a PUE of 1.5 means that for every watt going to computation, an additional 0.5 watts goes to cooling and other infrastructure. Well-run modern data centers aim for PUE around 1.2–1.3, but older facilities can be significantly higher.
The reason energy consumption has been climbing so rapidly isn't just that more models are being trained – it's that each successive generation of frontier models has been substantially larger than the last, and scaling laws in deep learning suggest that larger models trained on more data tend to perform better.
The relationship between scale and capability was formalized in a set of research papers – most notably the "Chinchilla" paper from DeepMind in 2022 – that mapped out roughly how much data and compute you need to train a model of a given size optimally. The finding was that most large models had been undertrained: you could get better performance by training a smaller model on more data, rather than simply making the model bigger. This was an important efficiency insight, but it didn't change the fundamental dynamic that improving capabilities requires more total compute.
Training GPT-3, released by OpenAI in 2020, was estimated to have consumed around 1,300 megawatt-hours of electricity. Estimates for more recent frontier models, while not officially disclosed, suggest training runs an order of magnitude more intensive. When you account for the fact that labs train many experimental runs before a final model, the real cumulative cost is higher still. A single frontier model represents not just one training run but hundreds of exploratory experiments that collectively consumed significant resources.
The energy consumption of AI training matters for several interconnected reasons that go beyond the obvious environmental concern.
The carbon footprint question depends heavily on where the data centers are located and what's powering them. Training a model in a region where the electricity grid runs primarily on renewables has a very different emissions profile than training in a region heavily dependent on coal or natural gas. Some major labs have made commitments to match their energy consumption with renewable energy certificates, but that's different from actually running on clean power – the certificates represent investment in renewable capacity, not a guarantee that the electrons flowing into the data center are green ones. The distinction matters.
There's also a concentration-of-power dynamic that's worth noting. Training frontier models at this scale requires infrastructure that only a handful of organizations in the world can afford and access. The capital required for a single training run at the frontier – covering hardware, electricity, engineering talent, and data – runs into hundreds of millions of dollars. That cost creates a structural barrier to entry that concentrates the most capable AI systems in a small number of well-resourced companies, largely in the US and China. Whether that concentration is a good thing for the technology's development and governance is a live debate.
And the energy demand isn't going away. Inference – actually running models to answer user queries – consumes energy too, and as AI capabilities get deployed into more products and services, inference energy at scale becomes its own significant factor. Some estimates suggest that inference will eventually dwarf training energy consumption simply due to volume, as hundreds of millions of users send requests every day.
The energy intensity of AI training has attracted serious research attention, and there are genuine efficiency gains being made – though whether they outpace the growth in overall demand is the harder question.
Algorithmic efficiency improvements have been significant. Techniques like mixed precision training (using 16-bit floating point numbers instead of 32-bit where possible), gradient checkpointing, and better optimization algorithms reduce the computation required to train a model to a given capability level. The Chinchilla scaling laws themselves represented an efficiency insight – training smaller models for longer on more data rather than just making models bigger.
Hardware is also improving. Each generation of purpose-built AI chips delivers more computation per watt than the last. NVIDIA's H100 and H200 GPUs, and Google's latest TPU generations, are substantially more efficient than the hardware from just a few years ago. Custom silicon – chips designed specifically for AI workloads rather than adapted from other purposes – tends to be more energy-efficient per operation than general-purpose hardware.
There's also growing interest in alternative training approaches: sparse models that activate only a fraction of their parameters for any given input (Mixture of Experts architectures), smaller "distilled" models that learn from larger ones, and techniques that allow models to be fine-tuned on specific tasks with a fraction of the compute required to train from scratch. These approaches don't eliminate the fundamental energy challenge but they do meaningfully change the economics for specific use cases.
The efficiency improvements are real, but so is the demand growth. Jevons' Paradox – the economic observation that efficiency gains in using a resource often lead to increased total consumption of that resource because the lower cost enables more use – seems to be operating here. As training becomes more efficient, labs use that efficiency to train bigger or more numerous models rather than to reduce total energy consumption.
Whether that trajectory is sustainable, and what role AI's energy footprint should play in broader conversations about climate commitments and data center regulation, is a question that's moving from academic circles into policy discussions. The EU's AI Act includes provisions around transparency of resource usage. Several US states have introduced or are considering data center energy disclosure requirements. These conversations are early but they're happening.
What's clear is that the energy question isn't a peripheral issue in AI development – it's a structural one. The cost of compute shapes which organizations can train frontier models, which geographies host the infrastructure, how the economics of the industry work, and what kind of environmental tradeoffs society is making in exchange for increasingly capable AI systems. Understanding the basics of why training is so energy-intensive is a prerequisite for thinking clearly about those tradeoffs.
How much energy does training a large AI model actually use? Published estimates vary, and most labs don't disclose exact figures. GPT-3 was estimated at around 1,300 MWh for a single training run. More recent frontier models are believed to require significantly more – some estimates suggest training runs in the tens of thousands of MWh range, though these numbers are difficult to verify without official disclosure.
Is running AI (inference) as energy-intensive as training it? A single inference query uses far less energy than a training run. But when you multiply even modest per-query energy use by hundreds of millions of queries per day, inference at deployment scale adds up. Whether inference or training dominates total AI energy consumption over time is an active area of research and debate.
Do AI companies use renewable energy? Many major labs and cloud providers have made renewable energy commitments, but the reality is complex. "Matching" consumption with renewable energy certificates isn't the same as running on clean power in real time. The actual emissions depend on when and where computation happens relative to when renewable generation is occurring on the grid.
Could AI training become dramatically more efficient? Algorithmic and hardware improvements are genuine and ongoing. But the history of computing suggests that efficiency gains tend to enable more use rather than less total consumption. Absent regulatory pressure or a fundamental shift in how the industry approaches scale, total energy demand is likely to continue growing even as efficiency improves.
Why can't AI be trained on regular computers? The number of mathematical operations involved is simply too large. A consumer GPU might perform a few teraflops (trillions of floating-point operations per second). A training cluster for a frontier model might aggregate tens of thousands of enterprise GPUs running for weeks. The scale difference is roughly analogous to the difference between a hand calculator and a national supercomputing facility.
Strubell et al. – Energy and Policy Considerations for Deep Learning in NLP (2019): https://arxiv.org/abs/1906.02629
Hoffmann et al. (DeepMind) – Training Compute-Optimal Large Language Models (Chinchilla, 2022): https://arxiv.org/abs/2203.15556
IEA – Electricity 2024: Analysis and Forecast to 2026 (data center section): https://www.iea.org/reports/electricity-2024
Patterson et al. (Google) – Carbon Footprint of Machine Learning (2021): https://arxiv.org/abs/2104.10350
MIT Technology Review – The carbon footprint of GPT-4 is larger than you think: https://www.technologyreview.com/2023/12/01/1084189/making-an-image-with-generative-ai-uses-as-much-energy-as-charging-your-phone
NVIDIA – H100 Tensor Core GPU architecture overview: https://www.nvidia.com/en-us/data-center/h100
