Hello and welcome to Eye on AI.
The biggest show in AI this week is Nvidia’s GTC developer conference in San Jose, Calif. One Wall Street analyst quipped the chipmaker’s confab was the equivalent of “AI Woodstock,” given all the heavy-hitters present from not just Nvidia, but companies such as OpenAI, xAI, Meta, Google, and Microsoft, and the presence of executives from major companies looking to implement AI, including L’Oréal, Lowe’s, Shell, and Verizon.
At GTC yesterday, Nvidia CEO Jensen Huang unveiled the company’s newest graphics processing unit (GPU), the kind of chips that have become the workhorses of AI. The forthcoming Blackwell GPU will have 208 billion transistors, far exceeding the 80 billion its current top-of-the-line H100 GPUs have. The larger chips mean they will be twice as fast at training AI models and five times faster at inference—the term for generating an output from an already trained AI model. Nvidia is also offering a powerful new GB200 “superchip” that would include two Blackwell GPUs coupled together with its Grace CPU and supersede the current Grace Hopper MGX units that Nvidia sells for use in data centers.
What’s interesting about the Blackwell is its power profile—and how Nvidia is using it to market the chip. Until recently, the trend has been that more powerful chips also consumed more energy, and Nvidia didn’t spend much effort trying to make energy efficiency a selling point, focusing instead on raw performance. But in unveiling the Blackwell, Huang emphasized how the new GPU’s greater processing speed meant that the power consumption during training was far less than with the H100 and earlier A100 chips. He said training the latest ultra-large AI models using 2,000 Blackwell GPUs would use 4 megawatts of power over 90 days of training, compared to having to use 8,000 older GPUs for the same period of time, which would consume 15 megawatts of power. That’s the difference between the hourly power consumption of 30,000 homes and just 8,000 homes.
Nvidia is talking about the Blackwell’s power profile because people are growing increasingly alarmed about both the monetary cost of AI and its carbon footprint. Those two factors are related since one reason cloud providers charge so much to run GPUs is not just the cost of the chips themselves, but the cost of the energy to run them (and to cool the data centers where they are housed since the chips also throw off more heat than conventional CPUs). And both those factors have made many companies reluctant to fully embrace the generative AI revolution because they are worried about the expense and about doing damage to net zero sustainability pledges. Nvidia knows this—hence its sudden emphasis on power consumption. The company has also pointed out that many AI experts working on open-source models have found ways to mimic some aspects of the performance of much larger, energy-intensive models such as GPT-4 but with models that are much smaller and less power-consuming.
Currently, data centers consume just over 1% of the world’s power, with estimates that AI is a fraction of that. Schneider Electric recently estimated that AI consumes about as much power annually as the nation of Cyprus. That number may climb rapidly due to AI. One expert at Microsoft has suggested that just the Nvidia H100s in deployment will consume about as much power as all of Phoenix by the end of this year.
Still, I have always thought the focus on AI’s energy consumption in data centers was a bit of a red herring, since most of the data centers of the cloud hyperscalers, which is where most AI is being run right now, are now powered by renewable energy or low-carbon nuclear power. And the fact that these companies are willing to contract for large amounts of renewable power at set prices has played a key role in giving renewable power companies the confidence to build large wind and solar power projects. The presence of these hyperscalers in the renewables market has meant there is more renewable power available for everyone. It’s a win-win. (Far more troubling is the water consumption needed to keep these data centers cool. Here consuming less power, and generating less heat, would have a more direct impact on sustainability.)
That said, AI is a global phenomenon, and there are some places where there isn’t much renewable power available. And if AI is adopted to the extent many project, and if AI models keep getting larger, it is possible renewable energy demand could outstrip low carbon supplies even in the U.S. and Europe. That’s one reason Microsoft has expressed interest in trying to use AI to speed up the process of getting new nuclear plants approved for construction in the U.S.
It is also true that AI’s energy consumption is among the many areas where our own brains are vastly superior to the artificial ones we’ve created. The human brain consumes about 0.3 kilowatt hours daily by burning calories, compared to about 10 kilowatt hours daily for the average H100. To really make AI ubiquitous without destroying the planet in the process, we may need to find a way to get artificial neural networks to operate with an energy profile that looks a bit more like the natural ones.
That’s essentially what the U.K.’s Advance Research and Invention Agency (Aria, which is the country’s answer to the U.S. Defense Department’s DARPA) is hoping to bring about. Last week, Aria announced it was committing £42 million ($53 million) to fund projects working towards reducing the current energy footprint of running AI applications by a factor of a thousand. It said it would consider radically different ways of building computer chips in order to do so, including chips that rely on biological neurons for computation instead of silicon transistors. (I wrote about one such effort in 2020.)
The effort is pretty sci-fi and may not yield the results Aria hopes. But the very fact the Aria challenge exists and that Nvidia is now putting energy efficiency on center stage at GTC are signs the world is getting serious about tackling AI’s carbon footprint. Hopefully, this means AI won’t destroy our efforts to build a more sustainable world.
There’s more AI news below. But first, if you’re enjoying reading this newsletter, how would you like to participate in a live version—chatting in-person and IRL with me and many of the world’s foremost experts on deploying AI within companies? If that sounds intriguing, please apply to attend Fortune’s Brainstorm AI conference in London on April 15 and 16. I’ll be there cochairing the event and moderating sessions. You will get to hear from Google DeepMind’s Zoubin Ghahramani, Microsoft chief scientist Jaime Teevan, Salesforce chief ethical and human use officer Paula Goldman, as well as Shez Partovi, the chief innovation and strategy officer for Royal Philips, Accenture’s chief AI officer Lan Guan, Builder.ai CEO Sachin Dev Duggal, and many others. Email [email protected] to apply to attend. I hope to see you there!
With that, here’s the AI news.
Jeremy Kahn
[email protected]
@jeremyakahn
AI IN THE NEWS
Microsoft hires DeepMind cofounder Suleyman and much of his Inflection team. Bloomberg reports that Microsoft is hiring former DeepMind cofounder Mustafa Suleyman, currently the founder and CEO of Inflection AI, to head a new consumer AI division, reporting directly to Microsoft CEO Satya Nadella. Many of those currently working with Suleyman are also moving over to Microsoft.
Inflection will continue as an independent company but shelve its Pi chatbot—which was designed to be an empathetic companion to users—and focus instead on selling AI solutions to business customers. Microsoft and Nvidia had previously invested in Inflection, which had raised $1.3 billion in a venture capital round in June that valued the company at a reported $4 billion. Inflection recently reported that Pi had one million active daily users. But Suleyman told Bloomberg that the company had not found a good business model for the chatbot.
Suleyman said that in his new role at Microsoft, he will be in charge of building compelling consumer products on top of underlying AI models, including both those Microsoft builds in-house and ones it receives through its partnership with OpenAI.
Apple reportedly in talks with Google to license Gemini models. That’s according to a Bloomberg story that cited unnamed sources familiar with the negotiations. The article said the iPhone maker is talking to Google about using Gemini to power cloud-based generative AI features on its phones, which might write documents or generate images, while it continues to work on building its own large language models that are expected to power a future generation of on-device AI applications.
DHS unveils AI roadmap. The Department of Homeland Security announced its AI roadmap this week. The roadmap will see the department pushing forward in three areas: using AI to promote DHS’s own mission, promoting nationwide AI safety and security, especially around critical national infrastructure, and cementing partnerships with both state and local governments and international partners. President Joe Biden’s October 2023 Executive Order on AI instructed DHS to prepare a plan to ensure the safety of AI systems used in critical infrastructure, like power grids, to reduce the risks that AI could be used to create bioweapons or other weapons of mass destruction. The executive summary contains some interesting ways DHS is already using AI to spot suspicious patterns of vehicle crossings at border points and locate child sexual abuse victims.
Chinese and Western academics issue stark warning on AI risks and share ‘red lines.’ Chinese and Western experts in AI and international security met in Beijing last week and agreed to several "red lines" that all nations should ensure AI does not cross, including AI with the ability to autonomously create bioweapons or perpetrate devastating cyberattacks, the Financial Times reported. The joint statement from the meeting of the International Dialogue on AI Safety was signed by several AI luminaries, including Turing Award winners and deep learning pioneers Geoffrey Hinton and Yoshua Bengio, Stuart Russell, a well-known computer scientist at the UC Berkeley, and Andrew Yao, among China’s most prominent computer scientists. Comparing the international agreement to similar agreements around weapons systems made during the Cold War, the group’s statement said that “humanity again needs to co-ordinate to avert a catastrophe that could arise from unprecedented technology.”
SEC fines two investment firms for “AI washing.” That’s the term for a company making misleading claims about its use of AI and the performance of AI systems. The SEC fined Toronto-based Delphia and San Francisco-based Global Predictions a combined $400,000 to settle civil charges, Reuters reported. The firms neither admitted nor denied wrongdoing as part of the agreement.
New pro-AI acceleration lobbying group sets up shop. Alliance for the Future, a new D.C. trade association affiliated with the effective accelerationist (or e/acc) movement has launched. The group wants to serve as a counterbalance to what it sees as undue influence in Washington policymaking circles from “AI doomer” groups that are concerned about AI’s existential risks. You can read more here.
EYE ON AI NUMBERS
$1 billion
That is how much training runs for the largest large language models may soon cost. That’s according to James Hamilton, a distinguished engineer at Amazon’s AWS. The figure appeared in slides for a talk Hamilton gave at a conference and recently posted to his blog. (H/t to Anthropic’s Jack Clark and his Import AI newsletter for highlighting the news.) In the slides, Hamilton noted that AWS had in the past year spent about $65 million training a 200 billion parameter model on 4 trillion tokens of data using 13,760 older generation Nvidia A100 chips for 48 days. But as Hamilton’s slides indicate, this is a “1 gen old” technology. Bigger models, trained on newer chips, probably cost 10 times as much. And the $1 billion figure comes from extrapolating those numbers out to the next generation of models.
FORTUNE ON AI
Sam Altman is over GPT-4: ‘I think it kind of sucks’ —by Chris Morris
How AI can make U.S. cities smarter, safer, and greener —by Nick Rockel
How AI could help make the IVF process easier —by Alyssa Newcomb
AI CALENDAR
March 18-21: Nvidia GTC AI conference in San Jose, Calif.
April 15-16: Fortune Brainstorm AI London (Register here.)
May 7-11: International Conference on Learning Representations (ICLR) in Vienna
June 25-27: 2024 IEEE Conference on Artificial Intelligence in Singapore
BRAIN FOOD
AI Doomers are a little weird. But so what? The New Yorker ran a big feature story by Andrew Marantz headlined “Among the A.I. Doomsayers” on the subculture of people in the Bay Area who are dedicated to AI safety research and trying to prevent what they fear may be extinction-level risks from advanced AI. Many of these people are affiliated with the philosophical movement Effective Altruism or the somewhat related Rationalist movement—both of which believe in applying cost-benefit analysis to figure out how to lead a moral life. They also live in group houses and enjoy cuddle puddles. The story is a fun read. But at the end of the day, I’m not quite sure what Marantz’s point is. I guess he wants us to question whether we should trust the AI Safety crowd’s pronouncements about AI gloom and doom because they are all a little weird. But the article doesn’t really try to engage with the core issue of whether AI actually might pose an existential risk to humanity, how big that risk might be, and what we should do about it. Instead, it basically pokes fun at the doomer lifestyle. But that doesn’t mean their ideas about AI necessarily are kooky. And given the consequences of getting this wrong, it would have been nice if The New Yorker had actually tried to examine the substance of the “AI doomer” vs. “e/acc” debate, rather than skimming the surface.
This is the online version of Eye on AI, Fortune's weekly newsletter on how AI is shaping the future of business. Sign up for free.