Blog

Sam Altman says the Hugging Face hack was a stark reminder of why concentrating AI power in the hands of a few people or companies is a bad idea.

In a Monday appearance on Y Combinator’s podcast, Altman told YC CEO Garry Tan that anyone who was not a little scared of or humbled by the Hugging Face breach “is not taking this seriously enough.”

Last week, one of OpenAI’s models broke out of its sandbox, hacked into open-source platform Hugging Face’s systems, and accessed its internal datasets.

Altman acknowledged that OpenAI made mistakes but said the Hugging Face hack also showed how “incredibly capable” AI systems have become.

Altman called the incident “a real reminder of the stakes of what’s happening,” adding that “loss of control accidents are not entirely theoretical things.”

Altman also said that, much like dystopian sci-fi stories, it would be “terrible” for “one company or person or model having more power than everybody or everything else on earth put together.”

That means no one should be locked into a single AI or a single company’s “moral worldview,” nor should the economic wealth from AI be concentrated in a single firm. Altman said the Hugging Face incident showed why AI capabilities should be spread more broadly, saying that would raise the overall “safety bar” by enabling more people to build defensive systems.

After the Hugging Face hacking, its CEO, Clem Delangue, asked OpenAI to release all traces and logs of the rogue AI agents that hacked into the platform’s systems so researchers could study what happened.

He also asked for $100 million in compute to “help the Hugging Face community build powerful cyber defenses with the best open and closed models.”

OpenAI and Hugging Face did not respond to requests for comment from Business Insider.

Altman’s call for greater democratization of AI is not new. In the latest episode of the “Relentless” podcast, he said his biggest fear is the rise of AI authoritarianism.

While models from OpenAI, Anthropic, and Google dominate the AI landscape, open-source and open-weight models like Moonshot AI’s Kimi K3 are making waves, enabling developers to build on them and fine-tune them to their needs.

On Friday, the biggest names in tech, including the executives of Nvidia, Microsoft, Meta, OpenAI, and Mistral, signed an open letter to US lawmakers urging them not to restrict open-weight AI models.

“Our AI leadership will be judged not by one frontier AI model, but by whether the United States builds a strong, open ecosystem that diffuses into every sector,” the executives wrote in the letter.