Open-weight AI models shift where power concentrates but don't eliminate the problem, according to Anthropic CEO Dario Amodei. In a public exchange on X with investor Gavin Baker, Amodei argued that artificial intelligence inherently consolidates control because of scaling laws, not government policy. Open weights help developers adapt models and keep data private, but those benefits shrink as models grow and the cost of running them climbs.
The debate centered on whether regulation would lock advanced AI inside a few companies or whether distributing models more widely would spread power more evenly. Baker, managing partner at Atreides Management, cited Meta CEO Mark Zuckerberg's view that concentrating AI among a small group of supposedly responsible organizations carries its own risks. He contended that releasing more models would give people access to systems that reflect their values and warned that Amodei's repeated emphasis on AI dangers could fuel opposition to new data centers. Amodei countered that the choice between regulation and distribution presents a false dilemma, arguing that rules based on objective standards can restrain powerful companies rather than protect them.
"AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation," Amodei wrote. He added that "open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips." Amodei said institutions at their best "vest power in ideas rather than people, and thereby decentralize that power," rejecting what he called Silicon Valley's oversimplified equation of regulation with regulatory capture. The Anthropic CEO wants models capable of enabling serious attacks to undergo mandatory safety testing, whether released as open weights or kept proprietary, calling open models without dangerous capabilities "a public good."
The infrastructure bottleneck is already visible. Five European companies recently committed to buying AI computing capacity that hasn't been built yet, underscoring how smaller teams remain vulnerable to hardware costs and availability even when they can rent access. Once a model's weights become public, the original developer can't take back every copy or stop users from stripping out safety features, a containment problem that worsens as capabilities increase. Amodei has backed frameworks that impose stricter requirements on frontier labs—those training models using more than 10^26 floating-point operations, or developers earning over $500 million annually—while exempting smaller competitors. He supports classification systems where models cross into frontier territory after hitting regularly updated benchmark thresholds, applying the same rules to open and closed releases.
Anthropic argues its proposals disadvantage frontier AI companies while giving challengers room to grow, and Amodei has called for testing processes that screen frontier models more rigorously than others. The company supported California's SB 53, which uses training compute as a threshold, and Amodei backs a FINRA-like standards body for AI. But capability doesn't always follow simple cutoffs—fine-tuning and external tools can add abilities training compute won't capture, and benchmarks rarely offer clean dividing lines. The U.S. Center for AI Standards and Innovation found that Moonshot AI's Kimi K3 completed a full 32-step simulated corporate network attack in one of 10 tries, despite trailing the strongest closed models in preliminary cyber evaluations and failing to achieve arbitrary code execution on any of 41 ExploitBench samples where top models averaged 20 successes. DeepSeek showed the same unpredictability when its smaller model beat its own flagship, proving that size alone won't reliably measure what a model can do. Regulators will need to identify when a challenger has entered frontier territory before its weights are released and modified beyond anyone's control. For developers, the practical question remains whether owning the weights provides genuine independence when someone else still holds the keys to the computing power required to run them.