A debate among some of the technology industry’s most prominent leaders is putting a new question at the center of the artificial intelligence discussion: How much control should companies have over the pace of increasingly capable AI systems?
The question gained attention in September after Anthropic CEO Dario Amodei argued that leading AI companies should take steps to slow the development of the most advanced systems and establish stronger safeguards around frontier models. OpenAI CEO Sam Altman and other technology leaders have also discussed the need for greater caution as AI capabilities continue to advance.
David Sacks, a venture capitalist and technology adviser to President Donald Trump, responded with a different argument. In a September 13 post on X, Sacks said companies developing frontier models could choose to slow their own development if they believe their systems present serious risks but questioned the need for outside permission or additional regulatory requirements.
The disagreement highlights a broader issue that extends beyond any one company: who should establish the boundaries for advanced AI development, and how should those boundaries be enforced?
A Debate Over “Pacing the Frontier”
Amodei’s proposal centers on the idea that the development of increasingly powerful AI systems may require additional safeguards as capabilities approach more advanced levels. His argument has focused on the possibility that AI companies could voluntarily establish limits, use independent evaluations and coordinate around safety measures.
Sacks’s response takes a different approach. He points to the enormous market position of the companies already developing frontier models and argues that those companies have the ability to make their own decisions about how quickly they advance.
In other words, the disagreement is not simply about whether AI safety matters. The more fundamental question is who should be responsible for determining what constitutes an acceptable level of risk?
The Market as a Safety Mechanism
One of Sacks’s central arguments is that market competition can provide a form of discipline. The leading AI companies compete for customers, developers, investment and market share. A serious safety failure could damage a company’s reputation, customer relationships and financial position. From this perspective, companies already have incentives to avoid releasing systems that create unacceptable risks.
Some analysts have described Sacks’s position as an argument for self-regulation by the industry’s largest AI developers. Enterprise DNA, for example, characterized his response as suggesting that competition between major labs can provide a form of market discipline.
There is an important assumption behind that argument: companies will have sufficient incentives to prioritize safety even when safety measures could delay a product, increase development costs or allow competitors to move ahead.
That assumption is one of the issues at the center of the larger debate.
The Case for Independent Oversight
Supporters of stronger external oversight raise a different concern. A company may have strong internal safety practices and still face pressure to release more capable systems quickly because competitors are moving forward.
That creates a potential coordination problem.
If several companies independently believe that slowing down would be responsible but also fear losing market share by doing so, voluntary restraint can become difficult to sustain.
Independent evaluations are one proposed solution. Outside organizations could assess advanced models for capabilities and risks without being directly responsible for developing or selling the systems being evaluated.
The details, however, remain unsettled. Questions include who would conduct those evaluations, what information companies would have to provide, how independent evaluators would remain, and who would have authority to act on concerning findings.
The debate therefore involves more than simply deciding between regulation and no regulation. It also involves designing systems that can produce credible information about the risks associated with increasingly capable AI.
The Problem With a Global Slowdown
Another complication is international competition. Companies and research organizations in multiple countries are competing to develop increasingly capable systems, creating a difficult environment for any agreement that depends on everyone moving at the same pace. A voluntary slowdown among American companies would have limited value if competitors elsewhere continued developing similar systems at full speed.
This concern has become part of the political debate over AI as well. President Donald Trump has argued against slowing American AI development and has emphasized maintaining U.S. technological leadership relative to China. Sacks similarly argued that new federal AI regulation was unnecessary and that developers should remain responsible for the safety of their products.
Regulation Could Create Its Own Tradeoffs
The discussion also has an economic dimension.
Large technology companies can potentially absorb compliance costs, hire specialized legal and technical teams, and participate in complex regulatory processes more easily than smaller companies. New requirements could therefore have different effects depending on the size and resources of an organization.
At the same time, the absence of clear standards can create uncertainty for businesses and consumers using AI systems.
For companies adopting AI tools, questions about liability, data handling, model safety, privacy and reliability are already becoming part of technology procurement decisions. The rules established for frontier AI could eventually influence the products and services available to businesses that never develop an AI model themselves.
This makes the regulatory debate relevant beyond Silicon Valley.
Safety and Commercial Incentives Can Point in Different Directions
One of the more complicated aspects of the discussion is that safety and commercial interests are not always perfectly aligned. Companies have a financial incentive to create reliable products that customers trust. A serious failure can produce significant costs. At the same time, companies also compete on how quickly they can improve models and introduce new capabilities.
That creates a tension between moving quickly enough to remain competitive and moving cautiously enough to identify problems before systems become more capable.
There is no simple measurement that resolves that tension.
AI developers can conduct evaluations and establish safeguards, but the effectiveness of those measures depends on what they test, how comprehensive the evaluations are and what happens when a potential problem is identified.
What Remains Unclear
It is not clear how a voluntary international agreement to slow frontier AI development could be reliably verified. It is also unclear how regulators could distinguish between legitimate safety requirements and rules that unintentionally reinforce the position of the largest technology companies.
Another uncertainty involves the definition of “frontier” AI itself. Capabilities are changing quickly, making it difficult to establish a permanent threshold for the systems subject to special oversight. There is also limited evidence available today about how future AI systems will behave as their capabilities expand. Predictions about extreme risks remain contested, and the severity and timing of potential harms are subjects of ongoing research rather than settled facts.
Those uncertainties do not resolve the policy debate, but they explain why the discussion has become increasingly focused on testing, evaluation and evidence.
The Debate Is Bigger Than One X Post
Sacks’s response is one part of a much larger argument unfolding among AI companies, researchers, policymakers and technology executives.
The disagreement is not simply about whether AI should be developed. Most participants in the debate recognize the technology’s potential benefits while also acknowledging that increasingly capable systems can create new risks.
But how should society balance those competing considerations?
A company-led approach emphasizes innovation, competition and the incentives businesses already have to protect their products and customers. An external oversight approach emphasizes independent evaluation, consistent standards and safeguards that do not depend entirely on individual companies making the same voluntary decisions.
Both approaches raise practical questions about implementation.
As AI systems become more capable, those questions will increasingly affect businesses, workers, consumers and governments far beyond the companies building the technology. The debate over pacing is therefore becoming part of a much broader discussion about how technological progress should be governed when the technology itself is advancing faster than many existing regulatory frameworks.
