The Convenient Fear

In short: Some of the loudest warnings about AI come from the companies building it. Whatever their origins, those warnings shift attention away from present, accountable design choices, like training data, tuning and release timing, toward a hypothetical machine nobody controls. Accountability should focus on the people making those choices.

Most industries try to convince you their product is safe. AI is one of the few where the people selling it keep telling you it might end the world.

That should strike us as strange. Tobacco companies didn't warn about cancer. Oil companies spent decades downplaying climate change. Social media companies insisted for years that their feeds were neutral. And yet the executives building the most powerful AI systems have signed public statements comparing the risk of their own technology to pandemics and nuclear war, testified to Congress asking to be regulated, and described their products in language usually reserved for weapons.

Some of the fear may be warranted. My argument is that, whatever its origins, it now works less as a warning than as a strategy. The story of AI as a dangerous, autonomous, almost alien intelligence serves the companies telling it, because it moves attention away from the thing they actually control: how the models are built. A frightening machine is a machine with a will of its own. A machine with a will of its own is something nobody can be fully blamed for.

Who Is Warning About AI Risk?

In 2023, the heads of OpenAI, Google DeepMind and Anthropic all signed a one-sentence statement saying that mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war. The same year, OpenAI's CEO sat in front of the Senate and suggested a new federal agency to license the most capable models.

Whether or not anyone intended it, look at the effect. The companies with the most advanced systems told the public their product was potentially catastrophic, then proposed licensing rules that critics warned would favor the largest labs, the ones best able to afford compliance. And they kept building.

The language matters too. Listen to how these systems get described: they "decide," they "want," they "hallucinate," they might "deceive" or "escape." Each verb puts the agency in the machine. Very little of the public story is told in the passive voice it deserves: the model was trained this way, the data was chosen this way, the guardrails were set here, the release date was picked for competitive reasons. The scary version is more dramatic. It's also far more convenient for the people making those choices.

Who Benefits From Fear of AI?

Fear does at least four jobs for the companies that spread it.

It makes the product sound more powerful than it is. Nothing sells capability like danger. If a system might threaten humanity, it must be extraordinarily capable, and an extraordinarily capable system justifies extraordinary valuations, investment and talent wars. The historian Lee Vinsel has a name for this: criti-hype, criticism that ends up advertising the thing it criticizes.

It shapes regulation in favor of incumbents. When the threat is framed as a future superintelligence, the proposed fixes tend to be licenses, compute thresholds and safety evaluations that only the largest labs can run. That raises the drawbridge behind them. Smaller competitors and open alternatives become the risky ones.

It points attention at tomorrow instead of today. A debate about a hypothetical machine that might take over in a decade crowds out the harms happening now: biased outputs in hiring and lending, creative work scraped without consent, workers displaced without a plan, confident errors in medical and legal answers. Those harms have plaintiffs, numbers and responsible parties. The apocalypse has none.

It diffuses responsibility. This is the one that matters most. If the danger lives in the technology itself, then the people who built it become stewards of something wild rather than authors of something designed. "We're doing our best to contain it" is a much easier position to defend than "we chose to build it this way."

What Does the Fear Hide?

A language model is not a creature. It's a long chain of human decisions.

Someone decided what data it would learn from, and what to leave out. Someone decided whose writing counted and whose permission mattered. Someone wrote the instructions for the people who rated its answers, which shaped what it learned to call helpful, harmful or true. Someone chose where the guardrails sit, what it will refuse, and how it should handle a question about politics, health or a person's name. Someone decided when it was good enough to ship, and someone decided the competitive timeline mattered more than another round of testing.

Every one of those is a design choice. Every one could have gone another way. And every one is largely invisible to the public, protected as a trade secret while the conversation stays fixed on what the model might someday do on its own.

I see this in my own work. When clients bring up AI risk, they almost always mean the dramatic version: the tool going rogue, the system doing something nobody asked for. Almost nobody asks what data the tool was trained on, who decided what it would refuse, or what the vendor is liable for when it's wrong. The fear story has trained us to ask the wrong questions.

When a model gives a biased answer, the fear story says the AI is unpredictable. The accountable story says the data and the tuning produced that answer, and someone signed off on both. When a model invents a legal citation, the fear story calls it a hallucination, as if the system had a bad dream. The accountable story says the product was released knowing it would state falsehoods with confidence, and was marketed for professional use anyway.

The difference between those two stories is the difference between a weather event and a product defect. One has no one to answer for it. The other does.

What Would AI Accountability Look Like?

If the danger is in the design, then accountability belongs in the design too. That would mean treating AI companies the way we treat anyone else who builds products that shape people's lives.

• Disclose the inputs. What data a model was trained on, at least in categories, and whether the people who made that data agreed to it.
• Document the choices. The rules given to the people who rate model behavior, the values the model was tuned toward, and the known failure modes at release.
• Open the model to independent testing. Not just company-run evaluations announced in blog posts, but outside auditors with real access and the freedom to publish what they find.
• Attach liability to deployment. If a company markets a model for medical, legal, financial or hiring decisions, it owns the predictable failures in those settings.
• Regulate present harms first. Bias, privacy, consent and labor impact are measurable now. They don't need a theory of superintelligence to justify action.

None of this requires believing AI is harmless. It only requires refusing to treat it as ungovernable.

The Case Against This Argument

An argument worth making should survive its best objections, so here they are.

Many of the people raising alarms don't work for these companies. Geoffrey Hinton left Google in 2023 specifically so he could speak about the risks freely. Yoshua Bengio is an academic. Hundreds of independent researchers share the concern. Calling the fear corporate marketing ignores people who have nothing to sell.

Some warnings came before the money. Concerns about advanced AI were being raised by researchers and philosophers long before these companies had products. The fear didn't originate in a marketing department, even if marketing has since found it useful.

Fear isn't free for the companies. Alarm invites scrutiny, lawsuits, regulation and public distrust. If the goal were simply to escape accountability, convincing people the product is dangerous is a strange way to do it.

Safety work can be real even when incentives are mixed. Companies can sincerely worry about risks and also benefit from talking about them. Motives are rarely pure. A self-serving warning can still be correct.

Both things can be true. It's possible that long-term risks are worth taking seriously and that the framing has been used to deflect present accountability. That has been the argument all along: not that the fear is false, but that it has been allowed to crowd out questions about choices being made right now, by people with names.

Close

The most useful question to ask about any AI system isn't "what might it do?" It's "who decided it should work this way, and why?"

The first question keeps us staring at the machine. The second points at the people. As long as the public conversation is about a technology that might slip out of control, the companies building it get to play the role of nervous guardians rather than accountable manufacturers.

I'd rather we stop being afraid of the model and start asking about the choices behind it. Who decided it should work this way, and why? That question has an answer. Someone should have to give it.

Frequently Asked Questions

Did AI company leaders warn about extinction risk? Yes. In 2023 the heads of OpenAI, Google DeepMind and Anthropic signed a one-sentence statement saying that mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war.

What is criti-hype? A term from historian Lee Vinsel for criticism that ends up advertising the thing it criticizes, for example warnings that make a technology sound more powerful than it is.

What are present-day AI harms? Biased outputs in hiring and lending, creative work used without consent, workers displaced without a plan and confident errors in professional settings.

Is AI risk real? Some long-term concerns are shared by independent researchers with nothing to sell. The argument here is not that the fear is false, but that it has crowded out accountability for choices being made now.

Evan Prout is a strategist in Grand Rapids, Michigan, who writes about how organizations think and decide with AI.