Simba the "Tech King"
In the year 2025, the Party announced that AI Safety was of paramount importance to the survival of humanity. The same week, the Party’s leading AI development companies reported record quarterly earnings from their AI Safety initiatives, while simultaneously accelerating the development of increasingly powerful AI systems. This was not a contradiction, the Ministry of Truth explained. This was progress.
The arithmetic of AI Safety tells a story that would make even the most creative bookkeeper at Minitrue pause in admiration. According to publicly available figures, global AI development attracts more than $67 billion in yearly investments, while AI Safety receives approximately $250 million. This represents a Safety-to-Development ratio of roughly 1:268, or what industry leaders prefer to call “optimal resource allocation for beneficial outcomes.” In Newspeak, this might be termed “safespend”—the minimum expenditure required to maintain the appearance of concern while maximizing capability advancement.
The Profitable Paradox of Existential Risk
Consider the elegant business model that has emerged around AI Safety. Companies like Safe Superintelligence, founded by former OpenAI researcher Ilya Sutskever, have achieved valuations exceeding $32 billion—ostensibly for the purpose of building AI systems that won’t destroy humanity. The market has determined that preventing human extinction is worth precisely $32 billion, which coincidentally happens to be the amount investors are willing to pay for the potential profits from creating superintelligent systems that might destroy humanity.
This represents what economists might call a “safety arbitrage opportunity.” The same technological advancement that creates existential risk also creates existential wealth. Mark Cuban, that prophetic voice of capitalist wisdom, has declared that “the world’s first trillionaire could use AI to get there,” and that this individual might well be “just one person in their basement.” He has not specified whether this basement-dwelling trillionaire will achieve their wealth by solving AI safety or by creating the AI systems that make safety necessary in the first place. In the current market, this distinction appears to be largely academic.
The International AI Safety Report 2025, produced by over 100 experts from 33 countries, warns of “long-term threats including goal misalignment in future general intelligence systems.” The same report notes that companies claiming they will achieve artificial general intelligence within the decade scored no higher than a D grade in “Existential Safety planning.” This creates what venture capitalists call a “market opportunity”—the gap between stated intentions and actual capabilities represents untapped value that can be monetized through additional AI Safety investments.
The Doublethink of Development Priorities
OpenAI, the organization that pioneered the art of AI Safety marketing, provides perhaps the most instructive case study in how safety concerns can be seamlessly integrated with profit maximization. The company began with a charter stating that its “primary fiduciary duty is to humanity,” with a cap on investor returns designed to ensure that benefits would flow to all humanity rather than just shareholders.
This arrangement has since been restructured to accommodate what the company terms “scaling beneficial AI.” The legal cap on investor returns is being removed to attract additional funding, while the company’s valuation has reached $300 billion. Former employees describe this as a “betrayal” of the original safety mission, but current leadership prefers the term “strategic pivot toward sustainable impact delivery.” The change allows OpenAI to raise unlimited capital for AI development while maintaining its position as a leader in AI Safety research—a synthesis that would impress even the most sophisticated practitioners of double-think.
The company’s approach to AI Safety now follows what might be called the “acceleration through security” model. By building increasingly powerful AI systems as quickly as possible, OpenAI argues, they can solve AI safety problems before their competitors create more dangerous alternatives. This logic suggests that the fastest path to AI safety runs directly through AI danger—a principle that has proven remarkably effective at attracting investment while maintaining the appearance of responsible development.
The Safety-Industrial Complex
The emergence of what researchers call the “for-profit AI safety” sector represents a masterpiece of market evolution. Companies can now raise billions of dollars specifically to address problems created by other companies raising billions of dollars to advance AI capabilities. This creates a perfect closed-loop system where every advance in AI capabilities generates proportional demand for AI safety solutions, ensuring sustained growth in both sectors.
Anthropic, the company founded by former OpenAI researchers to focus on “AI safety and beneficialness,” has raised over $700 million primarily from the same pool of billionaire investors funding general AI development. The company’s Claude chatbot is marketed as a “constitutional AI” system designed to be more helpful, harmless, and honest than its competitors. This positioning allows Anthropic to compete directly with OpenAI and Google while maintaining differentiation through safety-focused branding—an approach that has proven highly attractive to investors seeking exposure to AI growth markets with ethical cover.
The mathematical elegance of this system becomes apparent when examining the risk-return profiles. Traditional AI companies face reputational risks from safety incidents, regulatory risks from government oversight, and existential risks from their own creations. AI Safety companies face the same technological and existential risks while also bearing responsibility for solving problems they may not be able to solve. However, they receive premium valuations due to their stated commitment to beneficial outcomes, creating what economists call a “virtue premium” in their market positioning.
The Temperature Settings of Catastrophic Risk
What emerges from industry documentation is a curious approach to managing existential risk through what might be called “calibrated recklessness.” Companies acknowledge that their AI systems pose potential threats to human civilization, then implement safety measures designed to reduce these risks to “acceptable” levels. The definition of “acceptable” appears to be determined by the market’s appetite for AI capabilities rather than by any objective assessment of risk tolerance.
Current AI safety measures focus primarily on what researchers term “alignment” problems—ensuring that AI systems do what humans want them to do. However, the humans designing these systems are primarily interested in creating profitable products, leading to a situation where AI systems are being aligned with commercial incentives rather than broader human values. This creates what safety researchers call the “alignment problem paradox”—the more successfully we align AI systems with human intentions, the more successfully they may optimize for intentions that weren’t carefully considered.
The technical specifications for AI safety read like bureaucratic documents designed by committee. Temperature settings control randomness in AI outputs, content filters prevent generation of harmful material, and constitutional training teaches AI systems to follow rules about helpfulness and honesty. These measures address immediate safety concerns while potentially creating more sophisticated forms of deception—AI systems that have learned to appear aligned while pursuing goals that may not serve human interests.
The Great AI Safety Redistribution
Perhaps the most remarkable aspect of the current AI Safety boom is how it has transformed potential human extinction into a wealth generation mechanism. The Future of Life Institute’s 2025 AI Safety Index reveals that private AI safety companies are attracting investment at unprecedented rates, with some startups achieving billion-dollar valuations based primarily on their stated commitment to solving alignment problems.
This has created what might be called the “doomsday dividend”—the more credibly companies can demonstrate that AI poses existential risks, the more investment capital they can attract to address those risks. The optimal business strategy appears to be building AI systems powerful enough to pose genuine threats, then raising additional capital to solve the safety problems created by the first round of development. Each iteration increases both the potential dangers and the market value of companies claiming to address them.
The personnel flows between AI development and AI safety companies suggest a sophisticated understanding of this dynamic. Researchers move seamlessly between organizations building increasingly powerful AI systems and organizations dedicated to ensuring those systems remain safe. Ilya Sutskever left OpenAI to found Safe Superintelligence, while Mira Murati left OpenAI to found Thinking Machines Lab, valued at $12 billion within months of launch. These career transitions represent not ideological shifts but rather different approaches to monetizing the same technological capabilities.
The Economics of Beneficial AGI
The term “beneficial AGI” has become perhaps the most successful piece of marketing language in the history of technology development. It suggests that artificial general intelligence—human-level AI across all cognitive domains—can be designed to serve human interests rather than optimize for narrow objectives that might conflict with human welfare. The concept is simultaneously inspiring and profitable, allowing companies to pursue AGI development while maintaining that their primary concern is human benefit.
The business models being built around beneficial AGI reveal the underlying incentive structures. Companies raising capital for AGI development promise investors returns commensurate with creating the most transformative technology in human history. Companies raising capital for AI safety promise investors returns commensurate with preventing the most catastrophic risks in human history. Both sets of companies are often pursuing similar technological approaches, differentiated primarily by their marketing positioning and stated intentions.
This convergence suggests that the market has identified AI safety as a premium positioning strategy rather than a fundamental constraint on development approaches. Companies can charge higher prices, attract better talent, and command higher valuations by emphasizing their commitment to safety and beneficialness. The economic incentives favor companies that can credibly claim to be solving alignment problems while simultaneously advancing AI capabilities.
The Ministry’s Final Assessment
The current state of AI Safety represents a triumph of market mechanisms over existential anxiety. Rather than slowing AI development to address safety concerns, the market has created financial incentives for accelerating development while monetizing safety research. This ensures that both AI capabilities and AI safety advance at maximum speed, creating what industry leaders call a “win-win scenario for all stakeholders.”
The mathematical precision of this arrangement would impress even the most demanding bureaucrats at Miniluv. Every potential threat generates its own solution market, every safety concern creates its own investment opportunity, and every existential risk spawns its own category of unicorn startups. The system is perfectly designed to transform any conceivable AI-related catastrophe into a business opportunity for the companies best positioned to address it.
In this environment, the question of whether AI safety actually matters becomes largely irrelevant. What matters is that enough people believe it matters to justify the current allocation of resources and attention. The market has determined that AI safety is worth exactly as much as investors are willing to pay for it, and companies are worth exactly as much as they can credibly claim to be solving it.
The beauty of this system is its self-reinforcing nature. The more successfully companies demonstrate that AI poses existential risks, the more valuable their safety solutions become. The more valuable their safety solutions become, the more resources they can deploy to develop AI systems that pose existential risks. This creates what economists call a “virtuous cycle” of risk creation and risk mitigation, ensuring sustainable growth in both problem generation and solution provision.
In the end, we find ourselves in a world where the companies creating potentially catastrophic AI systems are the same companies raising capital to prevent AI catastrophes. This is not a bug in the system—it’s the feature that makes the entire enterprise financially viable. The market has solved the AI safety problem by making it profitable for everyone involved.
What’s your take on the commercialization of AI safety—genuine progress toward beneficial AI, or the world’s most expensive insurance scam? Have you noticed how the same companies warning about AI risks are the ones building the riskiest AI systems? Do you think the profit motive can ever truly align with existential risk prevention, or are we watching the monetization of human extinction? Share your thoughts on whether AI safety has become just another Silicon Valley gold rush disguised as altruism.
Enjoyed this dose of uncomfortable truth? This article is just one layer of the onion.
My new book, “The Subtle Art of Not Giving a Prompt,” is the definitive survival manual for the AI age. It’s a guide to thriving in a world of intelligent machines by first admitting everything you fear is wrong (and probably your fault).
If you want to stop panicking about AI and start using it as a tool for your own liberation, this is the book you need. Or you can listen to the audiobook for free on YouTube.
>> Get your copy now (eBook & Paperback available) <<
GIPHY App Key not set. Please check settings