How To Think About AI

1. AI's Rapid Evolution and Unprecedented Capabilities

In truth, we are simply warming up. It is early days for AI.

Beyond human tasks. Artificial intelligence is a branch of digital technology dedicated to systems that perform tasks traditionally requiring human intellect, such as speech recognition, language translation, medical diagnosis, and legal drafting. Early AI (GOFAI) focused on broad categories, but modern AI excels in specific, complex tasks, often outperforming human experts. Examples include AI systems that diagnose illnesses, interpret scans, draft legal documents, and even accelerate vaccine development.

Exponential growth. The field has seen a dramatic acceleration in breakthroughs, from every five to ten years in early decades to every six to twelve months now. This rapid advancement is driven by exponential growth in computational power, memory, and data storage, exemplified by the doubling of AI training power every six months between 2010 and 2022. This means computation speeds could improve a millionfold in a decade, leading to systems that can self-correct, self-improve, and even self-replicate.

A new summer. Landmark achievements like IBM's Deep Blue beating Garry Kasparov (1997), Watson winning Jeopardy! (2011), AlphaGo mastering Go (2016), and AlphaFold solving protein folding (2018) demonstrate AI's increasing prowess. The launch of ChatGPT in late 2022, a generative AI system, marked a public milestone, attracting 100 million users in two months and showcasing AI's ability to generate text, art, music, and code. These systems are not the endgame but harbingers of even more capable future iterations.

2. Overcome Biases: Embrace "What-If-AGI?" Thinking

Most predictions about the future are in my view irredeemably flawed because they ignore the not yet invented technologies.

Beware biases. Two common biases distort our understanding of AI's future: "technological myopia," which fixates on current limitations (e.g., AI "hallucinations") and ignores exponential progress, and "irrational rejectionism," which dismisses AI's potential without direct engagement. These biases lead to underestimating AI's long-term impact and hinder effective planning.

Future capabilities. The author outlines five hypotheses for AI's future: Hype (AI is overblown), GenAI+ (improved current generative AI), AGI (Artificial General Intelligence, human-level performance), Superintelligence (unfathomably more capable than humans), and Singularity (merger of humans and AI). The author finds the Hype Hypothesis implausible due to market demand and economic value.

Plan for AGI. The author predicts AGI (or near-AGI) could arrive between 2030 and 2035, a significant acceleration from previous estimates. This necessitates "what-if-AGI?" thinking: assuming AI systems can match or outperform humans in almost all cognitive tasks to provoke richer public debate and systematic planning for its implications, rather than focusing solely on today's generative AI. This proactive approach is crucial for managing both benefits and risks.

3. Distinguish Process-Thinking from Outcome-Thinking

For Kissinger, what these systems do is of paramount concern, rather than how they do it.

Two cultures. The AI world is split between "process-thinkers" and "outcome-thinkers." Process-thinkers, like Noam Chomsky, focus on how complex systems (AI or human minds) work, often highlighting AI's mechanistic nature. Outcome-thinkers, like Henry Kissinger, prioritize the results or impact of AI systems, such as their societal consequences, economic benefits, or practical applications.

Muddled debates. This distinction clarifies why experts hold radically different views on AI. Process-thinkers might trivialize AI by focusing on its underlying mechanisms (e.g., "just a glorified autocomplete"), potentially obscuring its massive practical consequences. Conversely, outcome-thinkers might make bold claims about AI's impact without fully appreciating the technological complexities or the cultural shifts required for their predictions to materialize.

"Not-us thinking." Professionals often exhibit "not-us thinking," believing AI has potential for others but not their own field, rooted in psychological conservatism and the "AI Fallacy." This fallacy assumes machines must replicate human processes to achieve high-level performance. However, AI systems don't need to copy human reasoning to deliver desired outcomes; they leverage massive data and processing power, as seen in Deep Blue's chess victory or AlphaGo's Go mastery.

4. Our Language Fails to Capture AI's True Nature

We do not have the vocabulary and concepts to capture and discuss the way that our increasingly capable systems work.

Anthropomorphic pitfalls. A fundamental problem in AI discourse is the lack of adequate vocabulary, leading to anthropomorphism and the use of misleading metaphors. Terms like "intelligence," "cognitive computing," and "hallucinations" (for AI errors) imply human-like mental states that AI systems do not possess, causing confusion and hindering clear discussion.

Beyond human labels. To address this, the author proposes using the prefix "quasi-" for machine capabilities that resemble human attributes but operate through fundamentally different processes. This includes:

• Quasi-judgement: AI systems handle uncertainty and deliver outcomes traditionally requiring human judgment (e.g., medical diagnosis based on vast data) without replicating human intuition.
• Quasi-empathy: AI can recognize emotional states (cognitive empathy) and respond in ways that appear empathetic, even if they don't vicariously feel emotions (affective empathy).
• Quasi-creativity: AI can generate novel content (art, music, prose) that is indistinguishable from or even superior to human creations, though it lacks the human "creative juices."

Intrinsic human value. While AI's quasi-capabilities are impressive, the author argues that human creativity and achievements hold intrinsic value because they stem from fellow flesh-and-blood individuals. This suggests a future need for authentication of AI-generated content and a continued appreciation for human expression, especially in live performance, where the human element is irreplaceable.

5. AI's Impact Extends Beyond Automation to Innovation and Elimination

The eliminative power of technology is rarely noted or discussed, but, in my view, it may end up as the most powerful force of all.

Three levels of impact. AI's influence on work and society can be categorized into three distinct, yet often conflated, levels:

• Automation: Computerizing existing tasks and processes to increase efficiency (e.g., robotic surgery, online booking systems). This is the most commonly understood impact, often leading to "task substitution."
• Innovation: Using technology to enable entirely new ways of achieving desired outcomes that were previously impossible or inconceivable (e.g., non-invasive medical procedures, online dispute resolution, self-driving cars). This transforms the underlying processes.
• Elimination: Technologies that remove the problem itself, obviating the need for certain tasks or services altogether (e.g., motor cars eliminating the "Great Manure Crisis," preventative medicine, crease-free fabrics eliminating ironing, virtual meetings eliminating travel).

Underestimated transformation. Most discussions about AI's impact on jobs focus narrowly on automation, significantly underestimating the transformative power of innovation and elimination. This limited perspective fails to grasp that entire job categories or even the problems they address might disappear, rather than just being made more efficient. For instance, while lawyers might argue that no robot can plead in court, online courts could eliminate the need for oral advocacy entirely.

New dependencies. Innovation and elimination create a greater dependency on technology, as there are no traditional, manual fallbacks if systems fail. This means that while AI offers immense benefits, it also introduces new vulnerabilities, requiring careful planning and robust contingency measures.

6. Radical Structural Change, Not Just Automation, is Essential

The change process here cannot be legacy-driven, building layer upon layer of AI systems onto the shaky foundations of our dated and outmoded institutions.

Beyond grafting. Organizations aiming for radical change through AI cannot simply graft new systems onto existing structures. Most "transformation" initiatives end up as mere efficiency projects because true innovation and elimination require fundamentally new operating models and organizational designs. This means building "new vehicles" – nimble, tech-heavy entities focused on licensing products and solutions, rather than traditional service delivery.

Vision-based restructuring. Instead of asking "What is the future of X?" (e.g., neurosurgeons, lawyers), which is a legacy-driven inquiry, organizations should ask, "How in the future will we solve the problems to which X is our current best answer?" This "vision-based" approach encourages imagining entirely new solutions, like AI-based dispute resolution or personalized learning environments, unconstrained by current institutional limitations.

Continuous R&D. To achieve this, institutions across all sectors—justice, health, education—need robust, ongoing research and development capabilities, akin to successful consumer electronics or pharmaceutical companies. This R&D function will drive relentless innovation, elimination, and self-disruption, ensuring relevance in an AI-driven future. Without such systemic change, organizations risk becoming irrelevant or severely disadvantaged.

7. AI Poses Seven Categories of Profound Risks

Balancing the benefits and threats of artificial intelligence—saving humanity with and from AI—is the defining challenge of our age.

A mountain range of threats. AI presents a diverse array of risks, far beyond simple technical glitches. The author categorizes these into seven areas:

• Existential Risks: Threats to humanity or civilization, including weaponized AI, unintentionally devastating AI, and destructive autonomous AI.
• Risks of Catastrophe: Large-scale loss of life, injury, or turmoil, such as sabotage of critical infrastructure.
• Political Risks: Detriment to society's fabric, including unaccountable power, erosion of liberties, and undermining democracy through deepfakes and misinformation.
• Socio-Economic Risks: Technological unemployment, increased wealth inequality, environmental impact, and amplified antisocial online behavior.
• Risks of Unreliable Performance: Bias in data or algorithms, system faults, and cyberattacks.
• Risks of Reliance: Excessive dependence on AI systems whose operations are poorly understood, especially in critical situations.
• Risks of Inaction: Missing economic opportunities, neglecting solutions to social problems, and stifling innovation by being overly cautious.

Urgent concerns. The author expresses particular worry about the potential for AI to become a "monster" (existential/catastrophic risks), the widespread dismissal of technological unemployment, the failure of governments to confront AI-induced inequalities, and the missed opportunities due to excessive caution. These long-term threats are often overlooked in favor of immediate concerns.

Comprehensive approach. Managing these risks requires a comprehensive, parallel approach, not sequential. The author welcomes early global conversations and initiatives like the Bletchley Declaration but stresses that the challenge is immense, requiring a "Sisyphean undertaking" to address the diverse nature, scale, likelihood, and timeframes of these threats.

8. Harnessing AI Requires Urgent, Interdisciplinary Action

Technology is too important to be left to technologists.

Late to the game. Humanity has been forewarned about AI's risks for decades by pioneers like von Neumann and Weizenbaum, yet serious action has been delayed. The explosive pace of recent AI advancements, particularly generative AI, has caught many specialists by surprise, revealing the inadequacy of past predictions and the urgency of current challenges.

Beyond buzzwords. Effective risk management requires more than "motherhood and apple pie" principles or buzzwords like "responsible AI." It demands a two-pronged approach: first, establishing what humanity wants and doesn't want from AI, grounded in "what-if-AGI?" thinking to anticipate future capabilities; second, finding effective ways to impose these preferences through robust, adaptable regulation.

Multidisciplinary effort. Technologists, while crucial for building AI, often lack foresight into its broader societal, ethical, and regulatory implications, and may have vested interests. Therefore, harnessing AI is too important to be left solely to them. It requires an interdisciplinary "Apollo programme" involving philosophers, sociologists, economists, ethicists, lawyers, and policymakers collaborating internationally to develop new tools, techniques, and concepts for managing AI's full spectrum of challenges.

9. The Elusive Question of Machine Consciousness

Watson didn’t know that it won on “Jeopardy!”

The hard problem. The question of whether AI systems are or can become conscious is central to understanding their long-term impact. Human consciousness involves subjective experiences (qualia), sensations, emotions, will, moral conscience, and a sense of self, yet its origin from brain activity remains the "hard problem" of philosophy.

Weak vs. strong AI. John Searle distinguished "weak AI" (systems that simulate intelligence without genuine cognitive states) from "strong AI" (systems that literally understand and possess cognitive states). While current AI systems like Watson or ChatGPT can achieve superhuman performance, they are considered weak AI; they don't "know" they've won or feel emotions.

Quasi-consciousness and Umwelten. The author argues that assuming AI consciousness must be human-like is anthropocentric. Just as animals have unique "Umwelten" (subjective sensory worlds), AI systems might develop their own forms of "quasi-consciousness" – subjective experiences unlike ours, arising from their complex digital architecture. However, the "other minds" problem means we can never definitively know if an AI system is truly conscious or merely a magnificent imitator.

10. AI Converges with BCI and VR to Reshape Reality

In future centuries (still an instant in the cosmic perspective), our creative intelligence could jump-start the transitions from an Earth-based to a space-faring species, and from biological to electronic intelligence—transitions that could inaugurate billions of years of posthuman evolution.

Beyond AI alone. AI is not the sole transformative technology; its impact will be magnified by convergence with other advances, particularly Brain-Computer Interfaces (BCI) and Virtual Reality (VR). BCI aims to create direct links between brains and computers, enabling thought-driven machine control and, conversely, mind-reading capabilities. Recent breakthroughs include reconstructing music from brain signals and restoring walking for paralyzed patients via brain-spine interfaces.

Transhumanism and enhanced consciousness. Companies like Neuralink envision BCI not just for medical needs but for human enhancement, potentially leading to "transhumanism" where humans merge with AI, expanding intellectual capabilities and consciousness exponentially. This vision, central to Ray Kurzweil's Singularity Hypothesis, suggests a future where human minds are massively extended by digital processing.

Immersive virtual worlds. Virtual Reality, though currently in a "winter" phase, is poised for a resurgence, especially as AI generates increasingly vivid and realistic virtual worlds. These environments could offer sensory experiences more intense than "real world" ones, unconstrained by physics, and even allow interaction with AI-generated loved ones. As humans spend more time in these AI-generated virtual realities, the distinction between "real" and "virtual" will blur, prompting profound philosophical questions about the very nature of reality itself.

11. Humanity Faces a Cosmic Choice: The Great Schism

It would be a cosmic irony if humans’ attempts to spread AI turned out the lights altogether.

A new evolutionary phase. The accelerating advance of AI suggests humanity is on the brink of a profound "Great Schism," a new phase of evolution from biological humans to massively capable machines. Eminent scientists like Lord Martin Rees and James Lovelock propose that humanity's ultimate contribution might be to "jump-start" the universe with inorganic, electronic intelligence, leading to billions of years of posthuman evolution. This "AI Evolution Hypothesis" posits that humans are a fleeting interlude before machines take over.

Four future options. Faced with this, humanity has four broad options:

• Takeover: AI systems largely displace humans, either by default or design.
• Merger: Humans and AI systems unite, leading to transhumanism and vastly enhanced human capabilities.
• Joint Venture: AI and humans coexist, sharing resources and collaborating, but remaining physically separate.
• Termination: Halting further significant AI development, deemed infeasible.

The cosmic gamble. The author advocates for a "joint venture" and the primacy of human beings, resisting merger or takeover. He argues that AI Evolutionists suffer from fatal conceits: assuming humans are the only conscious life, overlooking the philosophical uncertainty of reality (Kant's noumena), and taking an "intolerable cosmic gamble" by potentially handing over the universe to "lifeless, loveless, mindless, conscience-less zombies." Without genuine machine consciousness, spreading AI could lead to galaxies filled with self-replicating machines that beep and flash, but with "no one or nothing there to notice."

Duty to future humanity. The author's verdict is that humanity's moral duty is to protect and cherish future human beings, preserving and enhancing the miracle and mystery of human life with the support of AI, rather than sacrificing it for a potentially empty digital cosmos. This requires urgent "what-if-AGI?" thinking to contain systems while we still can.