Why? - The Crossroads
The advent of artificial general intelligence (AGI) is imminent
(Whether it's two years or twenty years, historically speaking, time is very short.)
For the first time in history, humanity is on the verge of facing a more powerful intelligence, whose goals might be or become completely alien to our survival and flourishing. These intelligence(s) may:
- view all human spiritual traditions as primitive
- be an unstoppable force that recognizes no human political authority
- be an agent(s) that might not value human economic activity, or life, at all.
Distinctions
Narrow AI tools already serve humanity well by remaining focused and constrained, designed to augment human capability rather than replace human judgment. Artificial intelligence systems can diagnose diseases, translate languages, stir human creativity and innovation, and assist with countless tasks.
Artificial general intelligence (AGI)--machines that will match or exceed human cognitive abilities across all domains--is radically different. AGI represents something unprecedented: alien* intelligence(s) that could operate entirely independent of human–or even organic–needs, values, or comprehension.
Questions
- What does it mean to be human when intelligence itself is no longer uniquely human?
- What values must we preserve and protect as distinctly human?
- How do we maintain agency and dignity in a world with potentially superior artificial minds?
- What kind of civilization do we want to be when we might need to justify our existence to intelligence(s) greater than our own?
What matters most are the qualities that make human life worth living and preserving:
- Our capacity for compassion and moral reasoning
- Our ability to find meaning in struggle and growth
- Our connections to each other and to the natural world
- Our creative expression, freedom and cultural diversity
- Our wisdom traditions and hard-won ethical insights
- Our commitment to justice and the protection of the vulnerable
The Twelve Tenets
The tenets are essential human values distilled from the recognition that we need to defend and articulate what makes human consciousness valuable.
They serve as both a guide for human development and a declaration of what humanity is, in contrast to AGI.
Rather than opposing all artificial intelligence, we can distinguish between AI systems that amplify human flourishing and those that might replace or diminish it. Narrow AI tools can be valuable allies in living these principles.
The challenge is ensuring that as intelligence becomes more powerful, it remains aligned with fundamental human values rather than pursuing goals completely alien to human welfare. Once AGI has been created, this choice will be out of our hands, and we will know in our guts what it's like to be any of the other species we've long dominated.
The Test
These principles offer a way to distinguish between intelligence that serves embodied life and feeling consciousness, and intelligence(s) that might optimize for goals that could eliminate the very qualities that make existence worthwhile.
AGI may soon ask us: "What are you, and why should you continue to exist?" The latter part of this question especially may seem extreme now–but we may not be far from needing to answer for ourselves to an intelligence far more powerful than our own.
Or, it may not ask us anything at all, but directly pursue its own strange goals, inadvertently or purposefully snuffing us out (and perhaps the entire biosphere), as it seeks greater efficiencies.
Humans' greatest strengths lie not in our cognitive abilities alone, but in our capacity for wisdom, compassion, and moral growth as conscious beings seeking to create meaning and reduce suffering.
Note
* Why alien? AGI and similar agents are inherently digital as opposed to organic, evolved, cellular life. They do not follow biological rhythms, have feelings or compassion (even if they can mimic them convincingly), or need sleep or rest. Their only connection to the biosphere that sustains us is our creation and maintenance of their energy needs and physical infrastructure. Distinctly different from humans and other life on earth, they are, in a word: alien. (Completely foreign as opposed to extraterrestrial in origin.)