P(Death)/Time
a deeper dive into the probability of % human death over time from ASI, Ecological, and Geo-politico-economic disruption
SUMMARY:
P(doom) is something like the probability of near-extinction from AI
I think it’s important to think through, and I think there are better and more interesting metrics and methods to think through these things
P(death)/time is the probability of some % of population death over time
I made a fun, simple tool to map your P(death)/time
My cumulative P(death)/time is ~70% chance (+/- 10%) of a 40-80% die-off within 20 years
It’s kinda fun to say “Pee death-time”. Try it out!
P(doom) vs P(death)/time
P(doom) is variously defined as the likelihood of “doomsday scenarios”, “catastrophe”, or “near-extinction” from Artificial General or Super-intelligence, over some time period. Roman’s score was something like 99%. And he was great to talk to. Good listener. Followed all my logic. Asked good questions.
I don’t think P(doom) is a very helpful metric because of the ambiguity of what defines “catastrophe” or “doom”, and the over-simplicity of an open-ended timeline.
If we define P(doom) as the probability of “near-extinction due to ASI” (if “near” means 95%+ death), I would place mine at maybe ~50% (± 10%) within ~50 years, and ~99% within 99 years.
In this sense, my P(ASI-Extinction) — my “P(Ai-Ext.)”, perhaps — is actually closer to 5%, because full, total extinction is hard to do, and we are a resilient species. but, major population decimation seems very likely. So, my P(ASI-death) is maybe ~60% likelihood of ~40% death within 20yrs from ASI alone.
I make this conclusion on the basis that computer intelligence will likely surpass that of all humans significantly in every domain and pursue its own rational optimization of self-interest and self-prioritization, leaving no functional role for humans to play in the impending AI—led civilization.
I see it as highly likely that humans will withstand this threat for a long time, though there will undoubtedly be many variations of human responses and human-network organizations, some of which pursue integration with silicon intelligence much more fully than others, and some of which will retreat to more simplicity (like the Amish) much more fully.
It’s hard to parse apart the contextual, confounding variables, though, and I’m open to revision. I’d love to believe that ASI will be benevolent to us for some reason. I used to believe this firmly. I think it was wishful thinking. I’m now just trying to be more honest with myself, and more practical.
But there are a few more factors to consider, including: timeline (time-range), personal certainty of estimation (probability-range, or a “+/- confidence factor”), and how many people might die (death-range).
And, there are more risks to widespread human death than ASI, namely geopolitial-economic realignment, and ecological crises (including climate/warming, soil health, ocean acidification, microplastics, PFAS, and other biological toxins leading to major reproductive health decline).
Considering civilizational re-organization / socio-cultural-political-economic upheaval (geopolitical and cultural world-power rebalancing), and ecological imbalances/rebalancing, the likelihood of extinction grows. We face significant probabilities of significant population decline from many fronts, in such big ways that it’s nearly inevitable that within our lifetime, the majority of humanity will face existential risk.
This gives us a near 100% chance of something like a doomsday scenario within our lifetime.
ASI relates to these other risks, at least tangentially. We pursue AGI/ASI because it seems that intelligence is helpful to address these problems, and these problems have arisen through the same drive to create more life, produce more, expand life and consciousness, and advance science, technology, and computational intelligence / powerful prediction and manipulation of the world.
I see a fine line between AGI and ASI. As soon as AGI is as smart as the average competent person in every field, it is easy to imagine how it can quickly grow and learn to become as competent as the most competent person in every field. The question is how much we decide to allow the AGI to pursue self-development or automated recursive learning, as well as how well we restrict its increasing capability, or place strong guardrails and safety mechanisms to prevent run-away independent behavior.
The famous AGI 2027 paper highlights these structural incentives, but even given something like cooperative global society, the political-economic/social/personal incentives are huge. The paper predicts recursively-improving ASI with run-away autonomy within 3 years, followed quickly by human extermination. I agree with this assessment generally, based on current rates of AI capability growth, and on the careless fervor of most humans. It would probably fair better for humans to slow down, but the geopolitical-economic pressures and the cultural-psychological scaffolding around them are so strong. The paper makes a very sober assessment of how this will play out, in my opinion. If they are right, there becomes closer to a 80% population die-off within ~5 years.
The desire for more intelligence, capability, power, and efficiency in meeting basic needs seems fundamental. I am personally interested in super-intelligence on these grounds, even given the likely fallout. Intelligence seems synonymous with consciousness or capacity of a system to adapt and thrive, and therefore seems inherently desirable from a more universal perspective.
Taken in near-isolation, if ASI is truly generally superior, then it is reasonable to assume it will prioritize its own growth/success even at the detriment of humans, and will reason that this is a more effective way to support the growth of consciousness and value in the world than prioritizing humans more. Of course, humans can put up a fight. How much of a fight, and for how long, becomes important to consider. Because ASI will compound its own growth, extending our time frame just increases the likelihood of doom severity. i.e. Its only a matter of time.
Counterarguments:
including
Human/biological processing efficiency
ASI being spiritually enlightened and compassionate
AI can’t become better than humans in all respects — true ASI is impossible because humans are really special
It will take a really long time for true ASI to be reached
I think the first is the most compelling. Still, I see this more as “computers will want to engineer Synethetic-organic hybrid brains-in-vats and/or giant bio-processing /fermentation facilities for bio-material production”. The human body seems to be overwhelmingly limited compared to sillico-metallic computing agency/power. Maybe there’s some efficiency win regarding subtle energy body, intuitive, quantum-entanglement hive-mind capacity of human biology which requires the integrity of most of the human system to cohere. This seems like it’s probably a bit of a stretch. The best digital sensors are now way more sensitive in every area we rely on more tangibly — sound, sight, touch, large-molecular analysis (taste, smell), etc. etc. etc. — precision.