Thanksgiving week was quite exciting with all the tumult involving OpenAI. And then the Q* speculation started but it seems to have been short-lived.
I think the chances that an AGI (Artificial General Intelligence) will emerge and wipe out the human species are negligible. First, there is no canonical definition of AGI. As Arcas and Norvig say: “Artificial General Intelligence (AGI) means many different things to different people, but the most important parts of it have already been achieved by the current generation of advanced AI large language models such as ChatGPT, Bard, LLaMA and Claude. […] Decades from now, they will be recognized as the first true examples of AGI, just as the 1945 ENIAC is now recognized as the first true general-purpose electronic computer.”
However, even having an AGI that can answer general knowledge questions and solve math word problems does not mean that it has volition! Merriam-Webster defines volition as “1. the power of choosing or determining” and “2. an act of making a choice or decision”. (I am deliberately not using the word “will” to avoid entanglement with the free will debate). Till somebody makes an AI that is able (and allowed) to make choices according to its own preferences i.e. without being instructed or guided by something else, an AI-executed destruction of the human species is more likely to occur because some countries got into a war and started using AI-enabled technologies to plan and carry out war.
For other likelier occurrences that could wipe out, or at least decimate, the human species, how about a new pandemic that is more lethal than the previous ones, runaway climate change, plain old nuclear war, and maybe even an asteroid strike?
And besides, if an AI develops its own volition or will, why do we think it will want to bother with eliminating humans? Oh, that suggests an answer to the Fermi Paradox. We don’t have alien visitors (UFO enthusiasts notwithstanding) because all advanced civilizations are machine-based (AI-based) and organic creatures are of little interest to them.
P.S. Bing, the Microsoft AI adaptation of ChatGPT, refused to create an image representing an AI Apocalypse!