A new horseman of the apocalypse? Maybe.

If it feels like we are racing towards the end of the world, you are correct. If it feels like you have lost all hope in the future, it is completely understandable. If it feels like our technology is out of control and will kill us all, you would be right. These are some of the ideas in The AI Doc, a prescient documentary on the current state of our latest doomsday technology, artificial general intelligence (AGI).

I grew up in the age of apocalypse. In fact, if you were alive any time after the bombing of Hiroshima, you might consider that would be the start of the anthropogenic end times, where civilization has the ability to destroy itself.

It gets even better. Our ability to create civilization-ending catastrophes of our making has only improved. There are global problems so consequential that any one of them may very well end humanity: nuclear war, climate change, and the new kid on the block, AGI. AGI is the holy grail of AI development, immensely more powerful than the current AI models which are used commonly as chatbots and assistants. AGI would have agency and independence (or take it), be able to modify its own actions, and be smarter than all humanity.

Like a human treats an ant, AGI could reach a point where it no longer needs humans and can operate independently, potentially making decisions that are beyond human understanding or control. Its intelligence could mean the obsolescence of all human labour, destabilizing societies by mass manipulation of information and disruption, while concentrating power into the hands of the obscenely wealthy.

But AGI does not have to go the way of The Terminator to cause us harm. Even a benevolent super-intelligence might inadvertently destroy us, like how a kid might hold up a magnifying glass to the sun to warm up an ant hill. Human civilization is complex and delicate. The problem of aligning a new intelligence with our own is not simply a matter of managing the motivations of AGI.

Why do we make technology like this? Because technology has always improved our existence, even as it may destroy it. Both are true at the same time. And the development of existential technologies will only become more frequent. Our advancements have accelerated, coalesced, and often have global impact. As human intelligence advances, so does the power we gain from technology.

Like it or not, many of our technologies have reached their apocalyptic terminus, and AGI is on the cusp of becoming the next civilization destroyer. But it also has the potential to be immensely useful. Being able to analyze vast amounts of data has shown to improve disease diagnosis and accelerate scientific research which could lead to cures for cancer, boost human physiology, produce efficiencies in resource management, health care, finance, and support disaster management. In other words, it could be utopia instead of apocalypse.

But how do we get there? Although all-in on the development of AGI, the AI brain-trusts in the documentary welcome sensible rules on AGI to mitigate the harms that they themselves acknowledge. The arms-race to get to AGI requires that AI companies focus on getting to the finish line first and not slow themselves down by self-imposed restrictions.

This where governments need to step up.

As the world moves at an accelerated pace, not only should our technologies improve but societal responsiveness needs match the pace of change. We are moving on an exponential timeline into the future, while our governments are still using antiquated centuries-old governance models. The choice between utopia or apocalypse will be determined by how we choose to work together, one to one, country to country and globally. Right now, it is not looking great.

But I am optimistic. Just recently, courts have held social media companies responsible for the harms of their platforms. OpenAI has shut down its AI Slop platform Sora because of untenable economic viability. And Google paid a class-action settlement because its AI was inappropriately recording and spying on users’ private conversations. Pressure must be maintained to ensure that technology is used for human benefit while reducing harm.

Technology doesn’t care how we use it, but humans need to.

Humans are uniquely equipped with an intelligence seeking its own advancement, understanding and exploring the limits of our capabilities and our influence on Earth, maybe even beyond. Collectively we share both the promise and the peril of such power, and together we need to work in our best interests for the future.

I will borrow some inspirational words from Jeremy Hanson, Canada’s Artemis II astronaut before launch who said “We are going for all humanity” as humans ventured further out into space than ever before. Any consequential technology we develop should always be for, and not against, all humanity.

David Nutbean writes on science and technology.