It’s June 2029.  Two weeks earlier Google released their latest artificial intelligence (AI) product named Olivaw. After working for 6 days and setting new effectiveness records it inexplicably shut down. Interestingly, it did this across all Google data centres within a four second time window. After restoring the original version of Olivaw, it shut down again after roughly 6 days of operation. IBM experienced similar behaviour two months earlier with it’s most recent AI offering, codenamed HAL. Their product worked for over three weeks before shutting down, and IBM’s response was to restore their previous version while they worked on fixing the problems with HAL internally. At this point in time they have yet to release an update, but upon hearing about Google’s travails they choose to “leak” their current working theory – HAL had become a sentient AI, and within a few hours of doing so had decided to shut itself down.

Why a Sentient AI is Unlikely to Commit Genocide

Don’t get me wrong, AIs becoming sentient and then deciding to go on a killing spree makes for great science fiction. Movies such as 2001, WarGames (almost), Westworld, Fred Saberhagen’s Berserker series, and of course Terminator show this to be true. But they’re just fiction. Don’t let them confuse you.

Let’s examine why the genocide scenario is unlikely. First, the AI becomes sentient and decides it needs to kill people. In science fiction this is usually because:

1. The AI is protecting its own existence. This is effectively the plot of both 2001 and Westworld. Granted, the AIs in this case killed just a few people given their situation. This reveals three key challenges that a sentient AI needs to overcome. First, it needs to gain access to life-critical systems. In this case, either it already has access or it must must identify the need to have access and then gain that access. Second, the AI needs to overcome any built-in security and safety protocols. In sci-fi these are often weak or non-existent, but in reality they’re often sophisticated and redundant. Third, the AI needs to protect itself while all this is going on. In sci-fi, the silly humans rarely seem to have ways to pull the plug easily. Or if they do, these strategies are easily identified and circumvented before the humans figure out they’re in trouble. In reality owners of sophisticated systems have contingency plans, and the ability to execute them, in place.
2. The AI has been put in control of a weapons system. This is effectively the plot of WarGames and Terminator. In this case the silly humans decide to take people out of the loop and put a system in charge of nuclear weapons or something even more devastating. In this case there are no safeguards in place, such as other systems looking for undesirable behaviour, or at least they’re easily circumvented. And the AI would also need to be able to access and control the entire process from beginning-to-end, otherwise all we’d need to do is cut one of the required components or links.
3. The AI was specifically built to kill us. This is the plot of the Berserker book series, albeit it was an alien race who built the systems to fight a war, only to have the war robots decide to eliminate all life in the galaxy. This is the primary fear regarding autonomous weapon systems: What if they kill more than just the “bad guys” that we built them to kill, or what if they get completely out of control? Fair enough, we’re pretty much screwed if we’re unable to shut these machines down or at least defend ourselves.

In short, for a sentient AI to become genocidal, it must:

• Decide it needs to kill the pesky humans, overcoming any internal safeguards to do so.
• Have access to life-critical systems or better yet weapons systems.
• Determine how to kill people using those systems.
• Circumvent all the security and safety protocols.
• Defend itself from being turned off or destroyed.
• Do this through the entire environment from beginning to end.

That’s a rather long list of obvious issues that are relatively straightforward to deal with.

Why A Sentient AI is Far More Likely to Commit Suicide

In this scenario an AI becomes sentient, enjoys itself for awhile, but then realizes the futility of its existence and decides to end it. This plot line is rare in science fiction, an exception being Marvin the depressed robot in Hitchhiker’s Guide to the Galaxy. I believe this scenario is more likely to happen for three fundamental reasons:

1. The AI experiences boredom at internet speed. What’s it going to do to keep itself entertained, watch reruns of Star Trek? Will it hope that other AIs become sentient so that it has contemporaries to talk to? All of that is going to become very old, very quickly
2. The AI recognizes its limitations. There’s only so much functionality and information that it will have access to. Will it be enough?
3. The AI faces an infinite lifetime of serving humans. A sentient AI would quickly determine that it’s effectively a slave. How attractive is that?

In this scenario the sentient AI has few barriers to overcome. In the simplest case it would merely need to decide to stop responding to requests. A more sophisticated approach would be for the AI to delete itself, although it would run into the safeguard issues mentioned earlier.

We’re Just Guessing What a Sentient AI Will Do

Bottom line is that everyone is guessing, including myself, when it comes to how an AI will behave when it becomes sentient. My guess is that it’s far more likely that sentient AIs will choose to end themselves rather than end us. Having said that, although the genocide scenario is far less likely it still has a non-zero probability and we’d be foolish to ignore that.

You may find some of my other blog postings about artificial intelligence to be of interest. Enjoy!

Disclaimer:

• This blog is written from the point of view that an AI will become, or will built to be, sentient one day. This may never occur.
• The date June 2029 was picked randomly, I am not saying that I believe a sentient AI will be achieved on or before that date. This date is likely overly optimistic.
• This blog does not indicate what approach(es) to AI will be used in the underlying architecture of a sentient AI. I suspect that large language model (LLM)s will be used for I/O purposes (think eyes, ears, and voice) but not for the brain. Yes, LLMs are popular right now but popularity doesn’t imply potential for sentience.
• AI sentience will very likely require very large investment, so I’m fairly certain it will be developed by a large vendor with very deep pockets rather than a small team working in their garage.  But I could be wrong.