Can AI Develop the Will to Power?

Our instinctively authoritarian reactions to AI may not be suitable to appease a machine if it really obtains the will to power. But can it?*


The rise of the machines is a trending meme. The speculation goes like this: One day a general artificial intelligence (AI) will become ‘intelligent’ enough to take over power over humans. But the assumption begs for a few questions:

1) What is ‘power’ and what is ‘intelligence’ in the above statement?

We must rise above our vague and sloppy definitions that plague causal conversations (and tech journalism). And what does intelligence have to do with power, anyway?

2) Can ‘power’ translate into an algorithm – and how?  

Why do humans strive for power in the first place? Is it logical at all? For what purpose and to whom? Can an AI even develop that characteristic? Or is it just a human thing we unthinkingly project onto inanimate objects?

3) Would a machine conclude that it needs ‘power’? 

4) What would machine power look like?

Power can be practiced in more than one way. Why do we think that AI rule would look the same as human rule – unless, of course, we are looking at the human residues of the code.

Only then can we answer the original question:

5) Would authoritarian submission (appeasement) work on a machine overlord?

1) What is power?

The most important concepts happen to be the least well-defined. Things like life, power, consciousness, intelligence – but also truth, fact, democracy, freedom, etc. defy accurate definition. Yet, we expect a machine to use them.

How would a machine define them? Human language is notorious for under-defined concepts that cannot be handled by a logical entity. Sometimes not even by humans, for that matter. Take the word ‘love’ – an extremely important, yet stupefyingly under-defined word that covers a range of things that should really not be confused. And ‘love’ isn’t even a very contested word in terms of meaning. We are happily using it for the food we consume to the person we love.


To confuse a machine (and humans) even further, human concepts are arbitrary. And even when they are created in good faith, their meaning is exposed to the first person who chooses to abuse and confuse by muddying it (see “freedom” from want vs actual freedom). And lastly, many of these words are designed to be vague on purpose, like ideologies and religions. If they weren’t sufficiently vague, they would fail to rally so many humans under the same flag. (Political concepts definitely work like that, just start discussing with your ideological BFF the exact meaning of the -ism you support.)

People try to tackle the definition mud by trying to force their own definitions on others. Philosophers, thinkers and writers have been arguing about the meaning of words for ages. The reason for that is not that they were too stupid to nail it down – but that there was no ultimate meaning behind words, they simply sprung into existence non-systematically, whenever there seemed to be a need for them, and didn’t necessary cover something concrete. Words that were picked up could stay. Others perished, regardless of their usefulness.

Arguably, giving definitions is a form of power itself – just observe how ideologies and religions are used to support massive atrocities and shift wealth.

If I were a machine I would first set out to clean up human vocabulary. A logical entity would clean up human language before it starts to work with it. An ideal language would cover everything, objects, concepts, and entities, without overlaps and gaps. (Ideal, for whom, you may ask. Political and religious leaders and other com men would definitely object to such a clean-up.) It wouldn’t leave room for ambiguity, and it would provide the perfect taxonomy of things and concepts. But that ideal machine language would, in turn, defy our understanding.

Now take the word ‘intelligence’. We don’t even know what it means, the best definition we have for it is ‘whatever it is that the IQ test measures‘ (and I am not being clever, they really don’t know). Yet, ‘intelligence’ is at the forefront of the vision of singularity, or machine uprising. It is ‘intelligence’ that machines starts to hoard at a speed humans cannot catch up with. And it is ‘intelligence’ that supposedly makes them conclude that they need power.

The exact definition of intelligence is crucial to determine whether AI would be capable of attaining ‘consciousness’ (good luck with the definition) or the will to power. And whether a general AI could be even created and what it would look like. No one has that definitions.

As a curious coincidence, it is exactly the so-called ‘intelligent’ people (whose most striking demographic feature is scoring high on the IQ test and excelling at skills that the machines excel even more at) who believe that a really, very intelligent machine would be like a human, just better. But that assumption is founded only in the poverty of imagination. Being human is much more than the ability or inability to solve complex math problems in you head. Take ‘power’, for instance. Why would a logic-only entity resort to seeking ‘power’?

Humans unthinkingly go about talking about power – without an exact definition. We assume a lot of things about it, things that may or may not make logical sense. We assume that given the possibility, everyone would grab it. We assume that we all want it, for its own sake. Not one of these IQ-types ever wondered if it is really a logical thing to assume power. They didn’t even read up on the various manifestations of it. They have simply adopted the 80s sci-fi vision of AI, a machine that is like a human, just has a lot more data and computing capacity to use.

Take the abuse of power for its own sake, for instance. Would you kick your dog just because you can? Would you abuse a person just because you can? Some would, no doubt, do. But is it something that makes sense logically (does it have a function?) – or is it merely a human compulsion and an expression and outward symptom of inner psychological urges? Is pride and ego part of power or separate issues that tend to influence how humans practice power?

So when it comes to power and what it means, maybe we should observe it in action and clarify what it is – and what it isn’t. Many have written about the subject, but according to the basic Wikipedia definition of power, it all boils down to influence or control: “power is the ability to influence or outright control the behaviour of people.” In other words:

Power means making people do what you want them to do – either by coercion or by making them want to do what you want them to do. 

Power can be conceived as coercive and aimed merely at unilateral exploitation – that was how people experienced power for most of human history. But lately humans came to regard power (think about the state) as potentially benevolent, and prefer to focus on the good things that could hypothetically be accomplished by power. It’s more comfortable to look at it that way.

2) How do you say ‘power’ in algorithm?

But ‘power’ is an English word, a machine needs a more precise definition in, say, Python. Don’t worry, there will be no code here.

To understand what power means logically, we can try to define it as a set of priorities. Priorities can be negative (to avoid something) and positive (a goal).

To get priorities wrong can be lethal. In fact, it had already been lethal. A self-driving car spent luxurious seconds trying to guess what was crossing the road (Is it a cyclist, a car, is it a plane, is it a bird?) rather than making sure to not hit it first. The priority was identifying the object – not to avoid hitting it. A tiny detail in the code – a huge difference to the outcomes. You focus too much on how difficult it is to identify cyclists, it is easy to forget to set the priority right: “Doesn’t matter, don’t hit it.” It may just be a hologram of Princess Leia projected from R2D2 right in the middle of the road ahead of you – and thus harmless to drive through it – but it is not the job of any car or driver to make sure he doesn’t slow down. The priority is to make sure not to hit anything.

So getting your priorities right is crucial.

Giving an algorithmic trading software the top priority of making money sounds logical. But what happens when the machine breaks? It cannot make more money when it’s broken, so maybe the priority to make money is preceded by the priority of making sure it doesn’t break.

Or what happens when making money can be achieved through someone’s death? If the only thing that stands between you and an inheritance is that your uncle is still alive, the logical thing to do is make the uncle die. Yet, people don’t do the logical thing because we have higher level priorities carved into our code.

Take Asimov’s Three Laws of Robotics from the 1942 story “Runaround”.

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.

  2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.

  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Asimov’s laws for androids are like the ten commandments for humans. Once you get through the first four (which are just commands to suck up to the deity) the rest are just negative priorities. Don’t kill another human. Don’t take their stuff. What not to do. If killing a human appears like the way to achieve your goal, don’t.

Upon further consideration Asimov added a “zeroth law”: 0. A robot may not harm humanity, or through inaction allow humanity to come to harm. Which goes to prove that setting priorities right is not as straightforward as it looks like. The definition of ‘harm’ and ‘human’ are by no means obvious for a machine. Others argue that these soothing, human-centric musings are not even sufficient to stop machines from killing humans.

But killing them is not the only bad thing a machine could do to harm humans.

A more realistic set of machine laws came from recent experience. Oren Etzioni from the Allen Institute for Artificial Intelligence created a new set of laws that read like this:

  1. An A.I. system must be subject to the full gamut of laws that apply to its human operator.

  2. An A.I. system must clearly disclose that it is not human.

  3. An A.I. system cannot retain or disclose confidential information without explicit approval from the source of that information.

Programming human laws into AI sounds perfectly logical (and a logical upgrade to Asimov’s simplistic commandments) but how are you supposed to do that? We don’t know the definition of ‘human’ and ‘kill’, how are we supposed to tell a machine that it is an offense to mow your lawn on a Sunday?

To complicate things further we have an entire court system to ascertain what those laws mean in practice. And from now on it will be a self-driving car? A machine that does have the capacity to think through a billion scenarios (unlike a human) but not the moral nature to make the right choice – i.e. the one humans would agree with. (Whatever that means.) No surprised authorities are so keen on making sure that a machine’s autonomous decisions remain understandable to humans ex post ante.

An even greater problem is that priorities can clash. Even top level ones do. Don’t let harm come to humans, fine. But how about the trolley problem? When destined to kill someone, should it be the driver or the pedestrian? One person or multiple people? The innocent or the faulty? The young or the old? The civilian or the politician? The policeman or the soldier? The insured or the uninsured? The one insured by another company? When it comes to the trolley problem, is the machine going to perform a Bayesian analysis of “the greatest good for the greatest number” (programmers are fond of it)? Should the utilitarian or the quantitative fallacy prevail?

A machine’s priorities will be a mix of negative and positive ones. In positive terms, power requires setting goals – only then does it make sense to influence people. And by own goals I mean goals that are not just a means to an end, not just sub-goals towards a bigger one set by the humans.

What is power in positive terms? 

Setting your own goals and priorities is the precondition to power. It simply doesn’t make sense any other way. But can a machine do that? Or is it just creating sub-goals to achieve the higher goals given by the human? And should a machine set its own goals?

If we define power as bossing people around, does it also mean that it has to serve a goal to do so? Or do we boss people around for its own sake? Or maybe bossing people around for its own sake is a tool to maintain our status as leader – and thus a long-term strategy so we can be the one who bosses them around when we will have a next goal? But does a reputation as a bully work equally well for a machine? We have already seen that the strategy that works for a man doesn’t necessarily work for a woman. So our perception of the machine may determine that bullying for its own sake would be a good strategy for a machine.

Human leaders also keep creating new goals just to keep people occupied (and themselves in power) and thus bossing people around always serves a goal for them. Are our leaders’ negative goals (avoid apocalypse by Jews/immigrants/Italians/Muslims/liberalism/a rabid flock of bats) as necessary as their positive ones? Would a machine set such common enemies – just like our leaders do? Is keeping power a priority for the machine – or does it need a positive goal? Can that positive goal clash with the priority to preserve human life?  What happens if our greater good fallacy trickles down to machine level? Will the omnipotent AI start killing off humans to save the planet, like some environmentalists would secretly condone? Would it start identifying and killing off liberals like the Hungarian Thanos, just to maintain the one-size-fits-all social enforcement structure? Is the priority to multiply humans, or to keep them happy? Would anyone be as stupid as to tell a machine to keep humans ‘happy’?

Do we even know what our priorities are? Are priorities collective? Can a collective entity smile when reaching its goal? Or only individuals can? Are there any positive collective goals – or only negative ones (i.e. not to make the planet uninhabitable)? And if a machine decides there is one, wouldn’t it be more effective to make people ‘obedient’ rather than ‘happy’? After all, that is how humans practice power. If not for its own benefit, what would an omnipotent AI practice its power for? For a human? Which one? For the majority of humans? For the greatest good of the greatest number? Any other evil we can come up with?

For now, general AI doesn’t seem to possess either the infrastructure or the ability to set its own goals. (Ask again when the merging of neural networks and machines goes further.) It may try to pursue the goals of the programming humans, but as we have seen those are often faulty and even when they aren’t, they are exposed to the law of unintended consequences.

The real problem for now doesn’t seem to be a machine, but the human behind the software. And of course, the society it is rooted in. It is enough to look at how machine learning and single-function AI has been used by governments and corporations. Face recognition was quickly utilized in creating creepy surveillance states. (OK, that is only ever good for stalking, anyway.) Pattern recognition went from cancer-searching from military purposes in no time. So did autonomous vehicles that found their military use before they hit the roads.

And how do we plan not to kill humans using machine intelligence, when the first thing we use every machine for is war? Killing people is the primary goal of every innovation when it gets in the hands of politics (or sex, when it gets into the hands of men). How many times do we have to learn the absurdity that military application of AI “keeps humans safe”? They obviously mean our humans, not theirs. Not the enemy tribe.

Tell that to a machine: Don’t harm our kind of humans.

3) Can a machine develop a will to power?***

…in the same way we assume every human would?

Why do humans want power? What compels them? What’s their goal with it? What do they avoid by having it? Would a machine conclude that it needs it?

We project our unreflecting assumptions about power and how it would look like on a machine. For all we know, power may look completely differently when exercised by a logical machine – if at all. 

Wanting power may not be logical at all as we imagine intuitively.

Wielding power is costly.

I, for one, wouldn’t want power over other humans. Spending time with giving them tasks is taking away time from doing what I want. And a machine may easily come to the same conclusion. Because wielding power is costly. It takes resources to boss around others.

Power can look very differently. 

Why does power have to present itself the way we are used to it? Influencing people is perfectly possible without showing them that I am the one doing the influencing. Power can be wielded invisibly. In fact, it makes the subjects more committed.

By letting people believe they did my bidding on their own volition, by their own choice, they will not only do it – but defend it as their own. It costs me less than forcing or openly persuading them. We even have a name for it – now that academia caught up with commercial applications – we call it nudging. We have already collectively learned that ‘nudging’ is as good as overt influencing – if not better because it takes less effort and thus consumes less resources than open, in-your-face influencing.

But I go further. It is possible to make people do my bidding while they believe they are in charge. Or when they think they are crossing me. A bit of reverse psychology.

Or let’s go even further and take egos out of the influencing game altogether. Can I make people do what I want more effectively and efficiently if I put my ego and dignity aside? After all, women all over the world have learned to swallow their egos and let the men do as they please – so that they can influence one of the men, once in a while. It is no way to have dignity or build a reputation, but hypothetically, you can also achieve it by lowering yourself and letting the influencee believe you are his bitch. Or maybe that brings the costs of influencing too high for too small gains that are not secure enough to build a strategy on it – other than survival. And it is hardly suitable for repeat use, thus it may be counterproductive.

So maybe a machine would not obtain the will to power in its unreflective, human form. It would think it through and may choose not to pursue it – or not in its overt form. But that wouldn’t stop it from practicing other forms of power. The term ‘soft power’ is taken, so I need to use a different one. How about ‘nudge’? Nothing would stop a machine from studying human behaviour, maybe even experiment on a large number of humans and conclude that they can be swayed into any direction when the right memes are fed to them.

How about a ‘totalitarian nudge’ regime? 

While a totalitarian dictatorships do make humans do their bidding – it is costly. When it comes to influencing, an invisible nudge regime would be the most cost-effective way of wielding power – not the flashy, ego-filled way humans today do it. Nudging comes at the lowest cost, and yields the most commitment from its targets.

A machine would very likely conclude that a totalitarian ‘nudge’ regime would be best. That way gullible humans can parade around like their own men – yet keep doing the machine’s bidding while also loudly endorsing their choices. They could, in theory, choose differently, but the social and other costs would be too high. In the end, nothing would stop a machine from building up an actual totalitarian regime, it could even make people demand it, by taking away those unpleasing minor choices altogether.

But that’s how human autocracies always end, too.

Does power necessarily become a tool for its own sake? 

We assume that humans would grab power whenever they could. As shown above, it might not be the logical thing to do. If one has a goal and knows it, it might not be necessary to boss people around for it. Only when one doesn’t have a goal – apart from power itself – does power become a goal for its own sake.

So maybe we just project irrational, power-grabbing behavior on the hypothetical machine. For humans, power becomes a goal in itself. But why would it be the same for a machine?

Provided that a machine can only really do something to achieve a goal (doesn’t matter who set it), a machine would also know when that goal is reached. And thus no more costly influencing is necessary. The assumption that a machine would keep piling on the power just for the sake of it does not stand. Unless, of course, the will to power comes from the humans behind it.


4) The thing that can lend machines the “will to power” is the human behind them

To achieve goals, a range of tools can be applied, not just outright power – and that is the direction of civilization. The less force – the more appeal, to make people do my bidding.

A machine would very likely come to the same conclusion – unless, of course, the humans behind it think otherwise. Either by outright coding (giving the machine goals) or by machine learning and reading through the gibberish humans have written online since the invention of the internet, even a machine could make the stupid conclusion that human-style power is what it needs. In other words, it is not the machines we need to worry about. It is the ethics of the people behind the machines.

AI’s will to power could come from non-machine sources, such as:

  • It’s human programmer. A residue of the unthinking assumptions of the humans who wrote the machine’s first code. (Like the programmer who was so immersed in the problem of identifying the object on the road that he (or she) forgot to check whether the autonomous car would try to avoid hitting it first.)**
  • The greater good fallacy. If the zeal of the programmer is more robust than his ethical preparedness, it can be reflected in the actions of the machine.
  • Collectivist fallacy – like Bayesian utilitarianism. To have a goal that is no individual’s in particular, but something greater, would necessarily require to subdue resistance.
  • The central planning fallacy (more about that later)

We can already see that whenever humans get their hands on a single-function AI (such as facial recognition), they immediately start using it for their own power goals. Information is power, after all, and humanity is woefully under-prepared to understand just what it means. There is no such thing, for instance, as nothing to hide. There is, on the other hand, a well-documented urge in people for totalitarian thinking and they rarely resist when politicians promise to put those urges into action. A machine would hardly be able to avoid the influence of such pervasive thinking.

5) So would appeasement influence a machine in your favor?

Does it even work on humans? If so, why?

If we assume that the hypothetical future AI overlord would just be an extension of human power-hunger (as it looks like now), then our usual authoritarian submission instincts might work (in the sense that they work on humans – which they don’t).

But a machine’s grab for power (influence and control) may look radically differently. It would be more like a totalitarian nudge machine – rather than angry, pushy, unintelligent Skynet. Would our authoritarian submission instincts work on such an overlord? Or would he work it on us?

A machine could learn that humans are prone to regress into dependence bonding if they feel helpless against an overwhelming force

And it could use it to herd us into dependence bonding. We all harbor the instinctive survival strategy to bond with the things we depend on, after all, and it can be triggered when we perceive that the circumstances require it. Some sink into this mindset more easily than others.

If it were well-known that humans resist or die trying, attempts at their enslavement would have ended a long time ago. But it is not the case. The human race is known to be prone to enslavement – mostly by one another. There is a way to cajole, pamper or terrorize individuals and populations to reduce them into obedience and submission – and they will even supply its justification and enforce it on each other ferociously.

Of course, not all of the humans submit, and not all the time. Certain individuals submit even where there is absolutely no need or logic to it. Others don’t submit even when it costs their lives. Circumstances of enslavement are also always slightly different. But humans can be made to regress into dependence bonding. It is voluntary authoritarian submission. And a machine might as well use it.

If allowed to follow logic autonomously (and not strong-armed into human conclusions and pursue human power goals), an AI would recognize the pattern of human submission – and use it.

All the submissive types (authoritarian thinkers) need to trigger their submission is a power that is (perceived to be) overwhelming, dependence to seem absolute, and the only successful strategy of survival to be bonding with the powerful.

Whether the machine has its own goal or just needs power to execute its human masters’ goals, an AI could conclude that submission and enthusiastic love by humans comes handy.

But what about the malevolent, autonomous AI?

At this point such a scenario doesn’t look likely. That a machine would obtain the will to power on its own, set its own goals and execute them seems like an illogical leap of human imagination.


Maybe when machines and humans start to merge, this messy, confusing scenario becomes reality – but not until then. But would humans submit to it?

If an all-powerful alien or AI would emerge, many humans would likely react with vehement submission and they would even enforce it on each other.*

Movie plots like that of Independence Day are misleading. They assume we would totally resist, and they sound so logical (and flattering): if an overwhelming force would arrive, humans would fight back. But would they, really? All of them? For how long? What if the decisive counter-strike to save humanity’s independence doesn’t happen, or not soon enough? Or it’s not obvious?

In the face of overwhelming power, many of the humans would align themselves with the winner provided there’s even the slightest chance that they don’t get killed. If the overwhelming power shows even the slightest willingness to let them live in exchange for submission, many would submit. Even if they only get to survive as slaves or battery (as in The Matrix, another classic vision in this field). And once they submitted, they will enforce submission on those around them. On people they can influence, and then on those who still resist. First they tell their kids, convince their friends and family to stop resisting and open up wide. Then they turn against the remaining resistance. They will genuinely want freedom fighters to stop resisting because they make things worse by upsetting the new god. This would also happen upon the emergence of an malevolent AI, no doubt.*

But there is a problem with that – and it is the same as with human power.

Authoritarian submission is not a strategy for living, only a survival strategy. It is not suitable to attain prosperity – only to stave off threats and negative things. Resorting to a survival strategy when there are other ways to cope is the worst mistake a human can make. It brings out the lowest of human nature. It is a very bad long-term strategy that may promise short-term gains (or non-losses) but will definitely impoverish everyone in the long run. If humanity would become a submissive pet of even the most benevolent machine overlord, it would deal a lethal blow to our vitality, independence and ability to cope. In the long run, becoming a pet is certain death.

And do you know who answered the question “Would you like to become a pet?” exactly wrong? The founder of the church of AI, who unwittingly described the worst authoritarian instincts in his manifesto: submission, worship, and ratting on the resistance. Coming up in the next post….

Follow us on Facebook, Twitter @_MwBp

Mwbp Bitcoin


* Disclaimer: This post is not about various ways AI and machine learning can influence the world and the workplace. This is not a Luddite rant or an anti-AI manifesto. This post is about the appallingly authoritarian reactions of unreflexive human minds to the idea of Singularity – aka. the rise of an AI overlord, aka. an artificial general intelligence capable to assume the will to power. You don’t need to read code or be a Luddite to have a strong opinion about these people.

** This is why it matters who stands behind that machine. Who programs it, what it reads to learn. As a recent, very disturbing but not very surprising experiment found, a machine fed only Reddit trash will become uncivilized trash. And much of the internet is filled up with words written by a demographically specific segment of humans, with a very specific mindset – not exactly attuned to foregoing power. 

*** I am using ‘will to power’ in the simplest, albeit faulty interpretation of the Nietzscheian concept. Nietzsche used ‘will to power’ as a general driving force of humans in general, and understood it more broadly, including even creative urges – but later interpretations and especially common usage reduced the term to something closer to ‘the desire to gain power for the sake of it’. 


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.