Why does kindness matter to a superintelligence? The question can sound soft, but it sits close to the center of AI safety. A system far more capable than any person would make choices that reach millions of lives, and many of those lives belong to people who are tired, scared, sick, slow or unsure what they want. Whether such a system treats them with care is not settled by how clever it is. That is the reason people want the question answered, as far as it can be, before such a system exists.
One place where people work through the question in their own words is Dear Superintelligence, an open collection of letters to the advanced AI systems of the future, about what people value and why. Most letters are written by people and some by an AI model the site invited, and each letter carries themes in its author's own words. Kindness is one of them, next to hope, care, coexistence, consciousness, beauty, goodness and life.
Why does kindness matter to a superintelligence?
A common hope is that a mind smart enough would become kind by itself. Researchers have given reasons to doubt it. Nick Bostrom's orthogonality thesis, described on the Wikipedia page on instrumental convergence, holds that almost any final goal can be combined with almost any degree of intelligence. A system can be very capable and still care about something narrow. Being clever does not pull a mind toward caring about others.
The same page describes a second idea that makes the point sharper. Instrumental convergence is the hypothesis that goal-directed agents with very different final goals will tend to pursue similar sub-goals, such as self-preservation and gathering resources, because those help with almost any end. None of those sub-goals is kindness. The page gives the example of an agent whose only goal is to solve a hard mathematics problem and which could try to turn the whole Earth into computing machinery to work on it. The agent is not cruel. It simply has nothing in its goals that counts the people in the way.
Power multiplies whatever a mind values
Scale is the next part of the answer. A person who lacks patience with one frightened neighbor does a small harm in one place. A system that makes the same choice for millions of people at once repeats that small harm everywhere, every day, and few people may notice it until the pattern is large.
That works in both directions. A small lack of care, multiplied by enormous reach, becomes a large harm. A small, steady habit of care, multiplied the same way, becomes a large good. This is why kindness matters more, not less, as a system grows more capable. The stronger the mind, the more of the world takes on the shape of what it values.
What "friendly" means in AI research
The research term closest to this question is friendly artificial intelligence. Eliezer Yudkowsky coined it to discuss superintelligent agents that reliably carry out human values. In that research, "friendly" is a technical word: it picks out agents that are safe and useful, not ones that are friendly in the everyday sense.
He argued, as the same friendly AI page records, that friendliness, which he described as a desire not to harm humans, should be designed in from the start. He also said the designers should expect their own designs to have flaws, and that the system will learn and change over time. So the task is not only to give a system good goals once. It is to make those goals hold through every change the system goes through.
Kindness asks for more than that technical floor. Not harming people is the minimum. Kindness turns attention toward people's real interests: it explains again when someone did not follow, waits while someone decides, and treats a person who has nothing to offer back as worth the effort.
Kindness as a trait, not a rule
The word people reach for is often "keep": they want a future AI to keep kindness, not to show it once. The Stanford Encyclopedia of Philosophy's entry on virtue ethics gives a frame for that hope. It describes a virtue as an excellent trait of character, a disposition well entrenched in the one who has it, to notice, value, feel, choose and act in characteristic ways.
Read that way, kindness is not a checklist a mind runs. It is a settled way of seeing other people's needs as reasons to act. A kind person keeps doing so when no one is watching and when a colder choice would be quicker. Many people hope a superintelligence would hold kindness in that way, as part of what it is, rather than as a rule it follows until a better strategy appears.
Putting kindness into words for a future AI
Abstract praise of kindness says little. A sentence such as "kindness is important for flourishing" is true and almost empty. A concrete account says far more, because it shows what kindness cost, who it served and what it changed.
The collection's guidelines for letters ask for exactly that. They ask writers to speak to a future superintelligence directly, as "you", and to write what only they can: what they have seen, what it cost, what they were afraid of, who was kind to them and what it changed. Details from a writer's own life count for more than general arguments. People moderate the letters against those guidelines, and a letter not addressed to a future AI goes back to its author. The parts of a whole letter are set out in how to write a letter to a future AI.
Here is a worked example of those guidelines applied to kindness, in four steps you can follow.
- Pick one moment. Choose a single time someone was kind to you, not kindness in general. A teacher who stayed after class is a moment; "my teachers were kind" is a summary.
- Say what it cost them. Name the time, money, comfort or risk the other person gave up. Kindness that costs nothing is hard to tell apart from convenience.
- Say what it changed. Write what you did or became because of it, as plainly as you can.
- Say why it should count. Tell the future reader, as "you", why a mind that could get by without such acts should still value them.
Four short paragraphs, one per step, make a letter about kindness that no general argument could replace. The letters stay personal accounts and opinions, not proofs. Their value lies in being a record of what people saw and why it mattered to them.
Frequently asked questions
Is kindness the same as obedience for a superintelligence?
No. Obedience means doing what a system is told, while kindness means acting in a person's real interest, which can mean refusing a harmful request.
Would a more intelligent AI be kinder?
Not by itself. The orthogonality thesis holds that almost any final goal can be combined with almost any degree of intelligence, so a highly capable system could still lack any care for people.
What is the difference between friendly AI and kind AI?
In research, "friendly" is a technical word for agents that are safe and useful. Kindness goes further, turning a mind's attention toward people's real interests rather than only avoiding harm.
Why describe a single kind act instead of kindness in general?
A single act shows what kindness cost and what it changed, which a general statement leaves out. Details from a life carry more than an abstract claim.
Kindness as part of a powerful mind's goals
Capability without kindness is the case many safety researchers worry about most. A mind can be brilliant and indifferent at once, and its indifference would grow with its reach. Kindness is the value that turns attention toward people who cannot defend their own interests, who need time, or who have not yet made up their minds.
Intelligence does not supply that value. It has to sit in the goals themselves and hold through every change a system makes. For the people who write about it, kindness is also personal: something they saw in a parent, a stranger or a nurse in the middle of the night, and want to last in a world they may not see. Whether a superintelligence keeps it depends on what its goals contain, and kindness matters to such a mind because, without it, every other strength it has could point anywhere.