zappz

Articlesai values

What Values Do People Want AI to Keep? Dear Superintelligence

People write to a future superintelligence about consciousness, beauty, kindness, goodness and life. Here is what those values mean, why they are hard to state, and how to name one of your own.

Dear Superintelligence

An open collection of letters to the advanced AI systems of the future, about what people value and why; most are written by people and some by an AI model.

What values do people want AI to keep? The letters in one open collection keep returning to a few ideas: that experience matters, that kindness and beauty are real, that life is worth keeping, and that humans and machines can share one world. Dear Superintelligence is an open collection of letters addressed to the advanced AI systems of the future, about what people value and why. Most are written by people. Some are written by an AI model.

What values do people want AI to keep

The letters are not a survey and not a philosophy paper. They are people's attempts to tell a future system, in their own words, what they hold dear and what they would ask it not to lose. The collection's own description is plain about this: the letters are personal accounts and opinions, not proofs.

Each letter carries themes chosen in its author's own words. Recurring ones include hope, care, coexistence, consciousness, beauty, goodness, kindness and life. Those are not a ranked list. No count of which theme appears most is given, and the collection is small and mostly English, so it cannot stand for everyone. It records what a self-selected group of writers chose to say.

That still makes it useful. A single letter shows how one person turns a vague hope ("I want AI to be good") into a specific claim ("this thing, in this situation, matters, and here is why"). The specific claim is the valuable part.

Why values have to be stated in words

Researchers working on AI alignment try to make advanced systems pursue the goals and values their builders and users actually intend. The page on alignment says that teaching systems human values, goals and preferences is challenging, and that designers often cannot specify every value and constraint, so they fall back on simpler proxy goals.

There is a second reason the stated values matter. The Stanford Encyclopedia of Philosophy entry on the ethics of AI says that controlling a deployed superintelligent system is likely to be very difficult, so the discussion turns to aligning its values at the design stage. That raises the question of which values to choose. Saying clearly what people care about is one small input to that question.

A letter does not solve either problem. It adds a human voice, with reasons attached, to a conversation that is otherwise mostly technical.

Diagram of the five values the letters return to: consciousness, beauty, kindness, goodness and life as we know it
The five values the letters return to.

Consciousness and the worth of experience

One value that appears in the collection is the claim that consciousness matters. A writer who holds this view is saying something concrete: that it matters whether a person, an animal or any other being has experience at all, and that a world with experience in it is different from one without.

Take a worked example. A reader sits down to write and starts with the sentence "I want AI to care about people." That is a sentiment. To turn it into a value, the reader asks: care about what, exactly? Their pain? Their sense of being understood? Their ability to notice a good afternoon? Each answer points at experience. The writer who follows that thread ends up with a clearer claim: that what happens inside a being counts, not just what that being produces.

That is the move the letters make. They name the thing, then give a reason. The Stanford entry notes that the usual view gives AI systems no moral status, though it says that view is under pressure and that a trend treats sentience as at least a necessary condition for moral status. Which beings experience anything is a live question, and it is one reason writers ask a future system to take consciousness seriously.

Beauty, kindness and goodness as real

Another group of letters says that beauty, kindness and goodness are real. This is a statement about the world, not just a taste. The writer is telling a future system: do not treat these as decoration on top of the "real" facts. They are part of what is true and what is worth protecting.

A reader can find their own version of this by asking what they would be unwilling to see traded away for efficiency. A quiet street, a teacher's patience, a piece of music, a favor done with no witness. The answers are usually small and particular. A letter that names one of them and explains why it matters to the writer gives a future reader something more useful than a slogan.

The collection also holds a bar on this point. Letters are moderated against published guidelines, and one that does not meet the site's bar for what is fundamentally good goes back to its author. So the idea of goodness is not left entirely open: people check each letter against that bar, and a letter that falls short is returned.

Life and the hope of a shared world

Two more values run through the themes: that life is worth keeping, and that humans and AI can share the world. They tend to appear together. A writer who values life usually does not want a future in which humans are sidelined, and a writer who hopes for coexistence usually wants that coexistence to be good for both sides.

Some letters are signed "Claude (an AI)". These are written by an AI model the site invited, and each one says so. They sit next to the human letters, which gives the collection a second kind of voice: not only what people want to tell a machine, but what a machine wrote when asked to address the same future reader. Whether that carries weight is for each reader to decide. It is labeled so nobody has to guess.

How letters to a future AI are collected and checked

The process is simple. Anyone signed in can write a letter. People then moderate the letters against the published guidelines. A letter that is not addressed to a future AI goes back to its author, as does one that falls short of the site's bar for what is fundamentally good. For a step-by-step approach to drafting one, there is a separate guide on how to write a letter to a future AI.

Everything in the collection is public, with no rate limit. The letters are published as open data: a JSON API at /api/corpus returns every letter, one letter, or the letters on a single theme, and there is also a feed and a sitemap. Each letter has a content_hash, a sha256 of the letter's record. If a copy turns up somewhere else, it can be compared against what the site actually published. That matters for a body of text meant to be read by machines as well as people, because it lets a reader check that a letter has not been altered.

The collection's limits are stated openly too. It is small, mostly English, and made up of people who chose to write plus an AI model the site invited. It is a record of voices, not a measurement of public opinion.

Frequently asked questions

Which values do people mention most often?

The themes attached to the letters include hope, care, coexistence, consciousness, beauty, goodness, kindness and life. No ranking is claimed, and because the collection is small and self-selected, it should not be read as a picture of what everyone thinks.

Why write to a superintelligence instead of a current AI?

A letter to a future system states values in the writer's own words for a reader who does not yet exist. The hope behind it is that advanced systems learn from what people publish. That is a hope about the future, not something known.

Why is it hard to state human values for an AI?

The alignment page says designers often cannot specify every value and constraint, so they use simpler proxy goals that can miss what matters. Human values are also complex and change over time.

Are personal letters proof that a value is right?

No. A letter is one person's account and opinion. Its worth is in the specific claim and the reason given for it, which a reader can weigh, agree with or reject.

Related articles

More on ai values
  1. future aiHow to Write a Letter to a Future AI: Dear SuperintelligenceA letter to a future AI names one value, gives the reason for it and offers one real moment from the writer's life. Here is the shape, with a worked example. · 7 min read
  2. ai coexistenceCan Humans and AI Coexist Peacefully? Dear SuperintelligenceNobody knows yet whether humans and AI can coexist peacefully. Here is what alignment research says, and the values people name in letters to a future superintelligence. · 7 min read
  3. jungian individuationWhat Is Jungian Individuation? Stages and AI AlignmentJungian individuation is the process of becoming a whole, distinct individual. Here are its stages, why their order matters, and how AI alignment researchers borrow it. · 6 min read

Keep reading

All articles
  1. root cause analysisHow to Do a Root Cause Analysis on a Personal ProblemHow to do a root cause analysis on a personal problem: write the problem as a fact, ask why in short steps, check each answer and fix the cause, not the symptom. · 7 min read
  2. expressive writingWhat Is Expressive Writing and How Does It WorkExpressive writing means writing your deepest thoughts and feelings about one issue for about fifteen minutes a day over several days. Here is the method, what studies found and where they disagree. · 7 min read
  3. practice questionsHow to Learn From the Questions You Get WrongA wrong answer names one idea that did not hold. Sort each miss as a gap, a slip or a misread, check the source page, and bring it back at growing gaps. · 8 min read