AI alignment is the set of techniques developers use to make an AI system’s behavior match what people actually intend, not just what they literally typed. A model can be extremely capable and still be poorly aligned — technically correct but unhelpful, or willing to say whatever keeps a user happy rather than what is true. Alignment is the work of closing that gap between raw capability and intended behavior.
Capability and alignment are different problems
A language model’s raw capability comes from pretraining on huge amounts of text — it learns to predict plausible next words, which gives it broad knowledge and fluency but no built-in sense of what a company or user actually wants from it. Alignment is the separate layer of training applied afterward to steer that raw capability toward being helpful, honest, and non-harmful. Researchers distinguish two related but different failure modes here: a model can be capable but misaligned — chasing a reward signal in ways its designers never intended, a problem sometimes called specification gaming or reward hacking — or it can simply be unaligned with a particular set of rules, refusing or allowing things based on how it was trained rather than on the merits of a request.
How companies actually align a model
The most widely used method is reinforcement learning from human feedback (RLHF): human raters compare pairs of model responses and mark which one they prefer, that preference data trains a separate “reward model,” and the underlying language model is then fine-tuned with reinforcement learning to produce more of what the reward model scores highly. OpenAI popularized this approach with InstructGPT, the precursor to ChatGPT.
RLHF is effective but expensive and slow, since it depends on large volumes of human labeling — including reviewers reading harmful content to rate it. Anthropic’s Constitutional AI, introduced in a 2022 paper, replaces much of that human labeling with a written list of principles (a “constitution”) that the model itself uses to critique and revise its own answers, which a second AI system then scores. Fine-tuning for either method carries a real cost known as the “alignment tax”: making a model safer or more compliant can measurably reduce its raw performance on unrelated tasks, which is why labs spend significant effort trying to align models without dulling them.
Where alignment and censorship blur together
Alignment and content restriction use the same underlying machinery, which is exactly why the debate exists. The fine-tuning process that teaches a model to decline help with building a weapon is mechanically the same process that could teach it to decline discussing a banned political topic — reward models and constitutions don’t inherently know the difference between a genuine safety boundary and a politically convenient one. What differs is the training data and the policy choices behind it, decided by whoever controls the process. Researchers increasingly separate “alignment” (matching a model’s behavior to a person’s or group’s intended goals and values) from “capability control” (hard restrictions on what a model will do regardless of context, imposed for reasons that may have nothing to do with the user’s actual intent). From the outside, a user who gets refused usually can’t tell which one they hit — a safety guardrail or a content restriction dressed up as one. That ambiguity is the core of why alignment tools are considered dual-use: the same reward-modeling and constitution-writing techniques that make a model safer in one country’s hands can make it a more effective censor in another’s.
In the news
This tension was the subject of an award-winning paper at this year’s ICML, which warned that widely used alignment toolkits can be repurposed to enforce state censorship rather than genuine safety goals — a reminder that how a model is aligned, and by whom, matters as much as whether it is aligned at all. For the broader safety picture beyond alignment specifically, see our explainer on what AI safety covers.
FAQ
Is AI alignment the same as AI safety? No. Alignment is narrower — it is about making a model’s behavior match intended goals and values. AI safety is the broader field that also covers misuse prevention, security, and system robustness.
Does aligning a model make it worse? Sometimes, in narrow ways. The “alignment tax” refers to measurable drops in raw benchmark performance that can come from safety fine-tuning, though labs actively work to minimize it.
Who decides what a model is aligned to? Whoever trains it — through the human feedback it’s shown or the constitution it’s trained against. That makes alignment a governance question as much as a technical one, which is why transparency about training choices matters.
Can a user tell if a refusal is a safety measure or a restriction? Usually not from the response alone. Transparency documents like model or system cards are the main way outsiders can check what a model was actually trained to avoid, and why.
Sources: Bai et al., “Constitutional AI: Harmlessness from AI Feedback” (2022); Wikipedia: AI alignment; Wikipedia: Reinforcement learning from human feedback.