Constitutional AI: Harmlessness from AI Feedback

Why it matters

It is one of the clearest published examples of a concrete alignment technique - not a principle or a policy, but a specific training method - and it names the exact tension the method is trying to resolve: making a model both harmless and willing to engage, rather than harmless by simply refusing everything.

Key takeaways

• The only human oversight in the method is a written list of principles (the "constitution") the model uses to critique and revise its own outputs - not per-example human harm labels.
• Two-phase process: a supervised phase where the model critiques and rewrites its own responses against the constitution, followed by a reinforcement learning phase using AI-generated preference judgments ("RL from AI Feedback").
• Aims explicitly for a "non-evasive" harmless assistant - one that explains its objections to a harmful request rather than refusing it with no explanation.
• Reports that chain-of-thought reasoning during the critique step improves both the judged quality and the transparency of the model's decisions.

Part of these reading paths

• AI Safety and Security