AI EncyclopediaArchitectures & models
Foundation Model
A model trained on broad data at scale, usually via self-supervision, that can be adapted - by fine-tuning, prompting or other means - to a wide range of downstream tasks, rather than built for one task alone.
In plain English
It's the difference between a Swiss Army knife and a single-purpose tool. A foundation model is trained once, broadly, and then reshaped for many different jobs - answering questions, writing code, summarising documents - instead of a separate model being built from scratch for each one.
Technical explanation
The term was coined in August 2021 by Stanford's Center for Research on Foundation Models (CRFM), part of the Stanford Institute for Human-Centered AI, to mean 'any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks.' It is a category, not an architecture - most current foundation models are transformer-based LLMs, but the term also covers multimodal and other broad-data models.
Why it matters
The term reframes how the industry talks about model economics: a small number of very expensive foundation-model training runs can be adapted, cheaply, into a very large number of downstream products - which is the business model behind nearly every major AI lab today.
Real-world example
GPT-4, Claude, Gemini and Llama are all described in the industry as foundation models: each is trained broadly once and then adapted - via fine-tuning, RAG, or prompting - into many distinct products and use cases.
Common misunderstanding
That 'foundation model' is a marketing synonym for 'large language model.' Stanford's own definition is broader - it also covers large multimodal and vision models, not text-only systems alone.
Something wrong here?
Every entry is hand-researched and hand-written by Nakoda. If a fact is stale, a source has changed or a definition needs sharpening, tell us and we will check it.