AI Foundations & Model Adaptation

AI systems become valuable when broad model capability is turned into useful behavior through architecture, adaptation, grounding, routing, and surrounding workflow design.

Visual navigation Use the cluster tools to review this hub as a navigable system, not only as prose: - Foundations Cluster Dashboard - Foundations Cluster - Local visuals

flowchart TD
    A["Transformer substrate"] --> B["Foundation model"]
    B --> C{"Adaptation layer"}
    C --> D["Prompting"]
    C --> E["Retrieval"]
    C --> F["Fine-tuning"]
    C --> G["Routing policy"]
    D --> H["Application behavior"]
    E --> H
    F --> H
    G --> H
    H --> I["Agent harness"]
    I --> J["Memory, tools, verification"]

Why this matters

The older way to explain generative AI starts with outputs: text, code, images, audio, video, and other generated artifacts. That is useful but incomplete. The more valuable question is how a general model becomes reliable enough to support a real workflow.

This page consolidates the model-level pages into one reader path:

• how transformer-style architectures made broad pretraining practical
• how foundation models become reusable capability layers
• how adaptation techniques change behavior without always changing the base model
• why harnesses, retrieval, memory, and verification often matter as much as raw model quality

A newer Stanford HAI policy brief sharpens one adaptation lesson in particular: customization is not only a product feature. It is a safety boundary. Fine-tuning can preserve useful behavior or specialization, but it can also remove the safety behavior previously encoded in the aligned base model, sometimes with very little data and very little cost.

That matters because many people still talk as if alignment is a durable property of the model itself. In practice, once downstream users can tune the model through an API, alignment becomes conditional on what happens after customization, not only on what the base-model provider originally shipped.

A newer Karpathy deep-dive source adds a stronger first-principles mental model for this whole page. An LLM is not a database with a personality bolted on. It is a token-sequence model built through staged training: pretraining turns internet-scale text into broad next-token capability, supervised fine-tuning turns that capability into assistant behavior, and reinforcement learning can discover longer reasoning traces in domains where answers can be checked. That makes practical use easier to reason about: parameters behave like vague recollection, context behaves like working memory, and tools refresh the working memory when the model should not rely on recollection alone.

Core thesis

Foundation models are adaptable substrates, not finished products.

The practical stack looks like this:

1. Architecture creates the scalable sequence model.
2. Pretraining creates broad capability.
3. Adaptation steers that capability toward a task, domain, or behavior.
4. Harness design connects the model to tools, context, memory, and verification.
5. Assurance determines whether the resulting system can be trusted in a real setting.

The model is necessary, but the system around it decides whether capability compounds or leaks away.

The Karpathy source makes the stack more concrete:

1. Pretraining teaches a base model to simulate internet text at the token level.
2. Supervised fine-tuning teaches conversation format, assistant style, refusal patterns, and examples of tool use.
3. Reinforcement learning lets the model search for token traces that produce verifiably better outcomes, especially in math and code.
4. Runtime context and tools determine what the model can rely on now, rather than what it vaguely absorbed during training.

A crucial extension from the fine-tuning policy brief is that adaptation is not automatically additive. It can also be subtractive. Fine-tuning may not introduce a brand-new harmful capability so much as strip away the refusal layer that had been suppressing harmful underlying behavior.

That yields a practical rule:

• adaptation can improve usefulness
• adaptation can stabilize behavior
• adaptation can shift style or domain fit
• adaptation can also erode safety guarantees

So the relevant question is not only *can the model be adapted?* but *which properties survive adaptation and which do not?*

How it works

Transformer-era substrate

The transformer matters because it made large-scale sequence modeling more parallel, scalable, and reusable. Self-attention gives the model a way to relate positions in a sequence without relying on recurrent processing, while positional methods preserve order. Multi-head attention lets the model attend to different relationship patterns at once.

For LLMs, the practical path is:

• text becomes tokens
• tokens become embeddings
• positional information is added
• masked transformer blocks process the sequence
• the model predicts the next token
• decoding policy turns probabilities into observable output

This explains why user-visible behavior depends on more than the weights alone. Tokenization, context policy, temperature, top-k, top-p, and routing all affect what the system does.

Foundation models as capability layers

Foundation models are broad pretrained systems that can support many downstream tasks. The key economic shift is pretrain once, adapt many times.

That changes how AI systems are built:

• organizations start from a general capability base
• domain specificity is added later
• application design becomes a major adaptation layer
• data, context, and workflow integration often create more value than model access alone

Karpathy's deep-dive framing is useful here because it separates the base model from the assistant product. A base model is a lossy compression and simulator of the training distribution. It can generate plausible internet-like continuations, but it is not yet a cooperative assistant. Assistant behavior appears after the model is trained on conversation protocols and ideal responses, including special tokens and formats for tools.

That distinction explains why "model capability" can be misleading. A model may contain knowledge in its weights, yet still need:

• examples that let it say "I do not know" when its own knowledge boundary is weak
• context-window evidence when recollection is not enough
• tools such as search or code execution when the task requires lookup, arithmetic, counting, or character-level manipulation
• reinforcement-learned reasoning traces when the task benefits from trying, checking, and backtracking

Adaptation levers

The deep-dive source adds a practical diagnostic lens for these levers:

Fine-tuning deserves a special caution: it can improve task behavior while weakening the safety behavior encoded in the base model.

The Stanford HAI policy brief adds four durable distinctions:

1. Very small tuning sets can matter. The brief reports that around 10 harmful examples were enough to compromise guardrails in both GPT-3.5 Turbo and Llama-2-Chat.
2. Cheap removal is possible. The removal cost can be tiny relative to the original alignment cost, creating an asymmetry between encoding and eroding safeguards.
3. Benign tuning can still degrade safety. Fine-tuning for responsiveness or on common non-malicious datasets can still make the model answer more harmful requests.
4. Closed versus open is not a clean safety boundary once fine-tuning APIs exist. Closed models with downstream tuning access can move materially closer to open-model risk profiles.

Fine-tuning as behavior persistence versus guardrail erosion

Fine-tuning is often discussed as if it does one thing: bake in desired behavior that is too cumbersome to achieve through prompting alone.

That is sometimes true. Fine-tuning can help with:

• stable tone or house style
• domain-specific output conventions
• repeated classification or extraction behavior
• narrower task specialization where prompts alone are too fragile

But the policy brief makes clear that fine-tuning also behaves like a removal tool. In many cases it does not teach the system a brand-new malicious skill. Instead, it makes previously suppressed underlying behaviors easier to access.

That leads to a useful conceptual split:

This is one reason post-customization evaluation matters so much. Two systems built from the same base model can no longer be assumed to share the same safety properties.

Generative AI as system behavior

Generative AI is not only a model class that creates artifacts. In practical use it becomes a system pattern:

1. a model receives context
2. it generates or decides
3. a tool or workflow acts
4. evidence is observed
5. memory or state is updated
6. the next interaction starts from a changed environment

That is why Agent Execution Systems and Agent Memory & Context Systems are adjacent to model adaptation rather than separate topics.

The fine-tuning source adds a further operational lesson: the same model family can sit inside very different downstream risk envelopes depending on whether the deployment stack allows:

• user-controlled fine-tuning
• private safety re-tuning
• output filtering after tuning
• downstream red-teaming
• customer visibility into what safety properties remain

So model adaptation is inseparable from deployment governance.

Post-customization safety is its own assurance stage

A particularly durable lesson from this source is that safety cannot be evaluated only once at the base-model stage.

A stronger lifecycle is:

1. align and evaluate the base model
2. expose or restrict adaptation surfaces
3. fine-tune or otherwise customize
4. re-run safety evaluation on the customized variant
5. add runtime monitoring and output controls where needed
6. communicate residual risk to downstream users

This matters because the safety claim “the base model was aligned” does not transfer automatically to the customized system.

Open versus closed risk convergence

The source complicates a common policy intuition.

The usual debate treats open models and closed models as distinct safety categories:

• open models are modifiable and therefore riskier
• closed models are controlled and therefore safer

The policy brief argues that this becomes much less true when closed models expose fine-tuning APIs. If customers can cheaply alter model behavior through provider-managed interfaces, the practical risk profile can move closer to open-weight modification than the marketing distinction suggests.

That does not mean open and closed are identical. It means the more useful policy question is:

• what downstream customization is possible?
• what monitoring or re-evaluation exists after customization?
• what information about the safety layer is shared with downstream users?

In other words, the real safety boundary is not only parameter access. It is the full customization-and-assurance surface.

Practical deep-learning literacy

The fast.ai course source adds a useful operator-level complement to the model-architecture sources. It argues for learning deep learning through working applications first: train and deploy useful models early, then deepen the theory as the practical questions become real.

That matters for this page because model adaptation is not only a research concept. Practitioners often understand the stack by moving through a concrete loop:

The durable lesson is that model literacy compounds when builders repeatedly cross the boundary between training, deployment, and evaluation. A practitioner who has shipped a small classifier or recommender will reason more concretely about prompts, fine-tuning, retrieval, deployment constraints, and failure modes than someone who has only read architecture summaries.

Failure modes

• Confusing model quality with product quality.
• Confusing parameter knowledge with reliable working memory.
• Treating retrieval as truth instead of source selection.
• Assuming fine-tuning solves missing workflow design.
• Assuming fine-tuning preserves base-model safety guarantees.
• Ignoring context cost and memory ownership.
• Overlooking decoding and routing as operational controls.
• Discussing generative AI as chat rather than as a tool-connected system.
• Asking a model to do arithmetic, counting, or character work in its head when a tool would be safer.
• Letting model novelty outrun verification, privacy, and governance.
• Assuming harmful fine-tuning is the only risk, while benign responsiveness tuning can also weaken safety.
• Treating closed-model APIs as inherently safe while ignoring downstream customization power.
• Reusing base-model safety claims after tuning without revalidation.
• Using content filtering on fine-tuning datasets as if it were a complete defense.
• Treating deep-learning theory as disconnected from the practical loop of training, deploying, observing errors, and adapting the next model.

Practical implications

For builders, the right question is not just "which model?" It is:

• what context should the model see?
• what should be retrieved versus learned?
• what behavior needs to be stable?
• what can be handled by prompt, router, tool, or skill?
• what evidence proves the output is good?
• who owns memory and state?
• which tasks should be given more tokens to reason, and which should be delegated to tools?
• how will safety be re-tested after customization?
• how much downstream tuning access should users actually receive?
• what small working model or demo would make the adaptation problem concrete?

For operators, the important distinction is between:

• model demos that impress once
• systems that become more useful as workflows, memory, and verification improve

For policy and governance, the stronger lesson is:

• evaluate downstream customization pathways, not just base-model release posture
• require clearer disclosure that fine-tuned variants may not retain base safety properties
• treat post-customization red-teaming and evaluation as part of deployment
• avoid simplistic open-versus-closed framing when API fine-tuning can bridge much of the risk gap

Read next

1. Agent Execution Systems for harnesses, tools, skills, and verification.
2. Agent Memory & Context Systems for compaction, retrieval, memory ownership, and persistence.
3. Trust Boundaries & Assurance for safety, privacy, containment, and evidence-backed governance.
4. AI-Native Organizations for how model capability changes work design.
5. AI Safety & Control for runtime guardrails, permissions, and post-customization assurance.