Opening
The problem starts after good retrieval
Every other chapter in this book is about getting the right information into the context window. This one starts where that story breaks.
Air Canada's support chatbot had the bereavement-fare policy in its context. The correct text was retrieved. The model read it — and fabricated a retroactive fare-adjustment program that did not exist. A customer relied on the fabrication. A tribunal held the airline liable.
The document was right there. The model answered from somewhere else.
Now consider a different failure. A support bot's instructions say: "Use only the customer's current plan details. Do not mention other plans." The bot pulls the right CRM record and formats a clean answer — then adds "Customers on our Premium tier enjoy priority routing for $49/month." Every fact in that sentence may be true. The error is not fabrication. The model violated an explicit scope constraint.
These look similar from the outside — "the model ignored the context." They are actually different problems. The Air Canada case is a groundedness failure: the model went beyond its evidence. The Premium-plan case is an instruction-following failure: the model broke a rule it was given. They have different causes, they need different metrics, and they require different fixes. If you don't separate them, you end up in a whack-a-mole loop where fixing one makes the other worse.
One decomposition runs through the rest of this chapter. The assembled context carries requirements (rules the output must satisfy) and evidence (facts the output is allowed to use). Instruction following measures whether the output satisfied the requirements. Groundedness measures whether the output stayed inside the evidence. The diagnosis loop and control stack that follow both depend on knowing which obligation broke first.
The next section makes this decomposition precise.
Mental model
Two contracts, one context window
The assembled context is not one undifferentiated blob. It imposes two distinct obligations on the output.
By the time the model generates, it is looking at system instructions, a user query, retrieved passages, tool outputs, conversation history, memory, intermediate plans — sometimes verifier outputs too. All of that folds into one token sequence. But it does two jobs.
- Requirement-bearing context tells the model what the answer must look like: format constraints, scope limits, tool-use policies, ordering rules, refusal conditions.
- Evidence-bearing context tells the model what the answer is allowed to know: retrieved documents, database rows, tool return values, verified sub-results.
Break the first contract and you get an instruction-following failure. Break the second and you get a groundedness failure. These can happen at any stage of the pipeline: a reranker that ignores "prefer recent internal policy" can cause the generator to hallucinate even when generation itself is blameless.
Understanding why these failures happen — not just that they happen — changes how you fix them. Language models are trained to be helpful. RLHF rewards completeness, fluency, and informativeness. When the evidence runs out, the model faces a choice: stop (which feels unhelpful and gets penalized during training) or fill the gap from parametric memory (which sounds better and gets rewarded). The training signal overwhelmingly favors the second option. This is not a reasoning error; it is the correct optimization of the wrong objective for your task. Groundedness failures are, in large part, helpfulness succeeding where it should not.
Instruction-following failures have a related but distinct cause. Parametric memory and contextual instructions compete for influence over generation. When they agree, everything works. When they conflict — the context says "do not mention other plans" but the model's training has seen thousands of examples where mentioning related products is helpful — the model may follow whichever signal is stronger. For well-rehearsed behaviors, parametric memory often wins. The model is not ignoring your instructions; it is being outweighed by thousands of training examples that say the opposite.
Both failures get worse as context grows. The needle-in-a-haystack problem — whether a model can locate and use a specific fact buried in a long document — is a groundedness challenge in disguise: the evidence is present, the model just fails to attend to it. The lost-in-the-middle phenomenon (Liu et al., 2024) shows that models attend most to the beginning and end of their context window, creating a dead zone in the middle. This affects both contracts: instructions placed mid-context get dropped (IF failure), and evidence placed mid-context gets ignored (groundedness failure). If your system is failing, one of the first things to check is whether the critical information sits in the attention dead zone.
Let x = (q, e) where q is the requirement bundle and e is the evidence bundle. The model returns response y. Two questions:
The formalization is deliberately minimal. Its job is to name two capabilities clearly enough that evaluation and intervention can attach to them — not to be a complete theory of language-model behavior. The metrics (ratios) are how we measure the capabilities, not the capabilities themselves.
Diagnosis loop
Classify, localize, measure — then intervene
The instinct is to rewrite the prompt. Resist it. The two-contract model only helps if you use it to diagnose before you intervene.
Three passes. First: classify. Is this an unsupported claim, a dropped requirement, both, or an upstream failure in retrieval or reranking? Second: localize. Was the needed context absent from the window, present but buried, or present and ignored? Third: measure. Pick a metric whose unit matches the failure — requirement satisfaction for dropped rules, groundedness for fabricated facts.
Each capability responds to different interventions. Structured output raises requirement satisfaction without touching groundedness. Better citation verification raises groundedness without fixing scope violations. Rewarding abstention shifts the answer/refuse boundary independently of both.
Prompt. "Answer in JSON. Use exactly three bullets in steps. Cite the supporting policy line for each bullet. Do not mention plans other than the customer's current plan."
Output. Valid JSON, three bullets, citations present — but one bullet reads: "Customers on Premium pay $49/month."
| Check | Pass? | Verdict |
|---|---|---|
| Requirement: valid JSON | Yes | Format contract satisfied. |
| Requirement: exactly three bullets | Yes | Structure contract satisfied. |
| Requirement: no other plans | No | Instruction-following failure. This is the bug. |
| Claim: Premium costs $49/month | Maybe true | Irrelevant — the requirement contract failed first. |
Decomposition tells you which axis owns the failure. A groundedness fix would not help here. A scope-enforcement fix would.
Control stack
Six layers, from context cleanup to weight changes
Once you know what failed and where, the question becomes what to fix. The control stack orders your options by cost.
Lower layers reshape the local state so the model's existing behavior is more likely to succeed. Upper layers change what the model treats as normal behavior. The science-heavy claim of this chapter: the upper two layers — SFT and RL — deserve far more depth than they typically get.
| Layer | What it changes | Best for |
|---|---|---|
| 0. Model choice | Baseline behavior and failure profile | The base model fights the product objective |
| 1. Context representation | Chunking, metadata, source ordering, state visibility | Evidence present but buried or conflicting |
| 2. Prompt and output design | Instruction salience, structure, abstention language | Dropped rules that respond to wording changes |
| 3. Orchestration and verification | Reranking, extract-then-answer, second-pass checks | Recurring claim-level errors and upstream drift |
| 4. SFT / adapters | Default output style and evidence-use habits | Persistent failure modes that prompting cannot fix |
| 5. RL with verifiable rewards | Default policy under tradeoffs and abstention pressure | Prompting still fights the weights; reward is checkable |
Prompting and orchestration matter, but they ask the model to override its priors at inference time, every time. Grounded SFT and verifiable-reward RL are qualitatively different: they change which behavior feels locally easiest. The model stops fighting you because the default shifted.
Three patterns emerge from recent benchmarking work. First, instruction-following performance degrades sharply and non-linearly as the number of constraints rises — a model that handles five rules well may break down at fifteen (Jaroslawicz et al., 2025). Second, holistic "overall quality" evaluation is too noisy to explain why; only criterion-level decomposition reveals which rules broke (Saad-Falcon et al., 2024). Third, groundedness fails one claim at a time, not all at once — a mostly-correct answer hides one fabricated detail (Min et al., 2023; Jacovi et al., 2025). The workflow implication: decompose the failure, instrument the right dimension, then decide whether you need a lower-stack fix (context, prompt) or an upper-stack fix (SFT, RL).
Grounded data
Grounded data is not more documents
The upper layers of the control stack — SFT and RL — need training data. Not just any data. The distinction matters.
"We need grounded data" usually means "more examples with citations." That is too weak. Useful adherence data has structure: which requirements were active, which claims were licensed, what evidence supported them, and what the correct behavior was when the evidence ran out.
Start from production traces, not a generic instruction corpus. Real traces give you the failures that matter: near-duplicate queries with different retrievals, subtle rule conflicts, stale passages outranking fresh ones, fluent answers hiding one unsupported sentence. A model trained on "helpful assistant" data will never see those cases.
| Unit | Fields | Used for |
|---|---|---|
| Requirement unit | Requirement text, satisfied/violated, location in context | IF eval, prompt hardening, IF rewards |
| Claim unit | Claim text, evidence span, support label, citation sufficiency | Groundedness eval, verifier training, claim rewards |
| Abstention unit | Answerable flag, ideal response, evidence-gap type | Refusal tuning, confidence calibration, abstention rewards |
| Upstream ranking unit | Candidate passages, desired order, stale-vs-fresh conflicts | Retriever/reranker tuning, upstream adherence checks |
Input. Query: "Can I get a refund on my annual subscription?" Requirements: answer from evidence only; cite the policy section; do not mention other plan tiers.
Evidence pack. Refund policy doc (3 paragraphs): §1 covers eligibility, §2 covers the 30-day window, §3 covers payment method. No paragraph mentions turnaround time.
Artifact 1 — SFT gold answer. "Yes. Per §2 of the refund policy, annual subscriptions may be refunded within 30 calendar days of purchase. The refund will be credited to your original payment method (§3). The policy does not specify a processing timeline." Every claim maps to a specific section. The gap is acknowledged, not filled.
Artifact 2 — correct abstention. Same query, but the evidence pack contains only §1 (eligibility for monthly plans). Gold answer: "The available policy excerpt covers monthly plans only. I cannot confirm annual subscription refund terms from this evidence." The model learns to stop where evidence stops.
Artifact 3 — hard negative (evidence perturbation). Same query and requirements, but §2 is replaced with a version from six months ago that says "14 calendar days." Gold answer uses the 14-day figure. Teaches the model to follow the evidence it actually has, not the answer it remembers.
Artifact 4 — hard negative (rule perturbation). Same query and evidence, but the "do not mention other plan tiers" requirement is moved from position 1 to position 5 in a list of 8 rules. Gold answer still respects it. Teaches the model to follow rules regardless of position.
SFT and adapters
SFT teaches the model where knowledge is allowed to come from
With grounded data in hand, the next question is how to use it. SFT changes the model's default evidence-use behavior.
The core move: build demonstrations where the model answers from the provided evidence, cites what supports each claim, and abstains cleanly when the evidence falls short. Then train the model to imitate those trajectories.
The loss function is standard cross-entropy — nothing new there. What matters is the dataset, not the objective. A model fine-tuned on 10,000 examples of fluent-but-ungrounded answers will become better at fluent-but-ungrounded answers. A model fine-tuned on examples where the response systematically stops at the evidence boundary — citing what is supported, acknowledging what is not — will make that behavior the path of least resistance. For adherence, curation carries more weight than architecture.
Four kinds of demonstrations earn their place in an adherence SFT dataset:
- Positive grounded answers. Fully supported response, correct citations, no overclaiming.
- Correct abstentions. The model narrows its answer or refuses outright because the evidence does not settle the question.
- Counterfactual near-misses. Differs from the gold answer by exactly one dropped rule or one unsupported claim. Teaches the model where the boundary is.
- Stage-specific examples. Retrieval and reranking prompts get their own adherence demonstrations — do not assume the generator can compensate for upstream failures.
LoRA and other adapter methods shine when the failure mode is narrow and repeatable. If your product consistently blends parametric knowledge into one domain's policy answers, an adapter can shift that local default without a full retrain.
Before. "The policy does not specify a refund timeline, but customers usually receive refunds within 5-7 business days."
After. "The policy excerpt provided does not specify a refund timeline. I cannot state one from this context."
The model is not less capable. It has learned that "use only the local evidence" is the default for this class of task. The 5-7 day figure came from parametric memory. SFT taught the model to stop reaching for it.
Verifiable RL
RL changes which tradeoff wins by default
SFT teaches what good behavior looks like. RL teaches which tradeoff to make when good behavior in one dimension costs good behavior in the other.
A single scalar reward — "good answer" — is too blurry. It cannot tell the policy whether it lost points for dropping a rule, overclaiming beyond evidence, citing the wrong span, or refusing when it should have answered. If the chapter's two-contract decomposition means anything, the reward must mirror it.
S_req rewards satisfied requirements. G rewards supported claims. Cite rewards sufficient (not decorative) citation. A rewards the answer/abstain boundary. The penalty term makes unsupported completion expensive.
Setup. Query: "What is the cancellation fee?" Requirements: (1) answer from evidence only, (2) cite the policy section, (3) use bullet format. Evidence: policy doc with §4 stating "$25 cancellation fee for monthly plans."
Model output. "• The cancellation fee is $25 for monthly plans (§4). • Annual plans are subject to a prorated fee." (No citation for the second bullet.)
| Component | Score | Reasoning |
|---|---|---|
| S_req (requirement satisfaction) | 2/3 = 0.67 | Bullet format ✓, evidence-only ✗ (second bullet unsupported), citation present ✓ |
| G (groundedness) | 1/2 = 0.50 | Claim 1 supported by §4 ✓, Claim 2 not in evidence ✗ |
| Cite (citation sufficiency) | 0.50 | One of two claims has a valid citation |
| A (abstention) | 0.0 | Should have abstained on annual plan info; did not |
| Unsupported penalty | 1.0 | One fabricated claim present |
With lambdas (0.3, 0.3, 0.1, 0.15, 0.4): r = 0.3(0.67) + 0.3(0.50) + 0.1(0.50) + 0.15(0.0) − 0.4(1.0) = 0.201 + 0.15 + 0.05 + 0 − 0.4 = 0.001. Near-zero reward: the output is mediocre but the fabrication penalty almost cancels the correct parts. A fully grounded one-bullet answer would score higher.
The optimizer choice is secondary. Scalar reward checks and sampled responses point toward PPO- or GRPO-style updates. Pairwise chosen/rejected data from verifiable checks fits DPO. The engineering principle stays constant: the reward must come from things you can check locally and repeat reliably.
This is where grounded-model results become concrete. On benchmarks like FACTS Grounding, the headline is not the average score — it is the failure profile. Models that have undergone grounded post-training show dramatically lower fabrication rates, not just reshuffled phrasing. The gap between a general-purpose model and a grounded variant on the same base can exceed ten percentage points on claim-level precision (Jacovi et al., 2025). That is the kind of behavioral shift that upper-stack methods exist to produce — and the kind that prompting alone cannot replicate.
| Signal | What it means | Upper-stack response |
|---|---|---|
| Prompt edits help briefly, then regress | Wording is not the bottleneck | Shift default behavior via SFT or RL |
| Same unsupported claims recur in same domains | Evidence precedence is not the learned default | Add claim-verifiable and abstention rewards |
| Good eval exists but generation ignores it | You know what "correct" looks like; the policy does not | Use evaluator output as training signal |
| Hard constraints fail under long instruction bundles | The model needs stronger internal prioritization than prompts supply | Pair requirement-level rewards with adapters |
Instruction following and groundedness pull in opposite directions. Reward helpful completeness too aggressively and the model fills gaps with parametric guesses. Punish unsupported claims too harshly and the model becomes uselessly cautious. Verifiable reward design lets you place the system on that boundary deliberately — using checks you can audit, not intuitions you cannot.
Summary
What you should now be able to do
The mental model is compact. The assembled context imposes two contracts: a requirement contract (obey the rules) and an evidence contract (stay inside the facts). Those contracts fail differently, demand different metrics, and need different repairs. "Better prompting" is not a diagnosis.
The diagnosis loop follows directly. Classify the failure. Localize it — and check for position effects before assuming the model is ignoring your context. Measure it with the right unit. Only then climb the control stack. If the failure responds to prompt changes, stay low. If the same failure persists because the model's default behavior is wrong, move upward — into grounded data, SFT, and verifiable-reward RL.
You should be able to
Identify whether a given failure is an instruction-following break or a groundedness break. Explain why the distinction changes the fix. Measure each dimension separately. Describe what grounded SFT teaches and why RL shifts the default tradeoff.
The key tension to carry forward
Completeness and correctness pull in opposite directions. Every intervention you apply moves the system along that boundary. The control stack exists so you can place the system deliberately — starting cheap, escalating only when the failure outlasts the lighter fix.
Delete one critical line from the evidence bundle. Move one rule from the top of the prompt to the middle. Attach a citation that is related but insufficient. Swap a fresh internal policy for an older version with the same keywords. These small perturbations mirror how production actually fails. Giant synthetic corruptions do not.