Key Principle
An LLM application is a transformation loop: the user's problem is converted into a text prompt, the model completes the document, and the completion is transformed back into the user's domain. The feedforward pass pipeline — context retrieval, snippetizing, scoring/prioritizing, prompt assembly — replaces ad hoc prompt writing with systematic engineering. The Little Red Riding Hood principle governs the entire process: prompts should closely resemble documents from the model's training set.
Why This Matters
Ad hoc prompt crafting fails at production scale. The feedforward pass provides a repeatable engineering process that separates what to include (content selection) from how to include it (formatting and assembly). The Little Red Riding Hood principle gives a testable heuristic: when output quality degrades, ask whether the prompt looks like a real document the model might have seen.
Without the priority structure in scoring, rich dynamic context crowds out the instructions that tell the model what to do with that context — the model gets lots of information but no clear task.
Good Examples
The feedforward pass pipeline:
- Context retrieval: Gather raw text — user input, documentation search results, instructional boilerplate.
- Snippetizing: Break context into prompt-appropriate chunks; convert formats as needed.
- Scoring and prioritizing: Assign integer priority tiers (all higher-tier before lower-tier) and floating-point relevance within tiers.
- Prompt assembly: Combine snippets respecting token budget, ordering, and training data resemblance. (Chapter 4)
The Little Red Riding Hood Principle in practice. "Don't stray far from the path upon which the model was trained. The more realistic and familiar you make the prompt document and the more similar it is to documents from the training set, the more likely it is that the completion will be predictable and stable." (Chapter 4) Use common document formats: homework problems, markdown, transcripts. Format prompts to look like documents where the desired output naturally follows.
GitHub Copilot's evolution. Quality "improved considerably" when snippets from neighboring open IDE tabs were incorporated (Chapter 1) — the augmented-input level. Acceptance rate served as the key evaluation metric because it correlated most with actual productivity gains. (Chapter 4)
Counterpoints
Completion models need explicit answer signals. For completion models (not chat), you must explicitly signal when it is time for the answer — e.g., a "## Solution 2" header. Omitting this causes the model to elaborate on the problem rather than solve it, because the most likely continuation is more problem text. (Chapter 4)
Priority inversion is silent. If task clarification (instructions) has the same priority as context (retrieved documents), rich retrieval can crowd out the instructions entirely. The model then has abundant context but no clear task direction. Ensure instructions always have higher priority. (Chapter 4)
Key Quotes
"Don't stray far from the path upon which the model was trained. The more realistic and familiar you make the prompt document and the more similar it is to documents from the training set, the more likely it is that the completion will be predictable and stable." — Berryman & Ziegler, Chapter 4
Rules of Thumb
- Structure every LLM interaction as a feedforward pass: retrieve → snippetize → score → assemble
- Task clarification (instructions) must have higher priority than context (retrieved content)
- Format prompts to look like real documents — homework problems, reports, transcripts — not ad hoc text
- When output quality degrades, ask: "Does this prompt look like a document the model saw in training?"
- Measure acceptance rate or similar implicit signals rather than explicit user votes
Related References
- LLMs as Text Completion Engines - The Little Red Riding Hood principle is a direct consequence of text completion
- What Goes Into the Prompt - What goes into the context retrieval and scoring steps
- Assembling and Structuring the Prompt - The assembly step in detail