Prompt Engineering for LLMs

LLMs are text completion engines — they produce the most likely continuation of a document, not "answers"
Every technique (few-shot, CoT, RAG, tools) works because it shapes the prompt to resemble training data patterns
The model has no internal monologue, no ability to backtrack, and no way to verify facts
Prompt engineering = understanding the model's constraints and exploiting them systematically
Evaluation must come first — non-deterministic output makes intuition-based development unreliable

Situation	Do This	Avoid This
Complex reasoning task	Add chain-of-thought before the answer	Expecting the model to "think" internally
Model ignores context	Move critical content to start/end (Valley of Meh)	Packing context in document order
Irrelevant output	Check if prompt resembles training data (Little Red Riding Hood)	Adding more instructions without checking format
Hallucinations	Use RAG to ground in retrieved facts; add verifiable artifacts	Telling the model "don't make stuff up"
Classification errors	Ensure options start with unique tokens; check logprobs	Assuming confident output = correct output
Complex multi-step task	Build a structured workflow with subtasks	Using a single conversational agent
Unsafe tool execution	Intercept in application code	Relying on prompt instructions for safety
Measuring quality	Build evaluation first; use SOMA for LLM-as-judge	Eyeballing outputs or trusting absolute scores

"At their core, LLMs are just text completion engines that mimic the text they see during their training." — Berryman & Ziegler, Preface

Overview