Troubleshooting and Fine-Tuning Claude

Key Principle

Claude is a debuggable system, not a black box — every failure has a measurable cause, so troubleshooting becomes symptom → category → fix routing rather than guesswork (Chapter 13).

"Every issue has a measurable cause — whether it's prompt clarity, configuration accuracy, or system load." (Chapter 13)

Failures fall into three categories: (1) prompting/context problems (vague or truncated output), (2) API/configuration problems (auth, malformed payloads, rate limits), and (3) performance/cost problems (latency, oversized prompts). The load-bearing insight: most "model failures" are actually configuration or payload issues, so naming the category keeps you from debugging the wrong layer.

Why This Matters

Without triage, an engineer blames the model for what is really a 401 (bad key), a 400 (malformed body), a 429 (rate limit), or a Timeout (oversized prompt) — and burns hours in the wrong layer (Chapter 13). [The specific HTTP codes and header behaviors are illustrative; flag as [likely inaccurate to the real product] — verify against current Anthropic API error semantics.]

The deeper failure mode is treating ambiguous or half-finished output as "the model is wrong" instead of as an interpretation gap you can close. Vague output comes from insufficient context, over-general prompts, or truncation — not incapacity. Each ambiguous response is a free diagnostic report telling you which constraint was missing.

Good Examples

• Explicitness as the antidote to ambiguity. Tighten a vague prompt by adding quantifiers ("five methods"), output structure, measurable examples, a persona, and a self-check directive. Each pass narrows intent; the goal is to detect and correct ambiguity systematically, not to eliminate it up front (Chapter 13).
• Programmatic self-clarification ("Claude as its own reviewer"). Feed Claude its own prompt plus response and ask it to flag gaps or unmet requirements. In automation pipelines there is no human to catch omissions, so the review must run in-loop (Chapter 13).
• Truncation detection and controlled continuation. Detect by checking for closing punctuation or syntax (., }, ;). Recover by either raising max_tokens by ~1.5× or sending a continuation prompt — "Continue from the last point, maintaining context and code correctness" — then concatenating the pieces. Automated workflows must never silently consume half-finished output (Chapter 13).
• Resilient request wrapper (safe_claude_call). Wrap calls in a retry loop (3×) with exponential backoff on connection errors; on truncation raise max_tokens and continue; on token overflow shorten input to ~70% and retry; log latency and tokens. Re-raise non-recoverable errors instead of swallowing them (Chapter 13). [Source code has known typos and uses fragile string-matching on error messages; treat the structure as the lesson, not the literal code — [likely inaccurate to the real product].]

Counterpoints

• Panicking at limits. Timeouts and token limits are guardrails, not failures — treating them as crashes leads to silent data loss instead of detection plus continuation (Chapter 13).
• Ad-hoc prompts. An unversioned prompt drifts and invites re-guessing; without a registry, good prompting is not reproducible across people or projects (Chapter 13).
• Tuning by vibes. Changing a prompt and eyeballing one new output conflates a real improvement with randomness — you must re-test the same inputs old-vs-new, with temperature=0 for reproducibility (Chapter 13).

Key Quotes

"The antidote to ambiguity is explicitness." (Chapter 13)

"Timeouts, token limits, and truncation are not signs of failure — they're natural guardrails." (Chapter 13)

"Every issue has a measurable cause — whether it's prompt clarity, configuration accuracy, or system load." (Chapter 13)

Rules of Thumb

• Triage every failure into one of three categories before touching the prompt.
• Assume "the model failed" is wrong until you have ruled out key, payload, and rate limit.
• Detect truncation programmatically; never trust output that lacks closing syntax.
• On truncation, raise max_tokens ~1.5× or send a continuation prompt and concatenate.
• Store every successful prompt as a versioned, parameterized template (name, category, {placeholders}, version) under version control, contributed via PR and schema-validated in CI.
• Run the Tuning Loop — Observe → Adjust → Validate → Automate — and never skip Validate: re-run identical inputs at temperature=0 so improvement is measured, not assumed.

Related References

• Prompt Anatomy — Specificity, Chain-of-Thought, and Context as Bandwidth - explicitness and structure as the substrate for clear prompts
• Prompt Recipes (The Cookbook) - the recipes that fill the versioned prompt registry
• Cost and Latency Optimization — Tokens as a Financial Control - observability and token budgeting that feed troubleshooting