Library
The Christensen Engine Has No Twin — But It Has Relatives in Surprising Places · 1 of 12
The Christensen Engine Has No Twin — But It Has Relatives in Surprising Places
technology MEDIUM

Conversational Assessment Design

The Christensen Engine Has No Twin — But It Has Relatives in Surprising Places

Key Principle

The primary value of a conversational assessment system is systematic information gathering and organization — not AI-generated insight. Assessment design should prioritize structured data collection, evidence tracking, and question selection that maximally reduces uncertainty. The assumption map (prioritized list of what to test, in what order) is a more valuable output than any verdict.

"the main driver of improved outcomes was collecting clinically relevant information ahead of the human assessment — not the AI's therapeutic capability." (p. 1, chunk 004) — Limbic

Why This Matters

Teams that optimize for AI-generated insight quality build for the wrong value proposition. The actual mechanism of impact is completeness and structure of collected data. A verdict ("good idea" / "bad idea") gives the user a reaction. An assumption map gives them a research agenda. The distinction determines whether the tool creates dependency or capability.

Good Examples

Evidence-Confidence Tracking: Multi-tier confidence ladder — Hypothesis → Single-source → Corroborated → Strong — that strengthens with return visits. No market equivalent exists. Without it, all evidence is treated equally regardless of corroboration, and the assumption map loses its prioritization logic. (p. 5, chunk 002)

Three-Valued Logic (Ada Health): Present/Absent/Unknown for every data point. Skipped questions don't penalize any hypothesis. Full re-computation from complete evidence set on every new input prevents state corruption from compounding errors. (p. 5, chunk 003)

Information-Theoretic Question Selection (Ada Health): Each question chosen to maximally reduce diagnostic uncertainty given current evidence. Fixed sequences cannot optimize for information already gathered — they either ask too many questions (fatigue) or too few (insufficient signal). (p. 5, chunk 003)

Automated Assumption Extraction (Strategyzer's methodology, automated): Importance × Evidence 2x2 matrix identifies riskiest assumptions across Desirability, Feasibility, Viability, Adaptability. State machine automates extraction by prompting category-by-category, enforcing completeness that manual generation cannot guarantee. (p. 3, chunk 005)

Citation Tracking as Trust Architecture (Sixfold AI): Every extracted fact links to its source document — score + reasons + evidence + sources. Without citation, AI scoring is an opaque number. With citation, it becomes an auditable argument. (p. 1, chunk 005)

The "Score Only What You Assessed" Constraint (Alex AI): "Alex can only score candidates on topics covered in the interview." (p. 2, chunk 005) Prevents generating scores for dimensions without evidence.

Founder Signal Engine Insight (ValidatorAI): Most founders describe customers vaguely ("people," "users") rather than specifically. Vague inputs guarantee generic outputs. State machines solve this by dedicating entire states to sharpening input quality before validation. (p. 3, chunk 005)

Counterpoints

  • The Buildpad Quality Problem: Phase transitions triggered on completion rather than quality. Users advanced with shallow input, producing an illusion of rigor. Guard conditions must measure input quality, not just presence. (p. 1, chunk 005)
  • Assessment Gaming: Users submit minimal-effort responses to advance quickly. Build effort-detection into guard conditions; switch to diagnostic questions after repeated low-effort input. (p. 5, chunk 005)
  • The Two-Pass Model: 30-second return visits for evidence submission using Nir Eyal's trigger-loading. Zero analogs exist — but no empirical validation data is presented either. (p. 5, chunk 002)

Key Quotes

"the main driver of improved outcomes was collecting clinically relevant information ahead of the human assessment — not the AI's therapeutic capability." (p. 1, chunk 004) — Limbic

"Alex can only score candidates on topics covered in the interview" (p. 2, chunk 005)

"Your state machine should define what 'done' means for each state as a measurable guard condition, not just 'user submitted something'" (p. 1, chunk 005)

Rules of Thumb

  • Build for data collection completeness, not mid-conversation AI brilliance.
  • Use three-valued logic: never force binary on incomplete data.
  • Select questions to maximize uncertainty reduction, not follow a fixed sequence.
  • Score only what you assessed — never generate unsupported evaluations.
  • Dedicate states to sharpening input quality before attempting validation.
  • Citation tracking transforms opaque scores into auditable arguments.

Related References