Decide What to Do with a Failed Experiment
Added 2026-11-04
Debrief, drop or pivot — first classify the failure.
We ran an experiment and the result was bad (details below). Help me judge: 1) whether the failure was the hypothesis, the execution or the measurement; 2) what each finding implies — drop it, improve and retry, or fix the measurement; 3) the one variable most worth changing if we continue; 4) how to convert existing work into other value if we stop — data, code, learning; 5) a rule for deciding it is worth one more attempt. Do not just say "learn from it".
Site original
Chinese original
How to use
失败实验最贵的不是失败，是没分清哪种失败。
Try tweaking it
- Ask what evidence would overturn the judgement.
- Have it argue both sides with three reasons each.
What you should get
- A decision table: assumptions, evidence, counter-evidence
- A next action with an observable signal
Common mistakes
- Asking only "should I?" and getting equally valid answers both ways.
- Never asking for counter-examples, so you only hear support.
How to check the output
- It states what evidence would overturn the conclusion.
- Both sides are argued, with assumptions marked as such.
Where it works
- Good for: splitting a decision into assumptions, evidence and counter-cases, leaving the call to you.
- Watch out: ask what evidence would overturn the judgement, or you will only hear supporting reasons.