AI / Analysis & Decision Making
Correlation or Causation
Two things moving together does not mean one drove the other
I believe [A] causes [B], based on: [observed correlation or data]. Falsify this causal claim: 1) List at least 3 third variables that could explain both A and B; 2) Point out the possibility of reverse causation — the path where B causes A; 3) Say under what conditions the correlation would disappear; 4) Give a test that could distinguish causation from correlation, and the data it would need; 5) If I only have the current data, how far can my conclusion actually go. Do not affirm my judgment first.
How to use
Read the third variables first, then go back and check your own explanation for the same gap.
Good for
- You have a correlation that is suspiciously clean — two curves nearly on top of each other — and you're about to put money or a strategy change behind it.
- Observational data backs your intuition but you can't state the mechanism — when you can't explain the “why,” a third variable is usually doing the work.
Not for
- You already ran a randomized controlled trial with a real treatment and control group — debating “maybe it's just correlation” wastes time; discuss effect size and external validity instead.
- What you actually need is “is this direction worth trying,” not “is this causal” — falsification keeps you stuck, while a cheap experiment answers faster.
Example
Input
Accounts that use the Team feature renew at 92%; those that don't renew at 71%. So we should push every account to adopt Team.
What you get
You want at least three third variables: large companies buy Team anyway and churn less anyway, and sales tends to pitch Team right before renewal. It should also name the reverse path — accounts already planning to renew are the ones willing to spend time configuring it. Then one test that separates them: randomly invite SMB accounts nearing renewal to enable Team and compare 90-day renewal.
Common mistakes
- Using it to “prove my causal claim is right”: you ask only for counterexamples and refuse the conclusion “this data can't distinguish,” squeezing out a pile of plausible-sounding mechanisms.
- Handing it correlational data and demanding a causal conclusion, then blaming it for “only saying it can't be proven” — it can't produce causation because the data doesn't contain any.
Try tweaking it like this
- Change “falsify this causal claim” to “design the smallest experiment that could falsify it” — you move from debate to a test you can actually run.
- Add “if you had to bet now, what probability would you put on the causal claim” — a binary judgment becomes a calibratable confidence.