A LinkedIn series on the framing decisions that determine whether analysis is causal.
A single churn rate cannot tell you which lever to pull. The aggregate decomposes into distinct mechanisms, from fit failure to competitive loss, each pointing to a different decision. The number is the start of the analysis, not the end of it.
Activation and retention are related but do not always move together. A lift can be real improvement, a pull-forward effect, a selection effect, or short-term behavior that fades. The causal question is whether the change kept users who otherwise would have churned.
Teams choose metrics because they are available, stable, and easy to move, then treat the proxy as the outcome. Pushing on a proxy is a treatment effect problem. The question is whether the effect transfers to the outcome it was supposed to represent.
One number before, another after, difference equals impact. It feels intuitive, but teams act on causal claims, not charts. A clean timeline describes change. Without a comparison group or a stronger design, it does not measure impact.
Parallel trends is not a technical footnote. It is the assumption that makes the comparison believable. When it breaks, the estimate can still look precise while pointing strategy in the wrong direction.
Not every business problem needs an experiment, but every impact claim needs the right identification strategy. Randomized experiment, DiD, ITS, and synthetic control each carry assumptions the data has to earn.
The hardest part is rarely running the model. It is knowing what kind of answer the data can actually support. That is a judgment problem, not a tooling problem.
Teams measure observed lift but rarely define the right baseline. Without a counterfactual, you are interpreting change, not measuring impact.
Teams scale the wrong strategy when they treat correlations as causal mechanisms. Signal, symptom, and lever are different.
Leadership in data work is making counterfactual assumptions explicit, especially when you can't run an experiment.
Teams confuse the three and end up making confident decisions on the wrong analytical foundation.