Skip to content
Was It Causal?

Your ROAS is up.
Revenue is flat.
Both numbers are correct.

A practical guide to marketing measurement when tracking, attribution, and experiments are imperfect, which is always.

See a worked example Start here

The gap, measured

CAMPAIGN ON Without it Treated incremental
Reported CAC
$59
Incremental CAC
$95

A six-week geo holdout across 40 markets put the lift at +7.0% (95% CI +3.7% to +10.4%). Reported CAC came in 38% below the incremental figure.

Simulated data 3-week average Read the analysis →

Start with the question

What are you trying to understand?

The method follows from the question. Each of these opens to the evidence that answers it, and to what that evidence still will not tell you.

  • Can I trust my tracking? Observe

    Audit identity, consent, and event definitions. Measure coverage loss by segment rather than as one accuracy number. Random loss costs you precision. Systematic loss costs you validity, and only the second one changes your conclusions.

    What it will not tell you Give you the true conversion count. Better tracking narrows the gap between what you see and what happened. It never closes it.

    Go deeper →
  • Did this campaign create incremental growth? Estimate

    A holdout: geo, user, or cluster, depending on what you can withhold cleanly. Settle the design before the estimator. If you cannot construct a credible control, no model recovers one for you.

    What it will not tell you Tell you the effect at a spend level you did not test. A holdout measures the campaign you ran, not the one you are planning.

    Go deeper →
  • Why do my attribution models disagree? Describe

    They are different rules for splitting credit across the same observed journeys. None of them estimates a counterfactual, so disagreement between them is the expected result.

    What it will not tell you Be fixed by choosing a better model. No attribution model becomes causal by becoming more sophisticated.

    Go deeper →
  • How do paid search and SEO interact? Explain

    Pause or geo-test branded paid search and measure total branded conversions, not paid conversions. You are asking how much paid captured demand that organic would have caught anyway.

    What it will not tell you Be answered from channel reports. Both channels count their own conversions, and the overlap is invisible to each of them.

    Go deeper →
  • Can I use a marketing mix model? Explain

    With two or more years of history, real variation in spend, and experiments to calibrate against, yes, for budget-level questions. MMM is a strong allocation tool and a weak causal one.

    What it will not tell you Identify a causal effect on its own. Bayesian priors communicate uncertainty. They do not create identification that the variation in your data cannot support.

    Go deeper →
  • Where should the next dollar go? Decide

    Marginal return, not average return. You want the slope of the response curve at your current spend, ideally anchored by an experiment run at that spend.

    What it will not tell you Be read off a ROAS table. The channel with the best historical average is often the one closest to saturation.

    Go deeper →
  • Which outcome should my campaigns optimize toward? Decide

    The deepest event that stays reliable, timely, and frequent enough to train on. Those three pull against each other, and the right answer moves as your volume grows.

    What it will not tell you Be settled by picking the most valuable event. Optimizing toward a sparse, delayed signal produces worse delivery than optimizing toward a good proxy.

    Go deeper →
  • Who should receive a nurture campaign, or no action at all? Decide

    The people whose outcome would change because you contacted them. That is an uplift question, and it needs a randomized holdout to train and validate against.

    What it will not tell you Be answered by a propensity model. High propensity finds people likely to convert, which often means people who needed no contact at all.

    Go deeper →

Selected work

What this looks like carried through

Each of these runs a question to a recommendation, states its uncertainty, and says what it does not establish. Worked examples use simulated or public data and label which on every page.

25 more pieces across the four disciplines .

Why this is difficult

Measurement is a sequence

Each rung depends on the one below it. Attribution and incrementality are not competing answers to the same question; they sit at different rungs and answer different ones.

  1. 01

    Observe

    What happened?

  2. 02

    Describe

    Where did conversions appear to come from?

  3. 03

    Estimate

    What changed because of marketing?

  4. 04

    Explain

    How did channels and outside factors contribute?

  5. 05

    Decide

    What should happen next?

  6. 06

    Learn

    What uncertainty should we reduce next?

Four things make that sequence hard, and only one of them is a statistics problem. Teams that treat all four that way buy better models and stay stuck.

Tracking uncertainty

Consent, identity resolution, and cross-device loss make the events you observe a biased sample of the events that happened.

Operational uncertainty

Campaigns launch late, audiences overlap, exclusions fail quietly. What ran is rarely quite what was designed.

Causal uncertainty

You never observe the counterfactual. Every estimate of it rests on assumptions you can check but cannot prove.

Decision uncertainty

A clean estimate still has to survive margin, capacity, seasonality, and what the business is willing to do.

Know what you are holding

Different questions require different evidence

Three of these seven support a causal claim. The rest are useful, often more useful day to day. Calling them causal is where measurement programs go wrong.

Evidence Answers Causal?
Attribution Which touchpoints preceded the conversions I can see? A rule for splitting credit across observed journeys. Descriptive
Experiments What changed because of this intervention? The counterfactual is built by design, not assumed. Causal
Quasi-experiments What changed, where randomization was not possible? Causal only if the identifying assumptions survive diagnostics. Causal
Marketing mix models How did channels and outside factors contribute over time? Allocation-grade evidence. Causal only once calibrated to experiments. Descriptive
Predictive ML Who is likely to convert, activate, or churn? Ranks people by likelihood, not by persuadability. Descriptive
Causal ML Whose outcome would change if we acted? Needs randomized variation to train and validate against. Causal
Decision systems Which action, if any, should each customer receive? Turns estimates into policy. Judged by holdout, not by accuracy. Descriptive

Beyond the click

Acquisition is the easy half

A click is the start of a customer. The harder question is what happens next: which lead gets nurtured, which customer gets a retention offer, which account gets a rep's time, and who should receive nothing at all.

That path runs from rules to predictive models to uplift to constrained automated decisioning. Every step still needs logged decisions, holdouts, and consent. Automating a decision does not exempt it from measurement.

Explore the decisioning track →

The optimization signal ladder

  1. Click Fast and dense, and almost never what you want to buy.
  2. Lead Cheap to generate, easy to generate badly.
  3. Qualified lead Usually the right target for sales-led programs.
  4. Activation The product-led equivalent. A strong proxy for value.
  5. Purchase Reliable, but often too sparse for delivery algorithms.
  6. Retention Slow. Good for evaluation, rarely for optimization.
  7. LTV The right target, at the wrong latency.
  8. Incremental value What you want. Requires a holdout.

Systems

What makes evidence repeatable

Difference-in-differences is published and well understood. What most organizations lack is the plumbing that lets you apply it repeatedly, on a deadline, without breaking consent or colliding with another campaign. Each of these is a standing system for producing evidence about one kind of decision, and what separates them is the unit the business will let you withhold.

  1. 01

    Lifecycle Measurement

    Withhold: the person or account

    11 components

  2. 02

    Growth Experiments

    Withhold: the eligible visitor

    7 components

  3. 03

    Performance Marketing

    Withhold: the market

    8 components

  4. 04

    Organic Search

    Withhold: the page

    6 components

  5. 05

    Sales-Led Pipeline

    Withhold: the account or territory

    6 components

See the systems →

Four disciplines

Where the work lives

Attribution

Can we trust what we observe?

Tracking loss, identity, consent, event definitions, funnel grain, and attribution as descriptive evidence. Everything downstream inherits the flaws you accept here.

Incrementality

Did marketing change the outcome?

Counterfactuals, experiments, quasi-experiments, and the diagnostics that decide whether an estimate deserves to be believed. Causal ML sits here too, where the question moves from an average effect to whose effect differs.

Allocation

Where should the next dollar go?

Marginal rather than average returns, channel interaction, marketing mix modeling, and budget decisions made under real uncertainty.

Decisioning

What should happen after the click?

The optimization signal ladder, lifecycle experimentation, uplift modeling, and the systems that turn estimates into governed decisions. Where causal ML stops being an estimate and becomes a policy.

Who writes this

Reid Rhodes. Everything here comes from answering these questions with imperfect data and a deadline, which is the only condition they ever come up in.

Worked examples use public or simulated data and say which, on every page. No result here is presented as a real company's unless it is one, and labeled that way.

Get in touch LinkedIn More about this site →

What is here

Lessons
25
Worked examples
4
System components
38

Every lesson ends with what it does not establish, every worked example carries an interval, and every component names the decision it encodes and who owns it.