Context · Growth motion
Lifecycle-led
Existing customers, repeated contact, and a contact budget that runs out.
What changes here
The same four questions, and what each one runs into in this context.
Attribution
Sends are easy to attribute and easy to over-credit, because the audience was selected for propensity in the first place.
Incrementality
Persistent holdouts are cheap here and pay for themselves. There is little excuse for not having one.
Allocation
The budget is contact capacity and customer patience, not media spend.
Decisioning
The home of uplift. Targeting propensity spends the contact budget on people who needed nothing.
Written for this context
Incrementality
-
Choosing Between User, Geo, and Cluster Holdouts
The design decision is about what you can withhold cleanly, not about statistical efficiency. A procedure for picking the unit, in which power is the last question rather than the first.
-
Prediction Is Not Persuasion
A model that knows who will convert does not know who you can convince. The two questions have different answers, and only one of them can be checked against data you already have.
-
When the Experiment Leaks
Noncompliance, contamination, and spillover are three different leaks that bias in different directions. Intention to treat survives two of them, and the third is only fixed by the design.
Decisioning
-
From Segment to Audience to Treatment Policy
Segments, audiences, scores, assignments, and policies are five different objects. A result is only comparable to a previous one if you can say which of the five changed.
-
The Case for Doing Nothing
Contact fatigue, capacity, and cost mean no-action is a real treatment arm. Why it belongs in every decisioning system by default, and why organizations reliably leave it out.
-
Uplift Modeling in Practice
Where to cut the ranking, how much to withhold, and why the offline number is larger than the money. Estimating heterogeneous treatment effects when you have a randomized holdout, and validating the policy prospectively.
-
Why Agentic Marketing Still Needs Holdouts
Two agents, the same budget and the same population, differing only in what they are scored on. One improves its dashboard by 65% while destroying value, and the one doing the right thing looks worse.
- Worked example
Broad Nurture, Propensity, or Uplift?
One trial-to-paid nurture sequence, four targeting policies, and a randomized holdout. Compared on incremental conversions rather than model accuracy, which is the only comparison that changes the answer.
Systems
- System
Customer State Model
One row per person or account carrying consent, lifecycle stage, eligibility flags, and contact history. The foundation every audience, experiment, and policy reads from.
- System
Audience Builder and Capacity Planning
Audience definitions as versioned, reusable objects instead of one-off SQL, and the capacity check that decides whether a test was ever going to work.
- System
Contact Permission and Suppression
Lawful basis, channel permission, jurisdiction, purpose, global opt-outs, and complaint handling as one shared layer rather than a filter each campaign reimplements. Consent is one basis among several; suppression is a decision of your own.
- System
Assignment and Experiment Registry
Deterministic hash-based assignment, a program holdout with a scope contract, overlap rules, an outcome contract per experiment, and a registry that stops two campaigns from colliding on the same people.
- System
Exposure and Decision Logging
Logging what was decided and what was delivered, not just what was sent. The component that makes everything downstream measurable at all.
- System
Automated Measurement
A standing readout that runs eligibility, assignment, delivery, exposure, outcome, and guardrails on a schedule instead of as a bespoke analysis each time.
- System
Program Lift and Reconciliation
What the program holdout says the lifecycle policy was worth, reconciled against what the tested actions account for on a common value scale, with the residual reported as a residual. The question a per-campaign readout cannot answer.
- System
Evidence Review
The standing forum where estimates get ranked by what they assumed, a number is promoted to the one the plan uses, and a result is allowed to be inconclusive.
- System
Next-Best-Action Policy
Turning scores into decisions under real constraints: capacity, contact fatigue, margin, cooldowns, and the explicit option of doing nothing.
- System
Treatment Effects and Policy Evaluation
The bridge between a campaign's average effect and a per-customer decision: estimating how the effect differs by customer state, and scoring a candidate policy on randomized or logged data before it is trusted with volume. The component that produces the numbers next best action consumes.
- System
Policy Governance and Safety
Who may change the policy, what it may do, what stops it without a human, and how anyone knows whether it worked, whether the policy is a rules engine, a model, or an agent. The agentic case is the hardest version of the same component.