Product-led

Defining Events That Survive Contact With Operations

A lead, an activation, and a qualified opportunity are organizational agreements before they are data. What happens to a time series when the agreement changes and nobody writes it down.

The Same Click, Different Funnels

One ad click means something different in a self-serve checkout than in a sales-qualified pipeline. The funnel it enters decides the unit, the window, and the outcome you are allowed to attribute it to.

What Tracking Loss Actually Costs You

Consent refusal, identity resolution, and cross-device loss are not one problem. Separating loss that costs precision from loss that costs validity, and why only one of them is fixable with more data.

Choosing Between User, Geo, and Cluster Holdouts

The design decision is about what you can withhold cleanly, not about statistical efficiency. A procedure for picking the unit, in which power is the last question rather than the first.

Counterfactuals, Estimands, and Saying What You Mean

Before choosing a method, state what you are trying to estimate: which units, which treatment, which outcome, over what horizon. One campaign produces six different numbers, and only one of them is wrong.

Prediction Is Not Persuasion

A model that knows who will convert does not know who you can convince. The two questions have different answers, and only one of them can be checked against data you already have.

When the Experiment Leaks

Noncompliance, contamination, and spillover are three different leaks that bias in different directions. Intention to treat survives two of them, and the third is only fixed by the design.

Where Precision Comes From in a Holdout

Pairing, covariate adjustment, a weighted control, and randomization inference, applied to one geo test in two noise worlds. Most of the levers people reach for buy nothing in a geo test. The design and the inference are what move the interval.

When the Rollout Was Staggered

Markets adopt in waves and the effect grows after adoption. The regression everyone runs on that panel uses early adopters as controls for late ones and understates the effect by a third. What to run instead, and the plot that shows whether it can be believed.

Choosing the Right Campaign Optimization Event

The deepest event that stays reliable, timely, and frequent enough to train on. Alignment sets a ceiling you cannot buy your way past, precision decides how close you get, and only one of them improves with volume.

Why Agentic Marketing Still Needs Holdouts

Two agents, the same budget and the same population, differing only in what they are scored on. One improves its dashboard by 65% while destroying value, and the one doing the right thing looks worse.

Broad Nurture, Propensity, or Uplift?

One trial-to-paid nurture sequence, four targeting policies, and a randomized holdout. Compared on incremental conversions rather than model accuracy, which is the only comparison that changes the answer.

Customer State Model

One row per person or account carrying consent, lifecycle stage, eligibility flags, and contact history. The foundation every audience, experiment, and policy reads from.

Assignment and Experiment Registry

Deterministic hash-based assignment, a program holdout with a scope contract, overlap rules, an outcome contract per experiment, and a registry that stops two campaigns from colliding on the same people.

Exposure and Decision Logging

Logging what was decided and what was delivered, not just what was sent. The component that makes everything downstream measurable at all.

Treatment Effects and Policy Evaluation

The bridge between a campaign's average effect and a per-customer decision: estimating how the effect differs by customer state, and scoring a candidate policy on randomized or logged data before it is trusted with volume. The component that produces the numbers next best action consumes.

Policy Governance and Safety

Who may change the policy, what it may do, what stops it without a human, and how anyone knows whether it worked, whether the policy is a rules engine, a model, or an agent. The agentic case is the hardest version of the same component.

Surfaces and Assignment

Which parts of the product the system covers, how a visitor is assigned to an arm before they have an account, and how that assignment survives the moment they get one. The component where contaminated results start.

Units Above the Visitor

When the visitor is the wrong thing to randomize: teams that must see one price, marketplaces with two sides, referral loops, and shared devices. The component that decides how much sample a clean comparison costs.

The Metric Ladder and Guardrails

One metric that decides, a short list that can veto, and a record of how each proxy earned its place. The component that stops a fast loop from optimizing the top of the funnel at the expense of the bottom.

Test Design and Traffic

Which test may launch now, on whom, alongside what, at what allocation, for how long, and under what ramp and kill rule. The component that turns a stream of individual tests into a portfolio that can be run at once without the tests reading each other.

Sequential Readouts

When a test is allowed to stop, what it reports when it does, and the checks that run before anyone reads a lift. The component that makes a fast loop trustworthy, because reading a test daily and stopping when it looks good inflates a five percent false-positive rate to about a quarter.

Shipping Rules and the Registry

The rule that turns a readout into a decision, applied the same way to every test, and the registry that records what was asked, what was found, and what was done. The component that makes the tenth quarter cheaper than the first.

The Long-Run Holdback

A small standing share of visitors who never see any shipped change, so that once a quarter the sum of the program's wins can be checked against what the program added. The component that keeps a fast loop pointed at the outcome.