Systems

What makes evidence repeatable?

A system is a recreatable process for producing evidence about one kind of marketing decision, built so that an answer survives the quarter it was produced in and the next question costs less to answer because of the last one. There are five here, and what separates them is the unit the business will let you withhold treatment from: a customer, a visitor, a market, a page, an account. The design, the data grain, the estimator, and how long you wait all follow from that.

Show the data

System	One readout	Readouts a year	First readout detects	After a year
Growth	A signup-flow test on 60,000 visitors a week, split evenly, on a 5% activation rate	50	10.0%	1.4%
Lifecycle	A weekly campaign to 20,000 eligible people against a fresh 10% campaign control each week, on a 5% conversion	50	28.8%	4.1%
Performance	A geo test on 40 matched markets, half dark, read over a six-week window with a market-level noise of 10%	4	8.9%	4.4%
Organic	A template split across 4,000 listing pages, worth 459 equal ones once traffic concentration is counted, with page-level noise of 35%	3	9.2%	5.3%
Sales-led	A staged rollout across 24 territories, worth 20 equal ones once pipeline concentration is counted, with territory-level noise of 30%	2	37.9%	26.8%

Five systems, one year, and the smallest lift each could detect by the end of it, relative to the outcome each one measures. Each line starts where its first readout lands and steps down every time the loop closes, because pooled readouts sharpen the program-level estimate by the square root of their count. The two systems that randomize people finish in front. Growth runs a signup test on sixty thousand visitors a week and ends near 1.4%. Lifecycle starts near the back, since a weekly campaign against a ten percent holdout detects only a 29% lift on its own, and ends at 4.1% because it read 50 times while performance read four, organic three, and sales-led twice. Pooling assumes the program effect held still across the year and that each readout is an independent cohort, which is the condition the standing holdouts and the holdback state; where the same customers recur across weeks, information grows more slowly than the count. This is the precision of each program's pooled average effect, not the power of any single next test: fifty unrelated tests do not make the fifty-first sharper. Lifecycle's 29% relative lift on a five percent base is about 1.4 points. Arithmetic on stated designs · scripts/loop_cadence.py

Every system here is a machine for buying evidence, and the price is how much business-as-usual the organization will let you withhold. That appetite is the binding constraint, not the statistics. It decides what can be randomized, which decides the design, which decides everything after it. Growth experiments, the fifth, is the loop the other four approximate under scarcer units, and it is here because the surfaces before purchase are the marketing team's to measure. The figure above shows what that buys over a year: where each system's first readout lands, and how far its loop carries it by the end.

The reason to build one, rather than answer the question once, is speed with rigor. A marketing team should be able to launch, read, and act without waiting on an analysis, and the answer should be as sound as if a statistician had produced it, because one did, once, when the system was built. That takes three parts: a standing comparison so nothing has to be negotiated per test, decisions logged in a form the next readout can score, and a policy that consumes the estimate so the evidence changes behavior without a meeting. The data scientist moves from working the queue to building and reviewing what it produces. Lifecycle has all three parts and closes weekly. The others close slower, each hub says how much slower and what is missing, and the missing parts are the roadmap. Growth closes weekly too, on the surfaces before purchase, and its rigor lives in the metric ladder, the stopping rule, and the holdback rather than in the difficulty of getting a test approved.

What the five share

The same moves recur in every one of them, under different names. A standing comparison, so a readout never waits on a negotiation: a persistent holdout of customers, a holdback of visitors, a rotating set of dark markets, a reserved slice of rep capacity. Decisions logged with the probability they were chosen at, so a candidate policy can be scored on what already happened. The effect as assigned first and the effect on those reached second, with the exposure rate that connects them recorded rather than assumed. A registry that turns every result into the prior for the next design. And a review that is allowed to say nothing was learned, because a system that cannot say that will say something else instead.

Which system first

The five are not peers for any one business. The right first system is the one whose unit you already control and whose loop closes fastest on the decisions you make most often.

Subscription and consumer products with a list

First Lifecycle Measurement · then Growth Experiments

The contactable customer is the unit you own outright, the holdout is nearly free, and every campaign is already a test waiting for its control. Growth follows once the product surfaces before purchase are yours to change.

What else changes here →

Product-led software

First Growth Experiments · then Lifecycle Measurement

Signup, onboarding, and the upgrade prompt are where the business is decided, traffic is high, and a flag tool already exists. Lifecycle comes second, for the nurture and winback surfaces the product does not reach.

What else changes here →

Retail and omnichannel

First Performance Marketing · then Lifecycle Measurement

The market is the natural unit and offline outcomes are observed there, so a geo design is the strongest thing available. The loyalty list makes lifecycle the second build.

What else changes here →

Marketplaces

First Growth Experiments · then Organic Search

Both sides arrive through surfaces you control, and the two sides interfere, so the growth system with the market as its unit is the first thing to get right. Organic follows, because the listing pages are the acquisition channel nobody can switch off.

What else changes here →

Sales-led B2B

First Sales-Led Pipeline · then Lifecycle Measurement

The account is the unit and sales owns it, so the first job is machinery that produces a comparison from something nobody is protecting: staged rollouts and a prioritization log. Lifecycle handles the nurture that reaches contacts a rep never will.

What else changes here →

Paid-media-led acquisition

First Performance Marketing · then Growth Experiments

Most of the budget is bought media on channels that cannot address a person, so the geo design and the calibration loop are where the money is. Growth follows on the landing and signup surfaces the media sends people to.

What else changes here →

What is not here, and why

A system earns a place by five checks: it runs continuously with named owners, every question after it is cheaper, its last stage feeds an earlier one, it has a unit it is allowed to withhold, and it maps to a scope someone hires for. These were considered and failed one.

Brand and upper-funnel measurement: Borrows the market. Brand lift and long-term effects are performance’s geo design read on a survey or a slow outcome, which makes them a component performance should carry rather than a unit of their own.
Pricing and promotions: Borrows three units. A price test on the pricing page is a growth test, a targeted offer is a lifecycle test, a mass promotion is a geo test. Decisions served by existing systems, not a system.
Creative testing: Runs inside the ad platform on platform-reported outcomes, with no counterfactual of its own. It belongs in the spend spine as a dimension, and its results are only as good as the geo design that checks them.
Store test-and-learn: The closest call. Retailers run hundreds of matched-store tests a year on standing platforms, and the practice passes every check. The store is the market at a finer grain, same design and same arithmetic, and most of what gets tested is merchandising rather than marketing.
PR, events, sponsorships, influencers: Nothing can be withheld, so there is no standing comparison to build around. Measurable occasionally by geography or timing, and then as a performance test rather than a system.
The measurement platform itself: Identity, event definitions, and pipelines are what every system stands on, and a service rather than a system: nothing is randomized and no loop closes.