Skip to content
Was It Causal?

Audience Builder and Capacity Planning

Audience definitions as versioned, reusable objects instead of one-off SQL, and the capacity check that decides whether a test was ever going to work.

Foundation Shared B2C subscription B2B SaaS Lifecycle-led Marketplace

The decision it encodes

Who is eligible for this campaign, and can we actually serve them?

Almost every lifecycle program starts the same way. Someone writes a query that pulls the people who should get the campaign. It works. The campaign ships.

Six months later there are forty of those queries. Nobody can say which two overlap. The definition of “active” differs between three of them, and none of them matches the definition in the dashboard the VP looks at. When a test comes back flat, there is no way to tell whether the idea was wrong or whether the audience was wrong, because the audience no longer exists as an object anyone can inspect.

That is the failure this component prevents. Not through better SQL, but by making an audience a thing rather than a query someone ran once.

The decision it encodes

Who is eligible for this campaign, and can the operation actually serve them inside the window the business will wait?

Those are two questions and most teams only answer the first. The second is where lifecycle programs quietly die.

What an audience has to carry

A usable audience definition is not a list of customer IDs. A list goes stale the moment it is written and tells you nothing about why anyone is on it. The object needs to carry the reasoning.

FieldWhy it exists
DefinitionPredicate over customer state, not raw event tables
VersionDefinitions change. Old results must stay interpretable
Consent basisWhich lawful basis and channel permission this relies on
SuppressionsGlobal opt-outs, complaints, bounces, active support tickets
ExclusivityWhich other audiences this may not overlap with
Refresh cadencePoint-in-time or continuously re-evaluated
OwnerA person, not a team inbox

The version field carries more weight than it looks like it does. When someone asks in November why the September test showed a smaller effect than the July one, “the audience definition changed in August” is either an answer you can give in thirty seconds or one you cannot give at all.

The capacity question

Here is where most audience work stops and most programs go wrong.

You define the audience. It has 480,000 eligible people. You hold out 10%, split the rest across two creative variants, and the power calculation says you can detect a 4% relative lift. The design doc gets approved.

Then the campaign goes to marketing ops, who can send 40,000 emails a week without hurting deliverability. Six weeks of runway means 240,000 sends. You had 432,000 treatable people. You will reach just over half of them, and the effect you can actually detect is not 4%.

Nobody made a mistake. The statistics and the operation were planned in separate rooms.

Your audience

After consent and suppression, not your whole database.

Percent held back and contacted with nothing.

%

Variants tested against the holdout. More cells, less power each.

What the holdout is expected to do on its own.

%

What ops, deliverability, or headcount can actually serve.

How long the business will wait for a readout.

Power 80%. Alpha 0.05, Bonferroni-corrected across cells.

Capacity binds

You can serve 240,000 of 432,000 treatable people in 6 weeks. Serving the full audience would take 10.8 weeks.

Holdout
48,000
Per cell (served)
120,000
Total treated
240,000
Detectable lift
9.2%
If capacity were free
8.6%
Incremental conv. at MDE
703

Read it this way

Capacity is the binding constraint here, so adding people to the audience will not improve the test. Either extend the window, raise throughput, or cut a cell.

The number that matters in that panel is not the detectable lift. It is which constraint binds. When capacity binds, growing the audience does nothing, because you cannot reach the people you already have. Teams that miss this spend months expanding eligibility criteria and wondering why their tests stay underpowered.

Contact fatigue is a shared budget

One more constraint, and the one most often discovered too late.

Contact limits are a property of the customer, not of the campaign. If a person can receive four marketing emails a month and three programs each claim two, someone loses, and which someone depends on the order the jobs happened to run.

That makes contact capacity a shared budget requiring an allocation rule, which is a decision the system has to encode explicitly. Options in rough order of maturity:

  1. First come, first served. Whichever campaign resolves first wins. Simple, and it silently biases toward whoever schedules earliest.
  2. Static priority. Retention beats winback beats cross-sell. Predictable, and stale within a quarter.
  3. Expected value per contact. Each campaign bids with predicted incremental value. Correct in principle, and it needs uplift estimates you probably do not have yet.

Most teams should start at 2 and be honest that they are there. The failure mode is running 1 while believing you are running 2.

Maturity, honestly staged

Nobody builds this in one quarter. The useful question is which stage you are at and what the next one buys.

Stage 1: definitions in version control. Audiences move from ad hoc queries into reviewed, versioned files. No new capability, but results become interpretable six months later. Most of the value is here and most teams skip it.

Stage 2: shared eligibility and suppression. One consent and suppression layer every audience reads. Removes a whole class of compliance incident and makes overlap visible.

Stage 3: capacity-aware planning. Audience size, send capacity, and contact limits checked together before launch rather than discovered at readout.

Stage 4: allocation across competing campaigns. Contact budget assigned by expected incremental value. Requires uplift estimates and a working next-best-action layer, so it comes last for good reason rather than by neglect.

The capacity planner's formulas Technical

Detectable lift uses a two-proportion comparison between the holdout and one treatment cell. For holdout size n₀, per-cell size n₁, and baseline rate p:

SE      = √( p(1−p) · (1/n₀ + 1/n₁) )
MDE_abs = (z_1−α/2 + z_power) · SE
MDE_rel = MDE_abs / p

With k treatment cells compared against one control, alpha is Bonferroni-corrected to 0.05 / k. That is conservative. It is also better than the common alternative of running k comparisons at 0.05 and declaring the best one a winner.

Served population is min(treatable, weekly capacity × weeks). The planner reports detectable lift at both the served and the full population, because the gap between those two numbers is the cost of the capacity constraint, stated in the units the design doc used.

Two simplifications worth knowing. The calculation assumes everyone assigned is reached, which overstates power wherever deliverability or open rates intervene. Using an exposure-adjusted population is more honest, but that needs exposure logging in place first. It also treats weekly sends as reaching distinct people, which fails if the campaign contacts the same person repeatedly.

Context

Best suited for

Lifecycle, CRM, and retention programs that run repeatedly against overlapping populations, where the same person is eligible for several campaigns at once.

Data requirements

  • A customer state model with one row per person or account
  • Consent and channel permission resolvable at the individual level
  • Contact history, to enforce fatigue caps rather than estimate them
  • A send or delivery capacity figure from whoever actually operates the channel

What changes by context

B2C subscription
Population is large and capacity is rarely binding for email, so power is usually the constraint. Fatigue caps bind first, and they bind across programs rather than within one.
B2B SaaS
The unit is an account, not a person, which cuts effective sample size by an order of magnitude and makes most cell splits impossible. Rep capacity replaces send capacity as the hard limit, and it is far scarcer.
Lifecycle-led
Audiences overlap constantly, so exclusivity rules and the contact budget matter more than the definitions themselves. This is where an allocation rule stops being optional.
Marketplace
Supply and demand audiences need separate builders and separate holdouts. Treating a seller campaign as a buyer campaign with different filters is how spillover gets built in.

When it breaks down

  • Populations under roughly 20,000, where any usable holdout leaves cells too small to detect a lift worth acting on.
  • Very low baseline rates, where the detectable lift exceeds anything a campaign could plausibly produce.
  • Channels where you cannot suppress reliably, since a holdout that receives the campaign is worse than no holdout.
  • Organizations without a shared customer state model. Build that first; audiences on top of raw events will drift no matter how well this component is specified.