The Change Roadmap

The audit came back with 340 issues, color-coded by severity. Eleven are critical, sixty are high, and the rest are medium.

The critical ones are duplicate meta descriptions on a paginated archive that receives forty sessions a month. Nothing in the document distinguishes an issue that costs the business something from one that offends a checklist, because the severity scale grades conformance rather than consequence, and conformance is what a crawler can measure.

The decision it encodes

Which template changes next, and what is it worth?

Organic work is unusual in having an effectively infinite backlog and no natural prioritization. Every audit produces more items than anyone can ship, and the ordering that arrives with them is not an ordering by value.

Rank by traffic at stake, times a plausible effect

The value of a change is roughly the traffic it could touch multiplied by how much it could plausibly move it. Both halves are available: the first from the inventory, the second from whatever similar changes have previously returned.

That ordering is different from an audit’s ordering in a consistent direction. It promotes dull changes to large templates and demotes dramatic-sounding findings on pages nobody visits.

It has one structural blind spot, and it should be named. Traffic at stake is zero for a page that does not exist yet, so a roadmap ordered by it will never surface new content, however much demand is unmet. For those items the first factor is demand at stake, the query volume the new pages could plausibly compete for, and the second is a plausible share of it, which is a weaker estimate than a measured effect and should be labeled as one. Finding that demand is the discovery work this system takes as an input rather than describes.

What each item on the roadmap carries

Field	What it decides
Template and pages	What the change would reach, from the inventory rather than from an estimate.
Traffic at stake	Sessions on those pages. Half the ranking.
Plausible effect	What comparable changes have returned, with an interval. The other half.
Testable	Whether the template can be split, which decides whether this produces evidence or only an outcome.
Effective pages	Concentration, which decides whether a split on it could detect the effect being claimed.
Engineering cost	The denominator, and the reason a large win nobody can build stays below a small one somebody can.

Ship it as a split, and the roadmap refills itself

This is the edge that makes the system run rather than execute once.

A change shipped to a whole template produces a result nobody can attribute. The same change shipped to half a template produces the same result and an estimate, and that estimate is the “plausible effect” input for the next item of the same kind. The roadmap consumes evidence at the top and generates it at the bottom.

Say when a change cannot be measured, and ship it anyway

Some work is necessary and unmeasurable: a migration, a rendering fix, an accessibility requirement. Those belong on the roadmap with the measurement column reading none, rather than being given a fabricated projection so they compete on the same scale as everything else.

Naming them protects the estimates around them. A roadmap where every item carries a number is a roadmap where some of the numbers were invented, and a reader has no way to tell which.

Maturity, honestly staged

Stage 1: the backlog ordered by traffic at stake. No estimates yet, and it already displaces audit severity as the ordering, which is most of the value.

Stage 2: a plausible effect attached, however weak, with its source. The ranking becomes value rather than volume, and the weakest inputs are visible as weak.

Stage 3: testable and effective-pages recorded per item. Which changes will return evidence is known before they are scheduled.

Stage 4: measured effects feeding back as the priors. The ordering is grounded in this site’s own results, and the roadmap improves without anyone tuning it.

Ordering when most items have no estimate Technical

Early on almost nothing has a measured effect, and ranking by an invented number is worse than ranking by traffic alone, because it launders a guess through an arithmetic operation. The defensible early ordering is traffic at stake, with items grouped by change type, and the first test on each type scheduled to price that type.

Once estimates exist, the value of an item is the traffic multiplied by the effect, and the value of testing it is different: it is highest where the traffic is large and the effect is uncertain, which is the same expected-value-of-information ordering the performance test calendar uses for channels. Where a change would be shipped whatever the estimate said, testing it buys nothing however large it is.

One asymmetry worth building in. Organic changes are cheap to ship and expensive to reverse, since a reversal costs another crawl cycle and can look like instability to a ranking system. That argues for splitting more often than a lifecycle program would, because the split is also the safest way to discover that a change hurt.

Context

Best suited for

Programs with more identified work than capacity, which is all of them, and where the ordering currently comes from an audit tool or from whoever presented most recently.

Data requirements

• The inventory, for pages and traffic per template
• Past test results with the template type they were measured on
• An engineering estimate per item, at whatever precision is available
• A record of which items shipped unmeasured, so the gap in the evidence base is visible

What changes by context

B2C ecommerce

Product and category templates dominate traffic and behave differently, so effects transfer poorly between them. Prioritize measuring each type once before ranking anything by a borrowed estimate.

Marketplace

The listing template is most of the site, so a single test on it prices most of the roadmap. This is the setting where the loop pays back fastest.

B2B SaaS

Few pages and no split available, so the roadmap cannot be grounded in measurement and should say so. Ordering by traffic at stake with unmeasured effects is better than a ranking built on numbers taken from another site.

When it breaks down

• Sites too small to split anything, where the roadmap can be ordered sensibly and never validated.
• Organizations where the roadmap is set by an external agency paid per deliverable, since the incentive is item count rather than traffic at stake.
• Programs shipping several changes to one template at once, which is often the right operational call and destroys the attribution the loop depends on.