All templates
Growth testing · monthly · cancel any month
Most of what we try will not work. The job is finding that out in three weeks instead of three quarters.
A testing retainer sold on its wins is being sold backwards. The wins are rare by design. What you are actually buying is a rate of killing things, an argument you no longer have to have internally, and a written record of what has already been ruled out.
Killed
Ran, lost, written up, not coming back. This is the biggest column on every board I have ever kept and a page that hides it is hiding the job.
Referral scheme — Two rounds, both flat. Killed at day 19
Second onboarding email — Opened, ignored, no change downstream
Pricing page rewrite — Slightly worse. Reverted the same week
Second onboarding email — Opened, ignored, no change downstream
Pricing page rewrite — Slightly worse. Reverted the same week
Kept
Beat the agreed number and stayed beaten a month later. Rare, and worth the rest of the column to its left.
Shorter signup — Three fields instead of eight
Reply-to a person — Not a no-reply address
Reply-to a person — Not a no-reply address
Still running
No verdict yet, and deliberately no opinion in public until there is one. Replace all three columns with your own — an invented board is worse than no board.
Annual plan on the pricing page — Day 8 of 21
Weekly digest instead of daily — Day 15 of 21
Weekly digest instead of daily — Day 15 of 21
How a test is run
Six rules, and they are the whole method. They exist to stop the two failure modes: running things that could never lose, and keeping things because somebody senior liked them.
It has to be able to fail — If there is no result that would make us stop, it isn't a test, it's a launch with a nicer name. Those are fine — they just don't come out of this budget
The number is agreed before it starts — One number, written down, with the size of change that counts. Agreeing it afterwards is how everything becomes a modest success
Three weeks, then a verdict — Long enough to mean something, short enough that nobody has fallen in love with it. A test still running at week five has usually already told us
A tie is a kill — No result means no change, and no change means we take the simpler version. Ties are where retainers go to quietly waste a year
Nothing gets re-run because somebody liked it — It can be re-run when something real has changed — a new audience, a new price, a new season. "We should give it another go" isn't one of those
The record is yours — Every test, including the ones that lost, in a document you keep. If you hire somebody else next year, that list is the most useful thing you own
The number is agreed before it starts — One number, written down, with the size of change that counts. Agreeing it afterwards is how everything becomes a modest success
Three weeks, then a verdict — Long enough to mean something, short enough that nobody has fallen in love with it. A test still running at week five has usually already told us
A tie is a kill — No result means no change, and no change means we take the simpler version. Ties are where retainers go to quietly waste a year
Nothing gets re-run because somebody liked it — It can be re-run when something real has changed — a new audience, a new price, a new season. "We should give it another go" isn't one of those
The record is yours — Every test, including the ones that lost, in a document you keep. If you hire somebody else next year, that list is the most useful thing you own
What a month looks like
One planning hour
What we're testing next and why, argued out loud. You can veto anything with no explanation and that costs nothing.
Three or four tests live
Staggered, so a verdict lands most weeks rather than all of them arriving on the last Friday.
A written verdict each time
What ran, what the number did, and what happens now. Two paragraphs, not a deck, and it goes to whoever wants it.
No standing status call
There is nothing to say on most Mondays. Where something needs a conversation, it gets one on the day it needs it.
What this isn't, and when it should stop
It isn't an agency — Nobody is making the assets, writing the campaigns or managing the channel. Where that is the actual need, an agency will do it better and cheaper than I would
It isn't media buying — I don't run ad accounts and I don't take a percentage of spend. A percentage of spend pays me more for spending more, which is the wrong incentive to hand anyone
It isn't a growth promise — There is no number here that I can honestly commit to in advance, and anybody who does is quoting a number from someone else's business
It has an end date in it — Twelve months, written into the agreement rather than left for one of us to raise. If the useful tests have run out sooner, that is the honest time to stop
It isn't a content subscription — No fixed number of posts, emails or pages per month. Months differ, and a deliverable count would just guarantee busywork in the quiet ones
No notice period — Cancel in any month and it ends at the end of that month, with the record handed over. A retainer that needs ninety days' notice is being kept by paperwork
It isn't media buying — I don't run ad accounts and I don't take a percentage of spend. A percentage of spend pays me more for spending more, which is the wrong incentive to hand anyone
It isn't a growth promise — There is no number here that I can honestly commit to in advance, and anybody who does is quoting a number from someone else's business
It has an end date in it — Twelve months, written into the agreement rather than left for one of us to raise. If the useful tests have run out sooner, that is the honest time to stop
It isn't a content subscription — No fixed number of posts, emails or pages per month. Months differ, and a deliverable count would just guarantee busywork in the quiet ones
No notice period — Cancel in any month and it ends at the end of that month, with the record handed over. A retainer that needs ninety days' notice is being kept by paperwork
What it costs
The lab
$4,900 / month
- Three or four tests live
- A written verdict on each
- One planning hour a month
- Cancel any month, no notice
Half lab
$2,600 / month
- One or two tests live
- For teams doing the build themselves
- Same rules, same record
- Cancel any month, no notice
The setup
$3,200 · one off
- What can be measured, honestly
- What has already been ruled out
- A first quarter of test ideas
- No retainer attached to it
The setup is a real option — Some places should buy that and stop. If the measurement isn't there yet, a testing retainer is three months of guessing with a schedule
No percentage of anything — Not spend, not revenue, not saved budget. A flat monthly is the only version where my advice and your interest point the same way
Tools are yours and stay yours — Bought in your name, on your card, cancelled by you. Nothing routes through me and nothing pays me a referral fee
Smaller budgets — Under roughly ten thousand a month in total marketing spend, take the setup and run the tests yourselves. The retainer would eat the thing it is meant to improve
No percentage of anything — Not spend, not revenue, not saved budget. A flat monthly is the only version where my advice and your interest point the same way
Tools are yours and stay yours — Bought in your name, on your card, cancelled by you. Nothing routes through me and nothing pays me a referral fee
Smaller budgets — Under roughly ten thousand a month in total marketing spend, take the setup and run the tests yourselves. The retainer would eat the thing it is meant to improve
Ask a question
No call needed to start with. Send the question and you'll get a straight answer, including when the answer is that this isn't the right thing to buy yet.
Ask a question
The second half of the first question is the important one. Testing on numbers nobody believes produces confident, expensive nonsense, and that is worth an honest conversation before any money changes hands.
Live preview — every color, word and block is editable after you pick it.