What makes an early growth activity an experiment?
It has a question, a reason the activity might help answer it, a planned observation, and a decision you will revisit. Publishing three posts is an activity. Testing whether a specific audience understands and attempts a demonstrated workflow gives the activity a purpose. Choose the unresolved step before committing the time or money.
Start from the audience hypothesis and channel choice. You may need to learn whether suitable people can be reached, whether they understand the promise, or whether the product completes the task. Those questions need different observations. More clicks will not diagnose a failure that occurs after a relevant user starts the workflow.
Write a brief before the first invitation or publication
A short brief keeps the question stable while the work becomes busy. Name the audience, the task, what you will do, and what would make you continue, revise, or stop. Add the source of the data and the time needed before the outcome can be observed. Leave assumptions visibly marked instead of giving them the appearance of measured facts.
This fictional pilot concerns a booking app for tutors. It illustrates a plan for learning, not evidence that the activity will acquire users. The brief deliberately chooses a workflow observation rather than promising a fixed number of purchases.
| Field | Illustrative choice |
|---|---|
| Question | Can tutors who agree to a trial complete a sample booking from the guide? |
| Mechanism | A relevant example explains the task and leads to a matching demo |
| Action | Prepare one tested guide and invite suitable participants in an appropriate setting |
| Primary observation | The defined booking result, with assistance recorded |
| Budget | $40 cash and three hours of founder time, both tracked separately |
| Guardrail | Pause recruitment if a reproducible product failure blocks the trial |
| Review | Review completed sessions after the agreed week; keep later outcomes pending |
How should you account for founder time?
Record cash outlay and time as separate facts. If useful for planning, choose an explicit hourly value for your time and show the calculation. That planning value is an assumption, not an invoice or a universal founder wage. Include preparation, responses, support, and review rather than counting only the time spent publishing.
For the invented pilot, $40 cash plus three hours valued at $30 per hour gives a time-inclusive planning cost of $130. Suppose twelve relevant invitations lead to four replies, three agreed trials, and two completed tasks, one assisted. These numbers are fictional. No paid offer was made in this round, so the observations do not test willingness to pay.
| Calculation | Result | Interpretation |
|---|---|---|
| Cash divided by two completed tasks | $20 per completed task | Cash efficiency for the observed trial outcome |
| $130 planning cost divided by two completed tasks | $65 per completed task | Includes the stated founder-time assumption |
| Cost per new paying customer | Not established | There was no paid offer or observed purchase in this pilot |
Why is a small pilot different from an A/B test?
A before-and-after change or a few manually recruited trials is not automatically a randomized controlled experiment. Different participants, timing, tasks, or product versions can explain the difference. Record those conditions and describe what you observed. Do not label a change statistically significant simply because one small group completed more tasks.
Kohavi, Deng, and Vermeer’s research on A/B testing documents common intuitive misunderstandings in online experiments. Its lesson for this planning exercise is to treat formal experiment design as real work. If you need a causal estimate, use an appropriate design and sample plan. If you are investigating an early workflow, a carefully documented pilot can still inform the next decision without making that stronger claim.
Which observation should be your main success measure?
Choose the observation that answers the question. For the fictional tutor pilot, the defined booking result matters more than a guide view. The activation guide helps specify that result. Keep supporting stages such as replies, accepted trials, and setup failures visible, because they explain where the journey changed.
A primary measure does not make the other effects irrelevant. Record support burden, unsuitable participants, failures, and any complaints that affect whether the activity is worth continuing. Avoid changing the definition after seeing the results solely to make the pilot appear successful. Save the original brief with the review.
How do you interpret a pilot that gets no useful result?
Trace the observed stages. If nobody suitable agreed to participate, investigate recruitment and the invitation. If suitable people started but could not complete the task, inspect the actual failure. If the task worked but nobody needed it again, investigate frequency and fit. A zero at the final stage can have several explanations, and missing tracking is different from an observed zero.
Use the attribution guide to keep known sources and unknown journeys separate. The feedback guide helps decide whether the next response is a product fix, explanation, or different audience. Keep the diagnosis tied to evidence rather than replacing the whole channel after one disappointing week.
When should you review or stop the work?
Set the review window from the activity. A session can be discussed after it happens; organic search needs a longer observation plan. A weekly review of a new article may reasonably inspect publication quality and crawl access while acquisition remains unresolved. A budget limit controls spending, but it does not by itself establish that a sample is sufficient for a statistical conclusion.
Pause immediately when a defined guardrail is breached, such as sending people into a known broken flow. Otherwise, review at the planned point and record why you continue, adjust, or close the pilot. Add another activity when it answers a distinct question and you can support it. The weekly founder plan helps protect that capacity.
Keep the learning reusable
Save the brief, offer and product versions, source references, observed counts, costs, assistance, unknowns, and next decision. A later founder or operator should be able to understand why the experiment changed. Reusing a clear decision record is more helpful than remembering that a particular channel worked once.
Use the worksheet for one pilot at a time. Link it to the actual guide, newsletter issue, or partnership example being tested. The useful output is a better-supported next action and a clearer account of the resources spent, even when the activity does not yet produce a customer.