-
Sell it before you build it, or build what no customer could ask for?
Sam Parr
calls everyone who might buy, confirms the problem without pitching, then asks for money on the same call. He sold Trends and Hampton before either existed, and he reads a buyer who balks at a real offer as a problem that is not big enough.
Y Combinator partners
argued on YC's Startup Ideas You Can Now Build With AI episode that sell-before-you-build is outdated for AI products. A model can now produce output no customer would think to ask for. Their advice is to work with the newest technology and follow your curiosity toward what it can now do.
The split is whether the buyer can picture the product. Parr's businesses answered known problems, a peer group for CEOs and a research newsletter, where a 20-minute call surfaces the need. For a capability that did not exist last year, the person on the phone can only describe the old workaround. If the problem is known, sell first. If the capability is new, build first.
From Presell Calls
-
A reconstructed story, or a precise formula?
Bob Moesta
runs the tactical, qualitative lineage of Jobs to Be Done: interview people who actually switched, surface the forces around the switch, and study the struggling moment, because the causal story of one real switch teaches more than a tidy metric.
Tony Ulwick
runs the outcome-driven lineage, treating the job as a quantifiable statement with a direction of improvement, a metric, an object and a context, stable and testable at scale. The rigour is the point, because a fuzzy statement cannot be prioritised.
Both call themselves JTBD, which is why founders reading two sources come away confused. Moesta tells you why the people you talked to switched; he does not tell you how many of them exist. Pair the switch interview with sizing work before you bet the roadmap on a job you only heard described in a dozen rooms.
From The Switch Interview
-
Ask people how they feel, or watch what they did?
Sean Ellis
trusts a single survey question put to active users, how would you feel if you could no longer use this, and reads 40 percent answering very disappointed as the fit signal. One number you can run this week and compare against the last time you ran it.
Bob Moesta
distrusts what people say in a survey and reconstructs what they actually did, interviewing recent switchers to find the push, pull, anxiety and habit that moved them. Stated preference is not behaviour.
Stated preference against revealed behaviour is a real methodological split, not a style preference. Ellis tells you whether you have fit; Moesta tells you why, which is the part you need before you can do anything about a bad number.
From Product-Market Fit Test (40% Survey)
-
Define the metric, or trust the taste?
Ronny Kohavi
says write the success metric down before you build, ask how you would cheat it, and add the countervailing metric that catches the cheat. Any metric you can move you can also game, and an experiment with no OEC will faithfully optimise the wrong thing.
Archie Abrams (Shopify)
runs core product without traditional KPIs, on the view that intuition and taste guide the decisions that matter and a metric target quietly narrows what the team is willing to try.
The split maps onto what you are building. Growth and optimisation work needs Kohavi, because that is where gaming is cheap and the effects are small enough to fool you. A core product bet has no comparable baseline to measure against, which is exactly where a scorecard turns into theatre.
From The Overall Evaluation Criterion
-
Check every split, or ship the winners fast?
Ronny Kohavi
says no result counts as evidence until the split passes the sample ratio check. Lost traffic is biased traffic, so a failed split makes results look super good or super bad. His team at Microsoft blanked the scorecard rather than let anyone read one.
John McEvoy
ran about ten paywall and onboarding experiments on his own app over two or three months. Conversion went from 0.5% to 8%, and revenue from around $8K to over $30K a month. His rule: A/B test paywalls and onboarding flows relentlessly (Starter Story, 9:12).
The deciding variable is whether anything outside the test can confirm the win. McEvoy had that check, because monthly revenue nearly quadrupled and a broken split cannot fake revenue. A team shipping small lifts it can see only in the scorecard has no outside check. At the rates Kohavi reports, about one test in twelve has a broken split, so the SRM check is what stands between that team and a roadmap built on noise.
From The Sample Ratio Mismatch Check
-
200,000 users, or five?
Ronny Kohavi
says size the test before it runs: an MDE of 5% or less at 80% power, which for a store converting at 5% means about 200,000 users. Below that a significant result is likely exaggerated, so build the culture and the platform, and test only for large effects.
Jake Knapp
tests a prototype with five target customers on the Friday of a design sprint, not with internal stakeholders. In Sprint, five customers reveal the major patterns without statistical overhead, so there is no reason to wait for a large sample.
They answer different questions. Five sessions show whether people understand and want the thing, because confusion repeats as a pattern. They cannot show whether a shipped change moved conversion by 3%, which is the question Kohavi's power formula answers. Use five customers to decide what to build, and the power formula before you claim a change moved a metric.
From The Minimum Detectable Effect Rule
-
Who do you talk to?
Teresa Torres
optimizes for a repeating weekly rhythm with whoever the product can recruit, on the theory that the habit compounds.
Christian Idiodi
argues the opposite trade: pick a fixed set of reference customers, six to eight for B2B, and design with the same people until they will vouch for the product.
One method buys breadth and consistency; the other buys depth and a reference list you can sell with. Neither survives if you only do it once.
From Continuous Discovery Cadence
-
Map the problem space, or shape one solution and bet?
Teresa Torres
grows the tree downward from an outcome and refuses to let a solution into the opportunity space. Several solutions branch under one opportunity so the team can compare them, and every branch has to trace back to something a customer said in a story.
Jason Fried
shapes work the other way. A senior person defines the problem and a rough solution together at the right level of abstraction, leadership bets on that shape for a fixed six-week cycle, and the filter on ideas is whether they keep coming back rather than whether an interview surfaced them.
The deciding condition is whether you already are the customer. Basecamp builds for a market Fried has lived in for two decades, so his intuition is compressed evidence and the tree would mostly re-derive what he already knows. A team handed an outcome in a domain it does not live in has no such stock, and shaping from intuition there is guessing with a deadline attached.
From The Opportunity Solution Tree
-
Count named advocates, or measure the population?
Christian Idiodi
sets the bar at a countable set of specific people, six to eight in B2B and twenty-five in consumer, who will personally put their reputation behind the product. Not signups and not a satisfaction score, a named list.
Sean Ellis
reads fit from a survey across active users, with 40 percent answering very disappointed as the line. A hand-picked group of advocates is, by construction, the most favourable sample you could have drawn.
Idiodi is measuring depth, Ellis breadth, and each is blind to the other's failure. Twenty-five devoted references can still be very nearly the whole market; a healthy survey number can hide that nobody would stake anything on you. Get the references first, then check whether they generalise.
From The Reference Customer Method
-
Ship the roughest thing, or the most rigorous test?
Eric Ries
treats the MVP as an instrument for answering a question, which puts rigor at the centre. Minimum is context-dependent and often demands more care than teams assume, because a test run below the market's threshold of viability returns a result about your polish rather than your idea.
Garry Tan
tells founders to launch the jankiest version that does the job, on the grounds that time spent perfecting anything before contact with users is time spent on assumptions. Shipping rough and early is the fastest way to find out whether anyone cares.
The condition is the market's tolerance for rough. In consumer software with a forgiving audience the jankiest build is a valid experiment, and Tan's advice wins on speed. In regulated, enterprise or high-trust markets, the viable half is what makes the result readable at all, and shipping under that bar produces a clean negative about the wrong variable. Ries is not arguing for polish; he is arguing that you cannot learn from a test the customer refuses to take seriously.
From The Minimum Viable Experiment