Assuming you know:

1. What’s a Beta
2. What’s a Minimum Viable Product
3. Why some people keep their business in perpetual Beta

… and you agree that is important to Build – Measure – Learn, I will try to give you some tips to start planning your Beta based on lessons learned, articles I read, and things I discussed with my friends.

First things first…

If your business model is not focused in advertising, PLEASE forget your page-by-page Google Analytics report. Your pageviews and unique visitors say nothing about your product.

Start with your product

Before coming up with the numbers you wanna track, bring your business canvas to the table.

Right now you only have assumptions, but this will get it started. Worst case scenario, you will be able to cross out a hypothesis and try another one.

• Who is your customer: I’m not a big fan of customer archtypes but I have to agree it helps get customer insight while you don’t have any real data to dig.
• What are your revenue streams: This is a good place to start thinking about what is conversion for you.
• What’s your business proposition: A conversion is nothing more than the moment when your customer validate your business proposition in his or her head. It’s when this person realizes this might actually help and it is worth a shot.

Conversion: Decision path and its steps

I realize you might decide not to start selling your product during Beta, or maybe you have a Freemium model, with different types of conversion. But I guess it gets easier if you think about your customer’s decision path.

This is JobConvo’s promise:

JobConvo is a cost-effective solution to speed up your Recruitment process with asynchronous video job interviews. Creating a 30-minute interview script and sending out invites takes less than 5 minutes. Your candidates can choose the best time to record it and you are able to review it as soon as they submit the interview.

It has to be easy, fast and effective to the recruiters, and, of course, engage the applicants.

Take a look at my brainstorm about customer’s decision path for JobConvo Beta. Every process shape (the squares) in the fluxogram stands for a key metric.

Working with % helps figure out how much effort you need to convert a client. Higher the % closer they are to buy your product or be part of your customer development.

• 25% – Client relates to hypothesis: This person thinks our proposition is worth a look. Apparently, he relates to the problem we are claiming to solve.
• 50% – Client is testing the hypothesis: The solution seems compelling enough to spare a few minutes to actually test it. We’re on our way.
• 75% – Client is achieving actual response: This person is able to notice that 1. the product technically works, 2. the product is also compelling to his customer, in this case, the applicants.
• 100% – Client has enough data to make a judgement: I decided to call this conversion because in JobConvo’s Beta we weren’t trying to sell the product. We were more focused in validating the hypothesis.
• Pitfalls – Pitfalls are the places in your costumer’s decision path where things can go wrong: a hole in your hypothesis or a hole in your implementation. In this example, the first one might point a lack of engagement on the applicant’s side, and the second one might point a usability or technical problem: they can’t perform the submission.

When I say “higher the % closer they are to be part of your customer development” it doesn’t mean you don’t need to invest in customer development for ‘under 50‘s. You do need to work with them to figure out why they don’t relate enough to your value proposition to sign up/test it. The % only changes how to approach them, what leads us to the differences between an Open Beta and a Closed Beta/Pilot.

Open Beta vs. Closed Beta/Pilot

Okay, now you have a decision path, a few steps, and some smart pitfall triggers…

… And your metrics still can still point you the wrong direction.

Let’s say you decide for an Open Beta. You get the word spread and have lot’s of people testing it for you. It’s great: You will find out a lot of bugs, get some really nice usability insights and you will have enough input to improve your product.

But what if your Beta testers are not your target market. Are you sure you want to invest your time iterating based on needs of people that might not actually have the problem you are trying to solve?

This is what Closed Betas and Pilots are for. Of course it’s not easy to find a generous number of the right people to put together a Pilot, but there must be a way out.

My favorite pilot story is told by TaskRabbit‘s Leah Busque on Kevin Rose’s Foundation podcast. She had this idea that moms would be very interested in her model. They have a lot of small tasks to do everyday, several responsibilities, not enough time, and this actually fits the problem Busque was trying to solve. She went to some sort of support group meeting for moms, pitched her idea, and they bought it. The runners came from CraigsList. She interviewed one by one. Gym-Pact also used CraigsList to their first pilot in Boston.

For JobConvo Pilot we offered the platform (and support) to AIESEC Chile. AISEC is an independent not-for-profit organization run by students and recent graduates to offer young people the opportunity to participate in international internships in companies like Nike, Microsoft and PWC. They have the need (lots of applicants, not enough recruiters, challenging deadlines), they have enough autonomy to adopt a new service, they are willing to help entrepreneurs. Full-package!

They interviewed more than a hundred people using our service and provided enough high-quality data to improve our product.

Funnels and Cohorts

Let’s go back to the metrics.

Every member of the Lean Startup cult will try to come out with a funnel and/or a cohort. My 5 cents? Work with a nice funnel and leave your cohort for when your product gets a little bit more solid. You can play with a cohort table as much as you want, but in the initial stages of your product development you might want to save up some time to get other things done.

How do I look?

The truth is: Going well or not, in the end, you will have a lot of work to do. Try not to focus in rating your success. Spend all your energy (and money) getting your product right…

… but if it can help you move on, here is Fred Wilson’s (Union Square Ventures) 30/10/10 rule.

• 30% of the registered users or number of downloads (if its a mobile app) will use the service each month
• 10% of the registered users or number of downloads (if its a mobile app) will use the service each day
• The max. number of concurrent users of a real-time service will be 10% of the number of daily users

Just remember: More important than achieving 30/10/10 in your Pilot or Beta is gathering enough data to figure out what to do next!