Learn how to raise your odds of success

 

Experiments in Pricing

Written by Ash Maurya

What you charge for your product is simultaneously one of the most complicated and most important things to get right. Not only does your pricing model keep you in business, it also signals your branding and positioning. Unlike with other elements, it is harder to iterate on pricing. Once set, coming down is usually easier than going up. So when is the right time to tackle pricing? Most people choose to defer the “pricing question” as late as possible because they don’t think they (or the product) are ready. Something I hear a lot is that a minimum viable product is by definition (embarrassingly) minimal. How can you possibly charge for it?

A minimal product is not synonymous with a half-baked or buggy product. If you’ve followed a customer development process, your MVP should address the top 3 problems identified by customers as being important and it should do it well. 80% of your efforts should be going towards ensuring that by improving existing features versus cranking out new ones.

Steve Blank bakes price exploration right into the initial customer interviews. Price, like everything else, is built on a set of hypotheses that needs to be tested early. Steve makes you phrase the pricing question by first asking potential customers if they’d use the service for free. This is to gauge if the product’s value proposition is compelling at all. You then ask if they’d use the service for $X/yr. How do you come up X? You can simply roll the dice and adjust along the way, or use Neil Davidson’s excellent guide to software pricing to start with a more educated guess. Once your MVP is built, Steve asks you to sell it to your early customers. There is no clearer customer validation than a sale.

Sean Ellis, on the other hand, argues that achieving initial user gratification (product/market fit) is the first thing that matters and suggests keeping price out of the equation so as not to create unnecessary friction:

“I think that it is easier to evolve toward product/market fit without a business model in place (users are free to try everything without worrying about price). As soon as you have enough users saying they would be very disappointed without your product, then it is critical to quickly implement a business model. And it will be much easier to map the business model to user perceived value.”

Both Steve and Sean advocate removing price from the equation but at different points. Steve does it during the customer discovery process but makes you charge for your MVP. Sean does it with the MVP and only makes you charge after product/market fit. I can see the merits of both approaches and wondered which was right for my product: CloudFire – Photo and Video Sharing for Busy Parents.

Why Not Just Use Freemium?

On the surface Freemium seems like the best of both worlds: Get users to try your service without worrying about price, then up-sell them into the right premium plan later. However, the mistake many people make is giving away too much under the free plan which leads to low or no conversions. It’s human nature – we all want to be liked. More importantly we don’t yet have enough information to know how to price or segment the feature set. I made this mistake with my first product BoxCloud where an early visionary customer called me up and said “I really like your product and want to pay for it but your pricing doesn’t require me”. After a few further failed feature segmenting iterations, I decided to forgo the free plan and simply offered premium plans with a trial period. Sales went up and so did the quality of feedback which I attribute to the difference between customers versus users.

Hiten Shah shared a similar story with me around his experience with Crazy Egg. Even 37signals has greatly deemphasized their free plans to almost being fine print on their pricing pages.

Lincoln Murphy just published a timely white paper on “The Reality of Freemium in SaaS” which covers many important aspects to weigh in when considering Freemium, such as the concept of quid pro quo where even free users have to be giving something back. In services with high network effects, participation is enough. But most businesses don’t have high enough network effects and wrongly chase users versus customers.

What I particularly liked in this paper is calling out
“Freemium as a marketing tactic, not a business model.”

I strongly feel that, especially for SaaS products, starting with free and figuring out premium later (all too common) is backwards. If you know you are going to be charging for your product, start by validating if anyone will pay first. There is no better success metric and it leads to less waste in the long run. Focussing on the premium part of freemium first lets you really learn about your unique value proposition – the stuff that will get you paid. You can then come back and more intelligently offer a free plan (if you still want to) with the right success metrics clearly defined. Even if you think you have a one-dimensional pricing plan (number of projects) like I did, you’d be better served testing it with paying users because pricing experiments take a much bigger toll than other types of experiments.

Practice Trumps Theory

So how do I put all this to test:

The biggest mind shift in following a lean startup process is going from thinking you know something to testing everything you think you know.

Testing the Waters Early

I followed Steve Blank’s advice and built some pricing questions into my initial face-to-face customer interviews. Because CloudFire is a re-segmented product in an existing market, potential customers did rely on competitor pricing for reference points. This had to be balanced against the perceived value of our unique value proposition – saved time from faster and easier sharing of lots of photos and videos. Through these interviews, I determined that, like their sharing needs, my potential customers valued simple hassle-free pricing and $49/year for unlimited photo and video sharing was a fair price they were willing to pay. That is what I charged them once my MVP was ready.

Testing Web Visitors

I wanted to run the same set of pricing tests with web visitors that I did during my interviews. Short of split testing a free and paid version of the MVP, which is *technically illegal* and unfair to paying customers, I decided to split-test 3 different products with 3 different prices:

  1. $49/yr for unlimited photo and video sharing
  2. $24/yr for unlimited photo sharing
  3. FREE for 500 photos

All plans have a 14-day free trial with the exception of the free plan which is free forever.

Here are the variations I tested:

Original: Single unlimited plan

This is the simple single plan option I determined during the customer discovery process which served as the control.

Variation 2: Multiple plans

I segmented the product into 2 offerings: unlimited photos+video and unlimited photos only. I wanted to test price sensitivity and gauge interest in video sharing. Not many people I interviewed were currently taking lots of videos but they all wanted to be doing more.

Variation 3: Freemium

This has the 2 plans from above along with a limited free plan. Yes, this is a freemium case. I wanted to measure if a limited free plan would disproportionately drive the right type of traffic (busy parents in my case).

Variation 4: No Price During Introductory Period


I added a fourth variation to test Sean Ellis’ advice on removing price till product/market fit, but I tested this differently. I was not comfortable offering the full product at price and for FREE at the same time. So rather than including this page with my A/B tests, I instead tested it with new parents I interviewed.

The Results

First Place

Original: Single plan – Second Place in Conversions and Best Overall Performer
Surprisingly the original page was the best overall performer.

Second Place

Variation 3: Freemium – Most Conversions but Second Place Overall
Not surprisingly, the Freemium variation drove the most conversions but only outperformed the original by 12% and had the lowest retention. Referral stats combined with random polling/emailing revealed a majority of the users that signed up were just curious (and not parents).

Third Place

Variation 2: Multiple plans – Least Conversions and Worst Overall Performer
People reacted least favorably between the two paid plans.

Non-starter

Variation 4: No Price During Introductory Period – Non-starter
Parents I interviewed did not understand the introductory period without explanation and were reluctant to try the service without knowing how much the service was going to cost. Probing further, they weren’t willing to invest the time building up web galleries and inviting others only to find that the service might be priced out of their expectation.

What I learned

It does pay to align pricing with your overall positioning. Our unique value proposition is built around being “hassle-free and simple” and people seemed to expect that in the pricing model as well. A lot of our existing customers were already paying for their existing sharing service so the leap from free to paid was not a big one. While Sean suggests removing price before fit for consumer facing products, he suggests always charging for Enterprise customers to gain their commitment. This is another case where pricing needs to be explicit. Using Cindy Alvarez’s model, our customers appear to be Time-Poor, Cash-Rich. Offering no-hassle free trials was sufficient to remove the commitment risk. Money back guarantees might be another way to further lower this risk.

The biggest lesson learned though, is how accurate my initial customer interview findings were compared to all the hypotheses that followed. Pricing is more art than science and your mileage will vary, but whenever possible get out of the building, talk to a customer, and consider testing price sooner rather than later.

What do you think? Why do you think these variations finished the way they did? What other variations would you like to see us try? How else do you think we could increase conversions?

Become a Practice Trumps Theory member
(it's free)

Get access to more tactical content, videos, and hands-on techniques for systematically building a successful product...

  • http://www.mediascend.com/ Jackson

    Brilliant article, Ash.

    How did you come up with your $49 figure? Were you ever concerned with offering unlimited storage at a fixed price?

    I’m curious what metrics you used to allow you to feel comfortable with this pricing structure.

    Thanks.

    [Reply]

    Ash Maurya Reply:

    Jackson –

    I point out Neil Davidson’s e-book which is a great reference on pricing theory. I came to that price by first asking what I’d be willing to pay. My wife and I use CloudFire every day and are users #1 and #2. I also looked at the competitive landscape which is the strongest reference point customers use in an existing market.

    As to offering unlimited storage, you have to make certain assumptions on what the “actual average storage per customer” will be (something we measure) and also factor in the falling scale for storage/bandwidth pricing. The key is run some back-of-the-envelope calculations to justify the business model but then measure all the assumptions. There is also the possibility of offering future value added services like prints, dvds, etc. but the model has to work without them.

    [Reply]

  • http://www.mediascend.com Jackson

    Brilliant article, Ash.

    How did you come up with your $49 figure? Were you ever concerned with offering unlimited storage at a fixed price?

    I’m curious what metrics you used to allow you to feel comfortable with this pricing structure.

    Thanks.

    [Reply]

    Ash Maurya Reply:

    Jackson –

    I point out Neil Davidson’s e-book which is a great reference on pricing theory. I came to that price by first asking what I’d be willing to pay. My wife and I use CloudFire every day and are users #1 and #2. I also looked at the competitive landscape which is the strongest reference point customers use in an existing market.

    As to offering unlimited storage, you have to make certain assumptions on what the “actual average storage per customer” will be (something we measure) and also factor in the falling scale for storage/bandwidth pricing. The key is run some back-of-the-envelope calculations to justify the business model but then measure all the assumptions. There is also the possibility of offering future value added services like prints, dvds, etc. but the model has to work without them.

    [Reply]

  • Pingback: uberVU - social comments

  • http://softwaremaven.innerbrane.com/ Travis Jensen

    Really great article. Thanks for sharing your experience!

    [Reply]

  • http://softwaremaven.innerbrane.com/ Travis Jensen

    Really great article. Thanks for sharing your experience!

    [Reply]

  • JJ

    Having done lots of test/focus groups/surveys etc etc, my understanding is most people from technical backgrounds tend to view customer feedback as a very linear process esp. when related to pricing.

    Let me explain:
    Asking someone about pricing information will probably give erroneous data for a variety of reasons – mostly psychological.

    The act of putting money on the line is a very different psychological experience then simply answering a few hypothetical questions about a product.

    Nothing beats price split-testing – BUT the problem is the appeal/customer segment is probably different at different price points.- it’s all about testing combinations of appeals/prices.

    The name of the game is which price will give my the highest lifetime value of a customer.

    Example:
    Price: selling cloudfront on a free trial, subscription, one-off large upfront payment, series with a balloon, a balloon with a series etc etc

    Appeals: Cheap, Value, Time-Saving, You’re Super-Smart, Envy, etc etc

    Here are the results:
    Some appeals only work at certain price points.  The question is can you find enough customers for the price/appeal combination.

    [Reply]