On the importance of Experiments

Experiment is more than just a Product Feature

A feature is a regular extension of a product whereas an experiment is a branch or an off shoot. An experiment is typically measurable in a tangible way and has to achieve goals & provide ROI (return isn’t always $). In contrast, a feature is incremental value addition to the product. Lets consider the case of Swiggy. Providing GPS tracking of the delivery executive is a feature. But introducing Homely, which is their own cloud kitchen is an experiment. There is capital commitment and revenue expectation.

Experiments can be game changing

Mo’s law: You can’t get to the moon by climbing successively taller trees.

It is easy to take the example of AWS experiment of Amazon and declare that they are vital & game changing. However, given vast resources at Amazon’s disposal, this may not be the right example for reasons that I will outline in the next section. Closer to home, take the example of Zomato. One of the earliest unicorns of the country, they were making handsome revenues from the listing business. However, when their lunch was threatened by Swiggy, they responded quickly. Leveraging the huge network of restaurants, they experimented with a Prime like subscription offering called Zomato Gold. This was a run way success for the company. More details on the making of this: https://www.zomato.com/blog/the-making-of-gold.

Another great example that I must state is Swiggy Pop (single meal purchases). This is very similar to the 1-click checkout experience of Amazon.com except that the idea is presented in a different context for the purchase of your lunch / dinner. It clicked with the customers and opened up a significant revenue channel for Swiggy (getting featured in Swiggy Pop requires the restaurant to pay premium to Swiggy). More details on the implementation: https://www.slideshare.net/PallaviBhargava1/story-of-how-swiggy-pop-was-built-107976451

Your experiments will result in failure most of the times

Akin’s law #12: There is never a single right solution. There are always multiple wrong ones, though.

This is the very nature of your experiment, a process of exploration in the idea space. You cannot know if an idea will succeed (if you do, then it is merely execution not experimentation). And believe it or not, you are going to fail most of the times. So, chin up and take it in stride. Jeff Bezos famously says that 9 out of 10 experiments will fail.

At our startup Juspay, the success of the BHIM app (yes, we developed the app which was launched by the Prime Minister!) emboldened us to thinking we could replicate the success on the merchant side. We deployed QRs at thousands of petrol pumps, only to realise later that we couldn’t make it work, as the idea was a little too ahead of its time.

In my previous company BankBazaar.com, we created a stock portal (like moneycontrol.com) in association with MSN. The idea is BankBazaar did the technology heavy lifting and MSN brought the customers. We lingered for quite long with this & spent considerable manual effort only to see it shutdown after few years.

Another story that I came across in Hacker News:

I was an intern at Rethink (back then, Heartland Robotics) 2010~2011, it was an exciting place to work at. There were ~10 employees when I joined, and ~40 when I left. One of the early product mistakes they made was focusing way too much on low-cost. They compromised many necessary performance specs (repeatability, speed etc.) just to be able to make the robot out of plastic. They eventually pivoted to their second generation robot, Sawyer, which was made out of casted metal components. This gave the robot much better performance but I guess that it was too late.

Source: https://news.ycombinator.com/item?id=18140907

Increase the chance of success? Ans: Increase the number of experiments

Now that we know you are going to fail 9 out of 10 times (approximately) on your experiments, the follow up question is: how does one maximise the chances of success. The answer is not very difficult to arrive at: develop the ability do more experiments. The constraint is that the resources at your disposal are quite finite (more chances they are abysmally low!). This implies that you have to run all your experiments smartly & frugally.

Be ruthless about ending experiments

Entrepreneurs are creatures of passion. It is easy to get emotionally attached to an idea. Don’t let this blind you. Your success doesn’t come from any single idea. There is a lot more to this long game. Be willing to lose the battle so that you may win the war. Muster the courage to end the experiment and look forward to the next experiment.

I would like to end the post with a paragraph from Jeff Bezos’s one of the letters to the shareholders:

One area where I think we are especially distinctive is failure. I believe we are the best place in the world to fail (we have plenty of practice!), and failure and invention are inseparable twins. To invent you have to experiment, and if you know in advance that it’s going to work, it’s not an experiment. Most large organizations embrace the idea of invention, but are not willing to suffer the string of failed experiments necessary to get there. Outsized returns often come from betting against conventional wisdom, and conventional wisdom is usually right. Given a ten percent chance of a 100 times payoff, you should take that bet every time. But you’re still going to be wrong nine times out of ten. We all know that if you swing for the fences, you’re going to strike out a lot, but you’re also going to hit some home runs. The difference between baseball and business, however, is that baseball has a truncated outcome distribution. When you swing, no matter how well you connect with the ball, the most runs you can get is four. In business, every once in a while, when you step up to the plate, you can score 1,000 runs. This long-tailed distribution of returns is why it’s important to be bold. Big winners pay for so many experiments.

Source: https://www.sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm

Are you ready to experiment? I am excited to know what your next experiment is going to be!

In this post, I wish to underscore the importance of experiments in the context of startups. A startup beings with a core idea and the early months (12–24) are spent executing that idea for the most part. Once the startup has taken hold, there is a need to constantly redefine the product market fit. Growth doesn’t happen automatically, the company needs to keep moving forward oftentimes making decisions in the dark. The only way to be sure is to take few steps, measure the effect and then decide to move forward or retrace your steps. In essence, you experiment a lot to find out the few ideas that actually work.




2x Founder - Hyperface.co, Juspay.in

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Alternative asset managers, did you know a free scorecard could inform your fundraising success…

The Pain of an Entrepreneur

The Perils of Bad Growth Forecasting

What happens when non-founders take over a start-up?

16 Global Startups Selected for Blackbox Connect: Female Founders Edition

Thoughts on distribution strategy and CEO/firm misalignment

Hi, and I hope you’re doing well!

7 Common Mistakes Entrepreneurs Make in VC Pitches & How to Fix Them

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ramanathan RV

Ramanathan RV

2x Founder - Hyperface.co, Juspay.in

More from Medium

ESG Metrics Spark Police Raid, Angry Tweets And Boardroom Drama


Our Vision for Inflect Health — And the Future of Innovation in Healthcare

Driving Innovation with Purpose