Don't miss
  • 2,007
  • 5,500
  • 5,764
  • 121

Experiment and Learn: Free-to-play design rule 14

By on November 7, 2012

The basic rules of how to adapt to a changing environment, whether for single celled organisms in a primordial soup or for games companies in a transitioning market, are pretty simple:

  1. Experiment often
  2. Make sure the experiments are survivable
  3. Learn from the failures (and the successes)

For a creature in the primordial goo, the lessons are particularly harsh. Individual organisms, prompted by random genetic mutation, experiment with different survival tactics. Some of these experiments fail, and the individual protozoa dies, together with its particular genetic makeup. The species as a whole thrives as it learns how to adapt through a Darwinian process of trial and error.

That is not much consolation to the expired protozoa. Similarly, you will need to make sure that you are part of one of the successful experiments, not an evolutionary dead-end, as you transition your business or skills to free-to-play.

The good news is that you already know how to experiment. Nearly every game released has some experiments contained within. Experiments in gameplay, in technology, in story-telling, in distribution method; in something.

The games industry is good at experimenting. It is not so good at making sure that when an experiment fails, the company thrives. RealTime Worlds bet $100 million on the belief that it knew exactly what gamers wanted from a new MMO. It was wrong. Until we stopped counting, the GAMESbrief Job Loss Tracker followed all of the companies that collapsed or downsized as retail game sales declined and the publishers and developers who made them failed to adapt. Who knows whether the current titans of gaming like Sony and Nintendo will be the titans of tomorrow?

I work with clients who argue that they can’t make a game for less than $30 million. My response: you had better learn.

The market is changing incredibly rapidly. We now work in an industry beset by creative uncertainty (as it always was), technology uncertainty (as it always was) and business model uncertainty (this is new). Against that background, processes that helped companies achieve high Metacritic scores (which probably don’t matter any more) and reduce technological risk may actively hinder the experimentation necessary to carve out a new position in the new market.

The key to experimenting often and surviving the failures is to reduce the scope of the project that you are building. You are trying to discover whether there is a market for your product, not to drop it into a market with a limited number of products released every year through a limited number of physical distribution channels.

FlickrCC Andrew Kuznetsov

If I had $20 million, I would rather run 20 $1 million experiments, not one $20 million dollar experiment. If you work in free-to-play, I urge you to do the same.

Learn and keep learning.

Humans are amazingly good at seeing patterns. It helps us to make sense of the world. It also makes us imagine connections, correlation and causation where none exists.The secret of a good experiment is to know what you are testing for. Write down what you are doing, and how you will measure the results. Run the test and revisit the numbers. Explain to someone else the results of the test so that they can help you see if you are just seeing the results you want to see (confirmation bias). Even better, try to run a test to disprove your pet theory. Proving a theory wrong is much easier than proving it to be right.

Having learned something, make sure you use it. Share the information throughout your organisation. Keep a list of the tests you have run and the results. It will save you time later.

And then… Start again. This process never stops.

About Nicholas Lovell

Nicholas is the founder of Gamesbrief, a blog dedicated to the business of games. It aims to be informative, authoritative and above all helpful to developers grappling with business strategy. He is the author of a growing list of books about making money in the games industry and other digital media, including How to Publish a Game and Design Rules for Free-to-Play Games, and Penguin-published title The Curve: thecurveonline.com
  • http://twitter.com/Sikthehedgehog Sik

    “I work with clients who argue that they can’t make a game for less than $30 million. My response: you had better learn.”

    Lol yeah, you should tell them about all the indies that are making games with a budget of exactly $0. Making the game isn’t the issue, being able to reach the players is. Also I’m expecting to eventually some indie to rise out of nowhere and take everybody by surprised, and maybe even become one of the new titans. In fact, isn’t that how several of the current titans started?

  • http://twitter.com/carlodelallana Carlo Delallana

    Minecraft says hello

  • http://www.gamesbrief.com Nicholas Lovell

    There are companies starting with cheap-ish games too. NaturalMotion’s early iOS games, for example, or titles from Rebellion (a client) and others.

    It can be done. It’s just difficult to change the mindset sometimes.

  • http://twitter.com/Sikthehedgehog Sik

    Mojang still isn’t a titan – and by that I mean at the same level of importance as the biggest game companies these days.

  • http://twitter.com/carlodelallana Carlo Delallana

    Riot Games says hello. Established as an indie studio in 2006. Today they are dominating the F2P hardcore market with League of Legends.

  • http://twitter.com/Sikthehedgehog Sik

    Are they known for anything besides League of Legends? I sure don’t know them for anything besides that, and the Wikipedia page doesn’t seem to help either =P They’re big for sure, but not like they have gotten as much power on the industry overall as e.g. Activision, EA, Ubisoft, etc.

    Also AAA games are also done by not-so-big-name studios, they just happen to have publisher backing most of the time, so I doubt that making an AAA game instantaneously makes you a titan.

  • http://www.gamesbrief.com Nicholas Lovell

    Carlo, I’m inclined to agree with Sik. It’s a huge businesses but they were acquired for $400 million and are probably nowhere close in value to an EA, Activision etc.

    The real titans are on mobile in Japan, where DeNA and Gree are doing extremely well. I think it’s DeNA that just posted over $600 million in revenue in a quarter.