Game Companies Aren’t Tech Companies Part III: The Content Problem

Part 1 is here, while part 2 is here.

The only Content Problem worth saving. Do it Reed!

So maybe game companies aren’t tech companies. But as much as game companies seem to borrow from tech firms there’s even more to be said for the opposite. If Netflix, a subscription service with over 10,000 movies and TV shows, has its biggest competitor in a single game, Fortnite, then perhaps there’s more for tech to learn from games. And how games deal with The Content Problem is it’s defining characteristic. Of all forms of entertainment, games present the most compelling answer to the problem.

The Content Problem

The fundamental axiom of economics is that there are unlimited needs and wants and only limited means to fulfill them. The parallel for entertainment might consider that core content demand nearly always outstrips supply. For example, large swaths of Game of Thrones and Harry Potter fans are underserved by a couple books, movies and TV seasons. Executives try to fill the void with licensing: Harry Potter backpacks, Game of Thrones beer, etc. But filling the core content demands are impossible: it takes far more than 1 hour to produce 1 hour of Game of Thrones while the same it not true for games.

Consider the following:

The content consumed in a game like Overwatch or Clash Royale is the pursuit of strategy equilibrium and/or mastery of mechanics. A new unit in Clash Royale changes how players organize their decks, even if they don’t use the unit directly (they must counter it). This can provide hundreds of new hours of content to consume relative to the near 1 man-week of labor to produce the new unit. Therefore, the marginal content output of a given member of the 16 person (!) Clash Royale team is astronomical.

Furthermore:

The genius of PvP (Player v Player) environments is how they necessitate the emergence of a meta-game. In mathematics, Player vs Environment (PvE) resembles the field of optimization where strategies are static – one and done. PvP environments, however, resemble game theory models where it has been shown strategies evolve in an evolutionary process. This means equilibrium in PvP environments is constantly reshuffled with each balance change; the search for dominant strategies in an ever shifting equilibrium is the game itself.

The marginal product of labor for a given game developer completely outclasses a given producer on a movie or TV show by virtue of the medium, not the individual. Unlike games where assets are infinitely replicable, movies and TV face fixed constraints: Emilia Clarke or David Benioff can only be in a single place at a given time. They must also eat, sleep, socialize (sigh). Meanwhile, Captain Price faces no such constraints. There’s no more Game of Thrones to consume after the last episode cuts to black while there’s always another hour of Fortnite to play. How can Netflix and others adapt to the reality of these mediums?

The most straightforward strategy is a content arms race. Netflix continues to spend over $17B a year on original content while scooping up oodles of back catalog content. Of course, viewers must be interested in this content for it to be “effective” and the recommendation engine plays a strong role in this. But the the last episode of Stranger Things is just that, the reco engine cannot find fill the void while operating on the same indifference curve. The “more bodies” strategy to solving the content problem is expensive to execute and as we’ll see in part 4, struggles to achieve Marginal Cost = Marginal Benefit.

Reality TV is a response: less writers, editors, and CG needed to produce a given hour of content. Shows like The Amazing Race, Big Brother and Survivor can do 20+ seasons of 22+ episodes while Game of Thrones struggles with 7 seasons of 10 episodes despite having so many more crew members. Netflix’s speed of investment here is breathtaking. But the addressable audience is more limited in scope then traditional dramas. Netflix needs a bolder evolution to combat games: TV-as-a-service.

The forgotten genre of soap opera TV provides a near perfect blueprint. For those unfamiliar, soap operas are near year-round weekly serialized television shows. The unrelenting pace has resulted in popular series like General Hospital having 14,000+ episodes over 57 years. Netflix needs to heavily invest in moving shows to a similar formats: year-round production with weekly releases. There’s always another piece of content to consume right around the corner while the back-catalog for a given show is continually expanding for newcomers. In many ways, this mirrors match-3 level production. The number one reason why players churn from match-3 is a lack of new levels and a quick glance at community pages confirms this.

Sugar Rush or something, right?

King mitigates this increasing difficulty for example. This increases the time to completion but could also result in churn. Reality TV faces no such option. Another strategy is also possible however: branching narratives.

Increasingly, Hollywood is shooting movies back-to-back. It’s cheaper to continue production rather then stop and go. Why not do something similar to produce more content? In this case, shoot multiple perspectives in a given series simultaneously. Lord of the Rings production operated in a similar way with two production crews, but with a singular end product. Game of Thrones also operated in this way from a production standpoint, but again the end product was single episode. Why not dedicate an hour to each perspective? This easily multiplies a 10 episode season to 30 while holding down cost. Netflix can’t solve The Content Problem, but it can mitigate it.

Interestingly, Youtube has solved most of this problem via a two-sided marketplace. The sheer smattering of volume helps the supply-side problem even if a particular creator has a finite number of video (remember, you can still play a given game for an unlimited amount of time without “running out” of content). Youtube has encouraged users to subscribe to many different creators so regular release cadence is also accounted for as well.

Diminishing returns for linear content are extremely steep, few users will watch a film or movie more than once. Deepening the LTV of a viewer primarily come through more linear content: an extremely expensive proposition. To compete with games, TV and movies need far more supply. If technology and business models can really change creative product rather than be a vehicle for it, now has never been a better time to explore changes in storytelling.

Can We Get Players to Tell Us Their LTV?

Nook is all too familiar is compounding growth.

Eric Suffert acutely describes the dangers of extending payback windows. At every t+1 the accuracy of LTV declines while the variance in cohort profitability increases. LTV, however, is not an exogenous variable and clever design can incentivize players into revealing their long-run time horizons within a game.

Consider the design of a many subscriptions: you can pay a lower annual fee or a higher month-to-month fee. If you’re uncertain about the subscription, then the month-to-month is more economical while if you have more certainty then the annual fee makes more sense. The choice is a huge predictor of retention: annual users are far more likely to retain then month-to-month users. The mere inclusion of this annual/month-to-month choice gives users the opportunity self-segment into more predictable cohorts. Why can’t we use the same mechanics in game design to create more predictable LTVs?

Consider two possible goods for purchase via gems in Clash of Clans: a builder or gold. The builder increases the long-run growth rate of gold while the gold itself is a temporary boost in short-run capital stock. In layman’s terms: spending 100 gems on a builder might net you 200 gold today and 1,000 gold by D30 while spending 100 gems directly on gold may only yield 700 Gold today and 0 gold by D30. The builder is an annuity that pays dividends every period, the longer a player’s time horizon the more valuable the annuity.

Players who expect to have a long time horizon in a given title have an enormous incentive to purchase “investment” goods or goods that pay dividends overtime (battle passes similar to some degree). Not doing so results in a increasing opportunity cost penalty every period due to lost compounding growth.

F2P has experimented with direct daily annuities of hard currency. They offer players a discount over the standard IAP packs but must pay upfront to receive a daily allowance. Instead of a 30-day pass, why not ramp to a quarterly or bi-annual pass? Doing so would make LTVs more predictable early in given player’s lifecycle.

How to Measure Whales

"$10.00 LTV am I right?"
“$10.00 LTV am I right?”

You’ve soft launched your game, done a UA push, and a string of hope appears. Against all odds, a dominant cohorted ARPU curve emerges! Is this this an anomaly or have you caught a whale?

The first way to examine this is to perform cointegration tests between the cohorted ARPU curves, testing for stastistical significance. It may be true the difference in the curves are real, but that doesn’t answer if you’ve caught a whale.

In 1905, Michael Lorenz developed a method for measuring relative inequality between nations known as the Lorenz curve.

Just keep saying what % of the population owns what % of the wealth and it'll make sense.
Just keep saying what % of the population owns what % of the wealth and it’ll make sense.

The F2P application is to define wealth as revenue (either on a daily or game level) and players as the population in the context of free to play games. By measuring how bent inwards a cohorted Lorenz curve is relative to other cohorted Lorenz curves we can measure the ‘whali-ness’™ of different cohorts. Even better is how this reduces to a single metric – the gini coefficient. A gini coefficient of zero indicates a perfectly equal distribution of income, 10% of the population owns 10% of the wealth, 20% of the population owns 20% of the wealth and so on and so forth. A gini coefficient of 1 is the exact opposite – a single person owns 100% of the wealth.

This translates to what % of players are responsible what % of the revenue. Measuring gini coefficients across games rather than cohorts gives more insight into how a particular game monetizes – whether it’d be whale, dolphin, or minow driven.

Actionable insights might include how effective introducing ads could be. A high gini coefficient (very few players are responsible for revenue) might mean there’s a more fertile base to monetize on.

The main insight, however, is further understanding. It’s clear that success can come about in drastically different ways in free to play games, the gini coefficient is simple way to measure that.

Get more life out of your Lifetime Value Model! A discussion of methods.

Customer-Lifetime-Value

Predicting the average cumulative spending behavior or Lifetime Value (LTV) for players is incredibly valuable. Being able to do so helps figure out what to spend on User Acquisition (UA). If a cohort of players has an LTV of $1.90 and took $1 to acquire then we’ve made money! This helps evaluate how effective particular channels of advertising are as we’d expect different cohorts of players to have different values. Someone acquired via Facebook may be worth more then some acquired via Adcolony.

But wait there’s more!

My argument in this post is that LTV has great deal of value outside of marketing. In fact, LTV might have parts more valuable then the whole. How to predict LTV can adopt numerous approaches and each approach has associated benefits. Remember, there doesn’t have to be just one LTV model!

Consider four requirements we’d want out of an LTV model:

1. Accuracy

LTV predicted should be the LTV realized. Figuring out upward and downward bias in your coefficients is important here. This gives insight into the maximum or the minimum  to spend on UA depending on the direction you suspect your coefficients are biased towards.1

2. Portability

Creating models is labor intensive and even more so when doing so for multiple games. There are particular LTV models that sweep this aside called Pareto/Negative Binomial Distribution Models (NBD). Since they’re based only on the # of transactions as well as transaction recency they don’t require game specific information. This means you can apply them anywhere!

3. Interpretability

This one’s big and perhaps the most overlooked. Consider the Linear * Survival Analysis model approach to LTV. The first part is to predict when a particular player will churn. By including variables like rank, frustration rate (attempts on particular level), or social engagement we gain insight in what’s retaining players. This type of information is incredibly valuable.

  1. Scalability

If it’s F2P then there are going to hundreds of thousands to millions of players (you hope). I’ve seen some LTV approaches that would take eons of time to apply to a player pool of this size, our LTV should scale easily.

So how do the different approaches stack against one another?

Accuracy Portability Interpretability Scalability
Pareto/NBD2  / x x
ARPDAU * Retention3 x x
Linear * Survival Analysis4 x x x
Wooga + Excel5 x
Hazard Model6 x x x

Parteo/NBD is great, but it’s hard to incorporate a spend feature (it just predicts # of transactions).7 A small standard deviation in transaction value gives this model a great deal of value and something to benchmark against. This model also makes sense if data science labor is few and far in between.

ARPDAU * Retention is probably the approach you’re using; it’s a great starter LTV. If marketing/player behavior becomes more important, the gains to scale from a approach beyond this start to make more sense.

Wooga + Excel just doesn’t scale which kills its viability, but it’s conceptually useful to understand.

Linear * Survival Analysis  gives a great deal of interpretability that also sub-predicts customer churn time. This means testing whether the purchase of a particular item or mode increases churn time is done within the model. The interpretability of linear models also means it’s easy to see different LTV values for variables like country or device.

There are many, many different approaches beyond what’s been laid out here. Don’t settle on using just one model, each has costs and benefits that shouldn’t be ignored.