The Environmental Economics Approach to Liveops Content Management

I’m sure Candy Crush will put Hotelling over the line for a posthumous Nobel Prize.

In 1931, American economist Harold Hotelling published the seminal paper The Economics of Exhaustible Resources. Harold described a problem many firms face: how much of a non-renewable resource should they sell at any given time? This problem is more obvious when thinking about managing an oil supply, but just as relevant when considering how to manage match-3 levels.

For oil firms, since supply is always declining, price should move up at every \(t + 1\) period, holding all else constant. As a result, the optimization question emerges: Do I sell my oil now or wait for a higher price later? To solve, we first need to understand net present value or the idea that value now is worth more then value later. We can model this as:
$$ V_{today} = V_{later} / {(1+r)^n}$$
Where \(r\) is the interest rate or a proxy for opportunity costs and \(n\) is the number of periods. All we're saying here is that $1,000, say, 7 years from now is only worth $710 dollars today at a 5% interest rate. This is because I could take that $710 dollars, invest it, and in 7 years I'd have $1,000 given the 5% interest rate.
$$ $710 = $1000/ (1+.05)^7 $$

The more the interest rate rises, the more sense it makes to sell oil now and invest the money rather than waiting for the price of the oil increase. The oil firm needs to compare the rate at which the oil price increases against the rate of interest. As such, the price of the oil is strongly correlated with the increase of the interest rate over time.

Source

Hotelling's approach and intuition helped form what is called dynamic programming economics. More wrinkles have been added to the model to solve everything from cake-eating to managing how many fish should be caught. The tools of the field help us solve tricky now or later dilemmas.

Consider the release cadence of television show episodes. Should Netflix release all episodes in a given season all at once, in batches, or weekly? Perhaps even monthly? The problem is littered throughout gaming: should I use a limited-time direct store or an always growing catalog store? How should I distribute rewards throughout progression? Should I save content for a UA burst? And like the oil firm, we're in search of the content rationing solution that maximizes \(LTV\).

Match-3 Player Level Management

Consider how King should ration Candy Crush levels for new players. King is producing new levels at some constant rate \(p\). But we know that players are consuming levels at a greater rate than King can make them in any given time period \(t\). $$ c_t > p_t $$
At some point, a given player will run out of levels, the speed of which is governed by the size of \(c_t - p_t \). We can pick some random constants to visualize this:
We might also think of a player as having a churn probability on any given day since install. This declines over time: the best predictor of a player retaining on D30 is if they retained on D29 ( decreasing convex or \( \frac {\partial Pr(Churn)}{\partial DaysSinceInstall} {<}0 \)). This suggests elder players are less difficulty "elastic" or respond less to increases in difficulty.
And yet, we might imagine that the probability of churn increases every day without new levels as well (increasing convex or \( \frac {\partial Pr(Churn|NoNewLevels)}{\partial DaysSinceInstall}{>}0 \)). In the example above, the player ran out of levels on D205.
In the long-run, King could ramp up level production, but that would eventually hit diminishing returns and has its own labor costs. As a substitute, King's main lever to control player progression speed is difficulty. By raising difficulty, King can shift the line further right, as it would take longer for the player to reach the no content or exhaustion point. King could even tune difficulty such that \(c_t = p_t\) and the player would never run out of levels. On the other hand, if difficulty is too hard then churn will spike (low difficulty also causes this)! This suggests that King needs to solve an intertemporal difficulty optimization problem. That's a mouthful, but all it means is that King needs to balance the spike in probability from not enough levels against the spike in probability from increasing difficulty too much. To do so, we first need to model the total marginal effect of no new levels for every day since install, \(d\). This is the counterfactual churn probability of when there were levels against the current probability of churn when there are no more levels.
$$ \text{Total Marginal Effect of No New Levels} \\= \Sigma{[Pr(Churn|NoNewLevels)_d - Pr(Churn|NewLevels)_d}] $$

Whereas increasing difficulty to delay this from occurring has its own cost.

$$ \text{Total Marginal Effect of Increasing Difficulty} \\= \Sigma{[Pr(Churn|Difficulty_{x+1})_d} - Pr(Churn|Difficulty_x)_d ] $$

The solution to this system of equations is what we might call the maximum sustainable difficulty. And the intuition is similar to what first developed for the oil firm.

A bunch more stuff falls out of the model. For instance, King should significantly increase the difficulty of levels that are just near the exhaustion point, just as the oil firm would significantly increase the price of the last bits of oil. There's very little downside to doing so, as the player will be in a high probability space soon enough anyways. Furthermore, since King is always stockpiling levels, new players face an exhaustion point further away then players who downloaded the game at first launch. It's in King's interest to ease up on the difficulty for early levels since the area underneath the churn curve expands as the exhaustion point moves rightward.

There's much more room for expansion of the model and I encourage data scientists to play a strong role in systems design.

Ultimate Team, Fantasy Sports and the Sorare Thesis

“Something to aspire too”

Professional sports give us something to aspire too. Players are celebrated as heroes and children grow up wanting to become them. It’s no secret that the internet, and games in particular, have found even more ways to engross us in the world of sports. But the terms of that engrossment are not incidental, they’re crucial. NBA TopShot lets users “own” iconic moments. FIFA Ultimate Team (UT) has players collect star footballers. Fantasy sports gives betters big stakes based on outcomes. These platforms offer us an opportunity to insert ourselves closer to the action. French start-up Sorare fuses these aspects in a way we haven’t seen before; it’s the greatest challenger to UT and fantasy sports in years (sorry PES).

Ultimate Team

Some games seem to escape monetization punditry despite wild success (GTA:V!). EA’s Ultimate Team model goes relatively unstudied compared to the newest Supercell arrivals. Yet, Ultimate Team, and FIFA UT specifically (it’s in Madden as well as NHL), have clearly tapped into something:

In totality, UT represents around ~30% of EA’s revenue

These sort of numbers, overall growth rate, and sustainability make UT one of the most successful live service models of all time.

The UT game loop is eerily similar to many F2P collection RPGs: collect and upgrade rosters of “heroes” that increase in power level (called OVR or overall rating in UT).

All that’s needed is to replace Galaxy of Heroes screens with FIFA.
In many ways, UT is DEEPER than collection RPGs

By playing against a mix of CPU and real world opponents, players collect footballers who they can use to collect even more powerful footballers. One ingredient of the UT secret sauce is the lack of an end game. Payer churn is most likely to happen when content dries up but in Ultimate Team each yearly version of the game resets progression entirely.

Fantasy Sports

In parallel, we’ve seen an explosion of fantasy sports in DraftKings and FanDuel. Players must bet on key real-world sport stars performing well against a deep scoring system.

This is football right?

But generally, fantasy sports doesn’t offer between bet progression in the same way that UT lets players grow OVR overtime. DraftKings has entry tokens, a store of value, but that’s hardly progression. There is some persistence in that high scoring players receive a unique cosmetic and access to exclusive betting pools. When leagues span a season, persistence emerges, allowing players to effectively GM a team. This creates multitudes of implicit side-bets. If you think a player is not likely to come back from injury his market price should drop accordingly. If it doesn’t, you can exploit that knowledge to avoid owning an asset that doesn’t generate fantasy points. Studying prices via markets and making forecasts turns out to be a pretty fun game.

The Opportunity

Unsurprisingly, the total addressable market for sports is enormous. In North America alone, media rights, gate revenues, sponsorship, and merchandising represent around $75B in annual revenue. Sports games account for about $6.6B or 11% of the U.S. video game market. This likely translates to about $1-$2 spent on sports games for every $10 spent on “core” sports activity. If we add in collectibles (cards, memorabilia) and fantasy sports, we’re likely approaching $3-4 dollars for every $10 spent on “core” sports.

Who wouldn’t want a slice? Dominant leagues, however, maintain a moat: licensing. It’s been difficult to challenge FIFA since EA maintains a monopoly on simulation style football games with FIFA and many individual clubs. Konami and PES have faced bidding wars with EA and unsurprisingly the asset has gone its highest valued use (EA’s FIFA). But if you want a monopoly, you want it where the value is. I just think we’ve just misidentified the location. The “simulation” aspect of FIFA (i.e. control a player/team via joystick) is almost tangential to FIFA’s popularity. FIFA only became a monetization and engagement juggernaut after the release of UT. It started as paid DLC and long after FIFA’s initial release.

Scary to consider what these numbers are today… 50M? More?!

So where is the value if not in the simulation aspects of FIFA? Consider the explosion of FIFA Companion App downloads. The app lets owners buy and sell players from their mobile phones rather than a console. Based on store download rankings, it likely does north of 35M downloads per FIFA edition. Sensor Tower only provides the last 90 days of data, well after FIFA 20’s Oct 6th release, but we can still see how dominant it is in key countries.

Based on historical gamestat data, FIFA 21 has around 45M players at this point in its lifecycle. It’s highly unlikely all Companion App players own FIFA 21, but an even conservative attach rate would be more than 60%. However, if we restrict the funnel to only those who play UT, the attach rate increases! It’s not wild to imagine that 8 in 10 UT players use the Companion App. Players are spending more time engaging with the market then they are with moment-to-moment gameplay.

FIFA PS4 Player Count by Days Since Launch

About even Source

As we know, studying prices via markets turns out to be a fun game. It’s no wonder Football Manager, an entire game based around transfers, sells over 1M copies a year. Beyond a game mechanic, auction style monetization tends to align MB = MC. This blog has long argued effective monetization depends on prices reflecting player’s willingness to pay. To boot, Sorare doesn’t use loot boxes for price discovery, an inferior solution to their current strategy. Instead, Sorare direct lists players on the auction house and collects the proceeds. Players and developers have more aligned interests as Sorare’s entire revenue stream depends on the growth of the platform. If Sorare can make the cards valuable then it makes Sorare valuable and users of the platform more valuable. All of this has given Sorare’s community overwhelming positive sediment despite a top team costing over $500.

Community Reaction to New MLS Cards

Wait till you see EA’s F1 announcement Twitter thread

But beyond markets, other key aspects drive fantasy and UT:

  • Stakes (winning or losing matters)
  • Persistence (more powerful overtime, build identity)
  • Skill Progression (better short-run choices overtime)

The inclusion of stakes is rather obvious, but it doesn’t just have to be money. A top ranked game in a shooter is plenty tense enough. Or top Magic the Gathering matches as comparable in tension to top chess matches. The tension comes from what a win or loss would mean. It matters. Fantasy sports has this in spades, while FIFA has an extremely competitive ranked system and a strong e-sports community.

Persistence separates fantasy sports from some live service games. Many fantasy sports run over a short time period (a weekend) whereas persistence allows players to carry over the state of a prior session into a new one. Vertical progression is a literal interpretation of this.

Vertical progression or power progression means players earn more powerful and downside-free items overtime. Unlike, say, League of Legends where players earn different characters but the characters are intended to be equally powerful (horizontal progression). In Ultimate team, earning power is achieved by increasing the OVR of the team. Each player is tuned against a wall of stat dials, like this Kevin De Bruyne card.

Persistence also allows for the possibility of investment. Players play matches because it earns them characters. Higher level characters mean higher win probability. More win probability means earning more high level characters. But that entire loop may not complete in a single session. Players cycle throughout the loop, picking-off where they last left (in given OVR, currency balance, etc) from session to session. The expectation of future output based on today’s inputs is the definition of investment. Players can expect a given unit of input to pay a stream of benefits over their engagement with the game.

Persistence also lets players build in-game identities unlike, say, Chatroulette. How people think of themselves is largely a product of other people’s collective consciousness of what they think that person. A constant username, rank, icon, etc… builds relationships, identity, and a collective consciousness. MMO EVE Online, for example, has an entire history book devoted to the drama of the in-game events and player-to-player betrayals.

Skill progression/learning refers to players getting better at key moment-to-moment action/choices. It’s distinct from horizontal progression because it doesn’t necessitate the player unlocking new “choices” or expanding their in-game decision space (i.e. do I play as this Operator or this Operator in R6:Siege?). In CS:GO, this might refer to more accurate aim. In fantasy sports, this might mean signing the right players (“Goalies are less likely to get injured, so that investment is less volatile. I’ll sign a goalie”). Skill progression/learning hits diminishing returns quickly, so few players constitute the top of the sill distribution. It’s why ranked scores for players stabilize after a few games and only increase after hundred of hours of time spent. Mastering short-run gameplay is incredibly difficult.

Sorare

Sorare has a market where prices can fluctuate based on buyer and seller demand. We also know Sorare has stakes since it’s based on the blockchain and players can take money “out” of the game. If a footballer doubles in price, the player also gets real life rich. (Editor’s note: I made it through 1500 words before mentioning Sorare is on the Blockchain. It’s the least interesting thing about the platform.)

Yes, those are real prices.

Sorare’s vertical progression is delivered via built in levels to each card. From the website:

Every time a card is played in a game week, it accumulates experience (XP). The XP will help level up the card to a maximum of 20 levels in the space of 3 years.

A card with a higher level has a higher multiple on points. If a high level card does well, it scores a lot of points. Just fielding a player allows it to become more powerful. And it’s all monetized via auction house prices.

Skill based progression in Sorare is similar to fantasy sports: make correct predictions about footballer output. In Sorare, players put together “teams” of footballers to compete for points. If a footballer does well on the real life field, the card does well and thus the player does well. It’s unclear if players will get better at these sorts of predictions overtime, but I suspect few will. Unfortunately, this anchors progression to vertical growth. FIFA shines here as mastering controls and movement returns big gains in win probability. The process of mastery is also just so damn fun. That said, a series of 3rd party tools has emerged to help players excel at the skill based aspects of Sorare, so perhaps there’s more there than meets the eye.

Closing Arguments

UT and fantasy sports are unique ways to engross fans in the world of sports. Markets, stakes, persistence, and skill progression are unique mechanics only found in games. Yet, we haven’t seen a sports game combine them in the right proportions. Sorare can be the first to do so. FIFA and fantasy sports are primed for a challenger; there’s a lot of pie to be won. But Sorare still has a ways to go. Oddly enough, monetization is solved for (auction house), engagement is a work in progress and acquisition is completely unproven. That’s the opposite of almost all start-up roadmaps. As the below chart suggests, the market opportunity will likely involve Sorare shifting a bit more up and to the left.

It’s really acquisition that needs the biggest boost, 26k Twitter followers won’t cut it. As a16z has pointed out, it’s up to Sorare to get distribution, before UT/Fantasy sports gets innovation. Do so likely rests on:

  • Launching a mobile app
  • Significant UA
  • Signing more clubs

Other innovators are starting smell the opportunity as well, NBA Top Shot is moving to a player card model with “game” like aspects. This represents another threat and could box Sorare into football rather than expanding the platform to NFL, NHL etc. Hopefully Sorare’s recent $50M Series A will be put to good use.

A Simple Model of Cosmetics and Why They’re Hard to Sustain

Exactly as Valve planned it

In Six common mistakes when moving to live-service games and free-to-play, Ben Cousins argues that cosmetic-only monetization is a mistake:

The games that make billions from cosmetic-only economies typically only succeed because of the sheer numbers of players. On a per-user basis they actually have very poor monetization, relative to games that use more aggressive methods. This is because for a multiplayer game that is built from the ground-up to be about dominating other players, the proportion of the audience who are interested in self-expression via cosmetics is rather small.

He’s right. Traditional HD developers choose cosmetics because there are no core design implications. Cosmetics can be layered into nearly any game at any stage of production. But for it to “work” massive scale is needed and even then it’s risky. It’s no wonder that very few mobile games in the top 100 grossing that use cosmetics only monetization. But I also think Ben misidentifies the challenge of cosmetics. They are certainly not about self-expression (at least the successful ones). In fact, Ben is right: male-centric multiplayer games are about domination but this is why cosmetics are viable.

Ironically enough, this is best summed up by piece on “cam girl” economics:

Men want a few things, and probably one of the biggest is winning a competition.

You see, you’re not just trying to get a guy to pay you – you’re trying to get a guy to pay you in front of a bunch of other guys. This is a super key. A man wants to feel attention from an attractive women on him, and this is made even more satisfying when it’s to the exclusion of those around him. He is showing off his power by buying your happiness.

High-level cosmetics in multiplayer games often signal domination whether or not the cosmetic is attached to skill. Apex is particularly effective at this. Consider low and high level banners:

Signal and domination comes in all shapes and sizes

The stat tracker element directly shows a player’s time commitment to the game. We can also see the more exotic colors and shapes communicating danger. Sort of like the poisonous dart frogs: cute, but deadly.

Poison Dart Frogs are killer at PvP

And finisher animations are particularly humminating since both the player performing the finisher and the one being finished must watch.

Lots of prominence or {P_j} as we’ll see!

None of this should be particularly surprising, we are political beings after all.

While these might be good observations, we need a falsifiable hypothesis of cosmetics. This allow us to make more predictive statements about how a new cosmetic will or will not sell. And to an economist this means a model!

The Model

Consider a basic model of cosmetic demand:

$$D_i = (C_i{_j} -\mu{_j})P_jT_j$$

Where:

\(\bullet\) \(Di\) is demand for the \(i\)th cosmetic \(\\\) \(\bullet\) \((C_i{_j} -\mu{_j})\) is the level of differentiation of the cosmetic from the average cosmetic in circulation at that given cosmetic vector \(j\). Just think of \(j\) as any customizable “slot”. This benchmarks the cosmetic against its closest comparables in the same way you wouldn’t benchmark a goalie against a midfielder. \(\\\) \(\bullet\) \(P_j\) is the prominence of the cosmetic vector or how “featured” the cosmetic is in-game (a watch versus an entire costume) \(\\\) \(\bullet\) \(T_j\) is the amount of time the cosmetic vector is featured on screen, both for the owner and others \(\\\) There’s a lot to tease out of this model, but much of it is beyond the scope of this post. Let’s instead focus in on the inflation problem or what’s suggested by \((C_i{_j} -\mu{_j})\).
\(\\\) There are two elements: the cosmetics level of the game and player’s individual cosmetics level. Let’s imagine a player starting a game at time \(t_1\) and with cosmetic \(C_1\). During this period, the player derives a certain amount of cosmetic utility, \(u_1\). After earning a level, a player may unlock a higher rarity cosmetic (\(C_2\)) and choose to equip it. They move to \(u_2\) at \(t_2\) as a result.

Player Cosmetic Utility Choice Model: Getting a Better Cosmetic

Now, let’s consider what happens when a player unlocks another cosmetic (\(C_3\)), but this time the cosmetic is of lower rarity. Because \(u_2\) > \(u_3\) the player does not equip it.

Player Cosmetic Utility Choice Model: Getting a Worse Cosmetic

Players are benchmarking a given cosmetic against the currently equipped cosmetic in the same slot. Overtime players earn/purchase cosmetics that give them higher and higher utility. This makes it harder and harder to sell cosmetics – each one must make the player better off then the one before it. Every player in the game is going through this journey, and as such the average cosmetic level rises, just as we modeled in \(\mu{_j}\). And when this rises differentiation falls and with it quantity demanded. That’s a long winded way of saying that cosmetic inflation hurts monetization.

Horizontal or Vertical Progression

Progression is a powerful toolkit, even for cosmetics. One way to combat the inflation problem is essentially print more money. And by this I mean introduce higher and higher rarity. Riot, for example, introduced Ultimate II skins (3250 RP).

Bold to publicly publish this as it sets a mandate, but also ensures trust and protects value.

Alternatively, the horizontal way to attack this problem is to add cosmetic vectors. Dota 2 has mastered this with things like chat lines and ping cosmetics as customizable vectors. Of course, this too will face challenges are players collect more and more items in the given cosmetic vector.

Cosmetics are a hard road to follow and inflation is always chasing developers. It’s important to think carefully about mitigation strategies and how to grow cosmetic vectors overtime as you would any other element of a live-service.

The Economics of Battle Pass are Broken. Let’s Fix It.

To be fair “dippin dots” hangs around

Monetization’s modern paradigm is defined by a direct store and battle pass (BP). After years (and ongoing) criticism of loot boxes, Fortnite re-wrote the rulebook in a way that seems to make both developers and players happy. However, it’s important to consider that at sufficient scale any monetization scheme looks like a winner. It’s unclear if Fortnite is a winner because of the pass or despite it. For instance, the collapse of Clash Royale’s monetization can be partly traced to the introduction of its own pass.

Average top grossing ranking position on US, iOS— As of May 2020 (Source)

Reports of the death of loot boxes have been greatly exaggerated as well. Of top 10 grossing free-to-play games, 8 sold loot boxes in some form. Of the top 10 premium games, 4 did. It’s going to take more work to dislodge the loot box paradigm.

And it’s understandable. If a developer isn’t going to reach Fortnite scale, battle pass isn’t a sufficient monetization solution. But it doesn’t have to be this way. We’re in the early innings of BP; by breaking down the model we can pivot the pass from an engagement to monetization driver.

The Model

We can understand BP spend depth by benchmarking it against an \(average\; daily\; monetization\;cap\). The core challenge with the pass is the relatively small spend cap. To start, consider a pass with a fixed entry cost,\(fc\), and \(N\) tiers at a given price \(y_i\). $$\\{total\;monetization\;cap}=\sum_{i=1}^N {(y_i)} +fc$$
In part, the maximum spend is limited by the cadence of the pass. \(d \) is the pass length, in days. Dividing by this gives us the average daily monetization cap (ADMC).
$$ \\ {average\;daily\;monetization\;cap} = \frac {\displaystyle \sum_{i=1}^N {(y_i)} +fc} {d} $$

We now have a model to compare different passes. Let’s examine Fortnite, Valorent, Call of Duty Warzone and Dota 2.

GameADMCTotal CapEntry CostTiersTier CostPass Length
Fortnite$1.90$160$10100$1.5084 Days*
Valorant$1.50$85$1050$1.5056 Days
Warzone$2.86$160$10100$1.5056 Days
Dota 2$6.39$1,010$102,000$0.50158 Days
100 HC = $1 *12 weeks avg.

Each game has a widely different approach. Some go for more tiers and longer length while others go for less tiers and shorter season length. While not accounted for above, it is also important to adjust for pass frequency over a year long period. Dota 2 has a 200% increase in ADMC over Fortnite, but Valve only releases the pass once a year. Warzone, on the other hand, maintains a 100 tier season over an 8 week span consistently. This would average 5-6 passes a year. Nonetheless, battle pass presents far less spend depth over other monetization vectors. In a loot box system, the spend cap is the average price price to unlock all content. In a direct store, the cap is correlated to the total sum of prices divided by the rotation cadence. In almost all games this puts ADMC north of ~$20.

However, we’ve still failed to account for opportunity costs to paint a complete profit maximizing picture.

The Pivot

Marginal Pricing

BP tier pricing is plagued by inconsistency. Most games use exponential time to complete curves, with each level taking more XP (and therefore time) to complete then the prior one. Here’s Fortnite chapter one, season four as an example:

* Assumes constant 8,000 XP Per Hour Earn Rate

And unlike RPGs, XP earn rate does not generally increase overtime. Yet, tier price is constant at $1.50 despite increasing XP requirements and constant earn rates. This is a rather odd proposition as it suggests that the usual $1.50 tier price forgoes a differing amount time depending on the tier number. By dividing by average time to earn we can understand the cost of an hour forgone per tier number. For example, tier 100 may take 6.5 hours to earn, and with a price of $1.50 this means the player pays $0.23 per hour if they choose to purchase the tier ($1.50/6.5hr to earn). Tier 1 only takes 12 minutes, suggesting a hourly price of $120 ($1.50/0.125hr to earn)!

This incentivises players to withhold tier spending until the end of the season.
More bang per tier buck. Yet players who make it to the end of the season are the most price inelastic! Why not vary tier price to maintain a constant benefit? Here’s what this’d look like assuming $1.50 buys 6,000 XP instead of a fixed tier:

Tier 1 would cost $0.10 since it forgoes very little time while tier 99 would be $9.00 since it forgoes over 6 hours.

I chose 6,000 XP to illustrate, but any amount can be chosen as the constant. At around 7,000 XP per $1.50, marginal tier pricing produces a greater spend depth then constant tier pricing (99 tiers at $1.50 = $148.50 spend cap).

This also combats the sunk cost problem. If a player is in the middle of a long tier, the price to complete the tier remains constant in hard currency. Purchasing a tier doesn’t always boost a player to the same “spot” in the next tier. In that case, it’s easy to feel like the early XP earned in the long tier is “wasted” if the player is considering purchasing the tier outright.

Setting the optimal XP/tier price is an exercise in price elasticity. If we set a tier price to say, $10,000, almost no player would buy tier. On the other hand, a price of $0.01 would produce very little revenue. Therefore, there exists a revenue maximizing amount of XP at a given price. While we may not know this exact number, it’s likely to be higher then current prices may suggest.

Tier UX

Moving to marginal tier pricing puts much more pressure on the UX wrapper. Frustratingly, despite exponential time to complete, nearly every pass displays the distance between levels as constant. All of these levels in Apex look just as easy/hard as one another:

There are two solves for this: (1) alter tier distance (increase spacing between levels consistent with XP required) and/or (2) hard level labeling.

First used by King’s groundbreaking experimentation team, hard level labeling simply labels a label as hard (as long as it actually is). This encourages booster spend by a significant margin, as it tells players it’s optimal spend in the level. This has spread to other match-3 titles and now we find super hard levels.

Simply labeling a BP tier as hard should produce the same effect. This becomes a bit trickier for exponential curves as all the hard levels are backloaded. Shifting to an s-curve design avoids this as well as being a more equitable distribution of rewards.

I got another tablet. It has a pen. Scribbles are back.

More Tiers or Less Days

A simple way to increase ADMC is increase the cadence of the pass. Instead of 12 weeks to complete, refresh the pass after 8 weeks. In Fortnite’s case this would increase ADMC by ~50% ($1.90 to $2.86). We can model the marginal effects of changing the pass refresh rate:

Adding more tiers produces a similar effect but in reverse. Instead of 100 tiers, why not Dota’s 2,000? Cost and smart use of content plays a strong role.

Alter Cost Profile

Shortening cadence or adding tiers runs into supply side problems. There ain’t no such thing as free content. Furthermore, it’s wise to consider the vector with the highest return per price of content: does adding an Epic outfit to the direct store produce more marginal revenue then adding it to the BP? But even before answering that question we need to consider the cost of producing items all together. Despite Epic employing hundred of employees and engaging in outsourcing, ~12% of tiers in the pass are “costless”. This means they consist of either currency or boosters rather than distinct items. 

Having cheap items altogether is another avenue. Apex has mastered this. Consider the stat tracker: it’s simply a tally of a particular stat. Or loading screens: an old piece of 2D art. This type of content is extremely cheap to produce. The foundational conversations of a game economy should include content vector specifications.

An Apex banner has 8 customizable slots. That’s a ton of cheap vectors!

Valve has employed clever use of changing colors on cosmetics and Fortnite adopted something similar in their new crystal levels.

Each color is a distinct item that can fill a distinct tier
“Apply All” in Photoshop

Where To

MTX design has evolved and there’s no reason to think BP won’t do the same. At the end of the day, it’s a mechanic not a destiny. In future posts, I’ll expand on pivoting the pass even more. This is just an appetizer.

Is PS+ A Good Idea? I Analyzed 200+ Game to Find Out.

Psyonix credits PS+ with vaulting Rocket League to success. Mediatonic decided to follow suit with Fall Guys. And after its success, Destruction All-Stars delayed their launch to be included in the program. Does PS+ deserve all credit it’s been given? There are some stark trade-offs worth examining.

Prima facie, PS+ faces an uphill battle – developers receive near $0 for offering up their games. Additionally, any users brought in on PS+ incur additional server costs for the developer. Should developers really be so willing to throw their titles at the service?

The service charges players ~$5 monthly to receive two random games. These are full entitlements, not subscriptions. If a player leaves PS+ they still own the games. The billed benefits of the program are:

(1) the additional sales generated from other platforms during the PS+ period
(2) the “momentum” generated after the PS+ period to additional sales on Playstation
(3) MTX revenue from additional players

I wanted to examine expected PS+ player volume as well as sales uplift afterwords. Examining cross-platform effects are possible if the game is cross listed on Steam, but I’m lazy. And I have written enough code. So I’ve posted the spaghetti code for others to do so (and spot my errors!).

The Dataset

gamestat uses the Playstation API to estimate player counts. This is particularly apt for PS+ analysis as it excludes players who “claim” a game without playing it at least once. The dataset only includes “played entitlements”. We’ll cross-reference this against a list of PS+ PS4 games. Occasionally there are regional differences but I selected the North America list for simplicity and personal bias. Remasters or collections are attributed to the base titles and are not parceled out in API data.

The Growth of PS+

The subscription has grown overtime, consistently attracting more and more downloads for the average title. Fall Guys didn’t just outperform prior titles, it obliterated past comparisons.

Hall of PS+ Fame

TitlePS+ MonthNew Entitlements During PS+ Month
Fall
Guys
2020-0824M
Call of Duty: WWII2020-059.5M
Call of Duty: MW (2016)2019-038.5M
Call of Duty: MW 22020-076.9M
Call of Duty: BO III2018-066.4M
Easy to see what convinced Activision to go F2P for Warzone

PS+ Played Entitlements Per Title

In 2019, PS+ exploded with the median title having 2.46M downloads compared to 2018’s 665k. This only increased in 2020. Some of this is probably fueled by Sony dropping the number of monthly titles from 6 to 2 in February 2018 (thereby increasing download concentration) as well as picking titles with stronger Metacritic scores.

Additionally, the service moved away from indie darlings towards titles with more publisher heft. Past year’s Call of Duties have found a second life in the program. This success has only attracted more “old” AAA titles to the program. We also find titles previously released on other platforms choosing to launch on Playstation via PS+ such as Grow Home and Super Meat Boy.

Weeks After First Playstation Launch to PS+ Inclusion

But heighted downloads are not enough, developers need revenue upside especially when considering Sony limits the ability of PS+ launched titles to appear on other platforms. We need to consider increases in PC sales and post PS+ uplift in PlayStation sales.

Is There Post-PS+ Uplift?

At quick glance, Fall Guys shows a shape decline in entitlements after the PS+ program ends. That makes sense when a title goes from $0 to $20 for a large swath of players. However, there is a “handover” month in September that looks highly correlated with the prior month’s PS+ numbers.

Fall Guys ‘Played’ Entitlements

On the other hand, the effect is only predictive of sales in the waning days of the PS+ period wherein downloads numbers are smallest. Additionally, the lift is short-lived (~2-3 weeks after the PS+ period ends).

If we examine all titles, we find a PS+ “hump”: entitlements increase during the PS+ period and then decline.

Monthly New Entitlements Per Title: Before, During & After PS+

For a clearer picture, let’s connect entitlements 1 month before and after PS+ for those games with a non-PS+ launch period (i.e. excluding Fall Guys). We should see upward sloping curves if there’s a post-PS+ lift and indeed that is the case.

Monthly New Entitlements Per Title: 1 Month Before & After PS+

Even after all those free entitlements, there’s a still a good deal of players who purchase the game.

The mean uplift was +277K entitlements when comparing the pre-PS+ month and post-PS+ month. However, there’s a whopping standard deviation of +286K entitlements, with a median uplift of 160k+ entitlements.

Not all PS+ uplifts are created equal it appears. The biggest winners were already big franchises, further riding off of brand awareness and possibly some networking effects with so many newly added players. NBA 2K 2016 was the top earner, with over +1M entitlements compared to it’s pre-PS+ haul.

TitlePost PS+ Month Entitlement Uplift
NBA 2K 16′+ 1.2M
Call of Duty: BO III+1.09M
Just Cause 3+1.04M
Call of Duty: MW 2+1.03M
The Sims 4+1.03M
Killing Floor 2+970k
Call of Duty WWII+759k
Call of Duty: MW +678k
NBA 2K 20′+637k
The Last of US+631k

Adding It All Up

What’s not accounted for are players that downloaded the game during PS+ and only first played after the PS+ period. I’m going to guess and say that accounts for 20% of the entitlements.

We also need to consider that many games pair price sales during and after the PS+ period. I’d reckon a $30 average selling price of which $20 goes to the publisher after platform margins are accounted for. For every 100k units, this works out to $1.6M net marginal revenue before severs costs are subtracted or MTX is added. PC uplift isn’t accounted for and neither is the opportunity cost of having to be PlayStation exclusive for a period of time.

The best I can tell, PS+ revenue uplift isn’t lifechanging (know your OCOR!) and the scale of the uplift is correlated with how well you were doing anyways. That said, I can’t find a decrease in post-PS+ sales and that’s good for developers nervous about giving away their game for free.

Game Companies Are Not Tech Companies Part III: The Content Problem

Part I
Part II

The only Content Problem worth saving. Do it Reed!

So maybe game companies aren’t tech companies. But as much as game companies seem to borrow from tech firms there’s even more to be said for the opposite. If Netflix, a subscription service with over 10,000 movies and TV shows, has its biggest competitor in a single game, Fortnite, then perhaps there’s more for tech to learn from games. And how games deal with The Content Problem is it’s defining characteristic. Of all forms of entertainment, games present the most compelling answer to the problem.

The Content Problem

The fundamental axiom of economics is that there are unlimited needs and wants and only limited means to fulfill them. The parallel for entertainment might consider that core content demand nearly always outstrips supply. For example, large swaths of Game of Thrones and Harry Potter fans are underserved by a couple books, movies and TV seasons. Executives try to fill the void with licensing: Harry Potter backpacks, Game of Thrones beer, etc. But filling the core content demands are impossible: it takes far more than 1 hour to produce 1 hour of Game of Thrones while the same it not true for games.

Consider the following:

The content consumed in a game like Overwatch or Clash Royale is the pursuit of strategy equilibrium and/or mastery of mechanics. A new unit in Clash Royale changes how players organize their decks, even if they don’t use the unit directly (they must counter it). This can provide hundreds of new hours of content to consume relative to the near 1 man-week of labor to produce the new unit. Therefore, the marginal content output of a given member of the 16 person (!) Clash Royale team is astronomical.

Furthermore:

The genius of PvP (Player v Player) environments is how they necessitate the emergence of a meta-game. In mathematics, Player vs Environment (PvE) resembles the field of optimization where strategies are static – one and done. PvP environments, however, resemble game theory models where it has been shown strategies evolve in an evolutionary process. This means equilibrium in PvP environments is constantly reshuffled with each balance change; the search for dominant strategies in an ever shifting equilibrium is the game itself.

The marginal product of labor for a given game developer completely outclasses a given producer on a movie or TV show by virtue of the medium, not the individual. Unlike games where assets are infinitely replicable, movies and TV face fixed constraints: Emilia Clarke or David Benioff can only be in a single place at a given time. They must also eat, sleep, socialize (sigh). Meanwhile, Captain Price faces no such constraints. There’s no more Game of Thrones to consume after the last episode cuts to black while there’s always another hour of Fortnite to play. How can Netflix and others adapt to the reality of these mediums?

The most straightforward strategy is a content arms race. Netflix continues to spend over $17B a year on original content while scooping up oodles of back catalog content. Of course, viewers must be interested in this content for it to be “effective” and the recommendation engine plays a strong role in this. But the the last episode of Stranger Things is just that, the reco engine cannot find fill the void while operating on the same indifference curve. The “more bodies” strategy to solving the content problem is expensive to execute and as we’ll see in part 4, struggles to achieve Marginal Cost = Marginal Benefit.

Reality TV is a response: less writers, editors, and CG needed to produce a given hour of content. Shows like The Amazing Race, Big Brother and Survivor can do 20+ seasons of 22+ episodes while Game of Thrones struggles with 7 seasons of 10 episodes despite having so many more crew members. Netflix’s speed of investment here is breathtaking. But the addressable audience is more limited in scope then traditional dramas. Netflix needs a bolder evolution to combat games: TV-as-a-service.

The forgotten genre of soap opera TV provides a near perfect blueprint. For those unfamiliar, soap operas are near year-round weekly serialized television shows. The unrelenting pace has resulted in popular series like General Hospital having 14,000+ episodes over 57 years. Netflix needs to heavily invest in moving shows to a similar formats: year-round production with weekly releases. There’s always another piece of content to consume right around the corner while the back-catalog for a given show is continually expanding for newcomers. In many ways, this mirrors match-3 level production. The number one reason why players churn from match-3 is a lack of new levels and a quick glance at community pages confirms this.

Sugar Rush or something, right?

King mitigates this increasing difficulty for example. This increases the time to completion but could also result in churn. Reality TV faces no such option. Another strategy is also possible however: branching narratives.

Increasingly, Hollywood is shooting movies back-to-back. It’s cheaper to continue production rather then stop and go. Why not do something similar to produce more content? In this case, shoot multiple perspectives in a given series simultaneously. Lord of the Rings production operated in a similar way with two production crews, but with a singular end product. Game of Thrones also operated in this way from a production standpoint, but again the end product was single episode. Why not dedicate an hour to each perspective? This easily multiplies a 10 episode season to 30 while holding down cost. Netflix can’t solve The Content Problem, but it can mitigate it.

Interestingly, Youtube has solved most of this problem via a two-sided marketplace. The sheer smattering of volume helps the supply-side problem even if a particular creator has a finite number of video (remember, you can still play a given game for an unlimited amount of time without “running out” of content). Youtube has encouraged users to subscribe to many different creators so regular release cadence is also accounted for as well.

Diminishing returns for linear content are extremely steep, few users will watch a film or movie more than once. Deepening the LTV of a viewer primarily come through more linear content: an extremely expensive proposition. To compete with games, TV and movies need far more supply. If technology and business models can really change creative product rather than be a vehicle for it, now has never been a better time to explore changes in storytelling.

Part IV

Can We Get Players to Tell Us Their LTV?

Nook is all too familiar is compounding growth.

Eric Suffert acutely describes the dangers of extending payback windows. At every t+1 the accuracy of LTV declines while the variance in cohort profitability increases. LTV, however, is not an exogenous variable and clever design can incentivize players into revealing their long-run time horizons within a game.

Consider the design of a many subscriptions: you can pay a lower annual fee or a higher month-to-month fee. If you’re uncertain about the subscription, then the month-to-month is more economical while if you have more certainty then the annual fee makes more sense. The choice is a huge predictor of retention: annual users are far more likely to retain then month-to-month users. The mere inclusion of this annual/month-to-month choice gives users the opportunity self-segment into more predictable cohorts. Why can’t we use the same mechanics in game design to create more predictable LTVs?

Consider two possible goods for purchase via gems in Clash of Clans: a builder or gold. The builder increases the long-run growth rate of gold while the gold itself is a temporary boost in short-run capital stock. In layman’s terms: spending 100 gems on a builder might net you 200 gold today and 1,000 gold by D30 while spending 100 gems directly on gold may only yield 700 Gold today and 0 gold by D30. The builder is an annuity that pays dividends every period, the longer a player’s time horizon the more valuable the annuity.

Players who expect to have a long time horizon in a given title have an enormous incentive to purchase “investment” goods or goods that pay dividends overtime (battle passes similar to some degree). Not doing so results in a increasing opportunity cost penalty every period due to lost compounding growth.

F2P has experimented with direct daily annuities of hard currency. They offer players a discount over the standard IAP packs but must pay upfront to receive a daily allowance. Instead of a 30-day pass, why not ramp to a quarterly or bi-annual pass? Doing so would make LTVs more predictable early in given player’s lifecycle.

How to Measure Whales

"$10.00 LTV am I right?"
“$10.00 LTV am I right?”

You’ve soft launched your game, done a UA push, and a string of hope appears. Against all odds, a dominant cohorted ARPU curve emerges! Is this this an anomaly or have you caught a whale?

The first way to examine this is to perform cointegration tests between the cohorted ARPU curves, testing for stastistical significance. It may be true the difference in the curves are real, but that doesn’t answer if you’ve caught a whale.

In 1905, Michael Lorenz developed a method for measuring relative inequality between nations known as the Lorenz curve.

Just keep saying what % of the population owns what % of the wealth and it'll make sense.
Just keep saying what % of the population owns what % of the wealth and it’ll make sense.

The F2P application is to define wealth as revenue (either on a daily or game level) and players as the population in the context of free to play games. By measuring how bent inwards a cohorted Lorenz curve is relative to other cohorted Lorenz curves we can measure the ‘whali-ness’™ of different cohorts. Even better is how this reduces to a single metric – the gini coefficient. A gini coefficient of zero indicates a perfectly equal distribution of income, 10% of the population owns 10% of the wealth, 20% of the population owns 20% of the wealth and so on and so forth. A gini coefficient of 1 is the exact opposite – a single person owns 100% of the wealth.

This translates to what % of players are responsible what % of the revenue. Measuring gini coefficients across games rather than cohorts gives more insight into how a particular game monetizes – whether it’d be whale, dolphin, or minow driven.

Actionable insights might include how effective introducing ads could be. A high gini coefficient (very few players are responsible for revenue) might mean there’s a more fertile base to monetize on.

The main insight, however, is further understanding. It’s clear that success can come about in drastically different ways in free to play games, the gini coefficient is simple way to measure that.

Get more life out of your Lifetime Value Model! A discussion of methods.

Customer-Lifetime-Value

Predicting the average cumulative spending behavior or Lifetime Value (LTV) for players is incredibly valuable. Being able to do so helps figure out what to spend on User Acquisition (UA). If a cohort of players has an LTV of $1.90 and took $1 to acquire then we’ve made money! This helps evaluate how effective particular channels of advertising are as we’d expect different cohorts of players to have different values. Someone acquired via Facebook may be worth more then some acquired via Adcolony.

But wait there’s more!

My argument in this post is that LTV has great deal of value outside of marketing. In fact, LTV might have parts more valuable then the whole. How to predict LTV can adopt numerous approaches and each approach has associated benefits. Remember, there doesn’t have to be just one LTV model!

Consider four requirements we’d want out of an LTV model:

1. Accuracy

LTV predicted should be the LTV realized. Figuring out upward and downward bias in your coefficients is important here. This gives insight into the maximum or the minimum  to spend on UA depending on the direction you suspect your coefficients are biased towards.1

2. Portability

Creating models is labor intensive and even more so when doing so for multiple games. There are particular LTV models that sweep this aside called Pareto/Negative Binomial Distribution Models (NBD). Since they’re based only on the # of transactions as well as transaction recency they don’t require game specific information. This means you can apply them anywhere!

3. Interpretability

This one’s big and perhaps the most overlooked. Consider the Linear * Survival Analysis model approach to LTV. The first part is to predict when a particular player will churn. By including variables like rank, frustration rate (attempts on particular level), or social engagement we gain insight in what’s retaining players. This type of information is incredibly valuable.

  1. Scalability

If it’s F2P then there are going to hundreds of thousands to millions of players (you hope). I’ve seen some LTV approaches that would take eons of time to apply to a player pool of this size, our LTV should scale easily.

So how do the different approaches stack against one another?

Accuracy Portability Interpretability Scalability
Pareto/NBD2  / x x
ARPDAU * Retention3 x x
Linear * Survival Analysis4 x x x
Wooga + Excel5 x
Hazard Model6 x x x

Parteo/NBD is great, but it’s hard to incorporate a spend feature (it just predicts # of transactions).7 A small standard deviation in transaction value gives this model a great deal of value and something to benchmark against. This model also makes sense if data science labor is few and far in between.

ARPDAU * Retention is probably the approach you’re using; it’s a great starter LTV. If marketing/player behavior becomes more important, the gains to scale from a approach beyond this start to make more sense.

Wooga + Excel just doesn’t scale which kills its viability, but it’s conceptually useful to understand.

Linear * Survival Analysis  gives a great deal of interpretability that also sub-predicts customer churn time. This means testing whether the purchase of a particular item or mode increases churn time is done within the model. The interpretability of linear models also means it’s easy to see different LTV values for variables like country or device.

There are many, many different approaches beyond what’s been laid out here. Don’t settle on using just one model, each has costs and benefits that shouldn’t be ignored.