It’s Not Data-Driven or Informed You Want, It’s Science

Thomas Kuhn actually wrote “The Structure of Scientific Revolutions” to figure out why he was churning from Game of War

“We want to be more data driven” or “We want to create a stronger data culture” are common refrains from organisations. Often these notions go unchallenged. The claim that organisations make better decisions with data at hand is, perhaps ironically, never investigated empirically. Dig a little deeper however, and it’s not the data that drives the value, but the better decisions made with empiricism. And empiricism, isn’t merely data, it’s a method of acquiring knowledge.

To even get off the ground analysing data, you need theory of measurement. What should we track, given limited engineering resources and raising storage costs? Claiming we should track, say payments and logins, at the exclusion of say, battle logs, implies a cost-benefit value ranking. It’s more valuable spending resources tracking payments. But these values are derived from somewhere. This “somewhere” are assumptions made about if-then statements. “If we raise price then quantity demanded will fall”, is a claim that can be shown on empirical or tautological grounds (1+1=2 because the rules define it as such). It’s true because whenever we raised price in the past quantity demanded fell or because we assume people respond to incentives (holding all else constant, higher price is a disincentive).

The methodological refinement process of these assumptions is crucial in evaluating the validity of these if-then statements. Will adding a clan system to the game maximize LTV relative to adding a login reward system? A method is not just necessary to evaluate this question, it’s impossible to do so without one.

The refinement of methodology through deliberation is at the core of what makes science, science. For science and its refinement process to even take place we need clear assumptions that produce those previously mentioned if-then claims. This is what I’ve seen Product Managers struggle with the most. Putting a feature on the roadmap has a set of implicit assumptions associated with it and these assumptions deserve to explicit so they can be debated and analysed.

As it turns out, the results produced by the scientific process are increasingly monetizable by the low marginal costs, stale fixed costs economics of software distribution. For example, if you find that people generally play your game on low-end devices and then start optimising for such devices, the benefit scales to million of players. The total addressable market is the extent of the market.

And to be clear, you don’t need empirical data to do science. Economists and Physicists make claims about the real world using accounting identities or tautologies like “something that is priced higher costs more” as a function of the rules (in this case, language).

Arguing to be data-driven or informed misplaces the value in the supply chain. Western science has never been more adored or employed then it has been now, something we all benefit from. But we need to more explicit in our endeavour – this isn’t about data, it’s about science.

Why aren’t you doing analysis on your impact?

If A Tree Falls In The Woods And Nobody Is There To Capture It On ...
“If an insight gets delivered and no one acts on it, did it make a sound?”

Performance based marketing makes intuitive sense; of course you want to optimize ROI on spend that compose 30% or more of a your firm’s expenditures. But here’s the kicker: if it makes sense for firms why not individuals? Shouldn’t we be tracking the impact of our output?

Are co-workers actually reading your analysis, concept art or brand pitches? Imagine if you knew how long internal onlookers spent reading or viewing: how might that change future output? For instance, we might be able to find the optimal length of a memo or detail of a UX mock-up. With more widespread read receipts (on email, calendar invites, PowerPoints) I’d think we’d learn discovery of information within firms is rather low because search costs are so high. I’m routinely shocked by low view counts on important internal Google docs while on the other hand observing the willingness of participants to take a firm stance on the conclusions. Many firms haven’t invested in strong intrawebs to create easy ways to access content or for creators to push them to relevant parties.

This means information travels via internal networks via word of mouth. With a given game publisher distributed over many countries and time zones, this simply doesn’t scale well. Easy consumer discovery, regular “pushing” of content, and tools assist rather than inhibit creation are paramount to spreading the gains from Haykiean localised knowledge.

Why does Epic need $1B in new funding? Is this all about Roblox?

Move over Gabe, there’s a new sheriff in town

Bloomberg is reporting that Epic is seeking to raise $500M to $1B in new funding. Tim Sweeney, Epic CEO and provocateur extraordinaire, owns 60% while Tencent owns the other 40%.

Why would Epic need more funding as it continues to rake in Fortnite money? Tim has bigger ambitions, let’s figure out what they are. As we’ve seen with Roblox, there’s incredible (and growing) money to made in user generated game content. Furthermore, consider the games spawned from mods: Counter Strike, the MOBA genre, Team Fortress, Auto Battlers, H1Z1 (which becomes PUBG which becomes Fortnite)… Imagine if you were able to take even 10% of the lifetime revenue of those titles. Despite Tim’s criticism of platform holders, Epic takes a cut of many games that run on Unreal as part of the licensing agreement. This is reportedly how Fortnite pivoted to BR: Epic got an enormous cheque when PUBG was blowing up.

The pieces have been coalescing:

  • Content creation tool in Unreal – which now scales to mobile
  • Direct to consumer distribution – Epic game store – which now scales to mobile
  • User Acquisition engine and revenue bedrock – Fortnite

But I think this play would blur more of the line between gaming and social networking. The biggest evidence comes from Epic’s acquisition of Houseparty, an app that essentially lets users join a Zoom call and play games together. How else do you explain that acquisition? It ties into their already created, but lesser known, Fortnite Party Hub app and recently announced Party Royale.​

The recent foray into non-gaming content via Fortnite concerts makes more sense in this light as well (or they’re trying to keep the Fortnite in the mainstream).

This helps explain hiring as well; in less then a year Epic has setup offices in practically every major gaming hub: Seattle, San Francisco, Stockholm, Montreal, Helsinki and Berlin. Recent jobs ads focus on the social.

Sort of Facebook circa 2010: lots of quizzes & confusion

Competition is spinning up in the meantime. Manticore games has already launched an alpha, and it’s hard to imagine Roblox being content with only owning the 10-14 yr old demo. It’s important to note this is nothing like a ‘metaverse’ as each game on the platform would retain a distinct identity, whereas a metaverse is closer to the failed PS Home or Second Life. Trying to squeeze hundreds of experiences onto one game presents a variety of complications and very little in the way of benefits.

Maybe when it’s all said and done, Fortnite let’s Tim become the social platform holder Zuckerberg always wanted to be at Facebook.

Why do contestents break the rules in Netflix’s Too Hot To Handle?

Too Hot to Handle's Francesca Reveals Retreat Life Was Strict ...

Economists like Tyler Cowen or Brad DeLong are too self-respecting to study reality shows. Fret not, this economist has no such self-respect.

Previously, we examined the economics of the reality show genre but just as interesting are the economics of the a particular reality show’s design.

To Hot Too Handle introduces of the most interesting examinations of communal property dynamics: a group prize is reduced when individuals act in their short-term private interest.

At a more practical level, the show gathers ten attractive 20 somethings into a villa in Mexico for three weeks. Cameras are littered around the villa with the exception of bathrooms (to be replaced with mics). The contestants are only informed of the rules once the cameras start rolling; if they masturbate, kiss, or engage in any sort of sexual activity the prize pool of $100,000 is reduced. It’s unclear to contestants how “expensive” each activity is or how the prize pool will be divided or won. Shockingly, interviews with the show’s producers reveal they didn’t have the rules or the costs figured out until they happened. While there’s no traditional contestant elimination process, producers will ask contestants to leave if they’re not invested in the “process”. Supposedly, the show wants to teach these singles how to form emotional rather then physical connections.

The spectacle for viewers is how hard it is for these contestants to keep in their pants – of the original $100,000, over $40,000 is lost. Seems like a lot, right? How could they give up so much money?

Well… It’s really not that much. On the face of it $100,000/10 = $10,000 per contestant. The tax situation matters greatly – U.S. contestants or those with residency in the U.S. will probably pay about 50% of that $10,000 in taxes. Interestingly, if the show took place in the U.S. rather than Mexico, all contestants would be subject to U.S. taxes. It appears to the case that the Brits and Canadians don’t face game show taxes.

On a expected payout basis, the costs are far less then they might appear:

  • $3,000 for a kiss is only $300 on a per contestant basis. Only $150 after taxes.
  • $6,000 for oral sex = $600 gross, $300 after taxes.
  • $20,000 for sex = $2,000 gross, $1,000 after taxes.

The show filmed for 3 weeks, at a max payout of $10,000 this is a yearly salary of $173k. Not bad, but many of the contestants already out gross that. Francesca Fargo is estimated to have a net worth of over $500k alone. Almost all of the contestants make money off their likeness or brand. Like Francesca, they model, sell clothing or act. Thus, building an Instagram following is directly connected to their revenue stream. Breaking the rules can help the contestants build that brand – losing out on $300 now could be much more in brand awareness later. Those without brands seemed to leave early or not attempt anything “interesting” – see Madison – a late arriver who never coupled up.

But the rules weren’t clear on splitting the prize and contestants could have been under the impression only 1 or 2 would win. Under an expected value model the payout is the same: $10,000 ($100,000 *10% chance of winning). However, if you feel as though you’re a weak contestant you might estimate yourself at less than 10% probability to win. I think this was the case for sorority girl Hailey who broke the rules a mere two episodes in and had no interest in continuing.

I think there’s room for improvement in the show’s design. It was rather strange to reveal to contestants who the rule-breakers were so early in the show. This introduced social shaming as retaliation for rule breakers, speculation and investigation makes for far more drama. If the show was about temptation, why not focus more on the money or relationships? Maybe contestants can choose to eliminate their show’s squeeze – money AND sex as tests of genuine connection. Discounting seems like a great lever for drama injection – this week sex is 50% off! Adding new contestants didn’t seem to work, everyone had coupled up by the time they got there. Subtraction or an elimination is lot more fun.

Well, here’s to a solid season two. Hopefully, the show remains tongue in cheek. But not literally – that would be a rule violation.

The Economics of Reality TV

The Circle Brazil | Netflix Official Site

In case you haven’t been paying attention, Netflix just re-discovered the reality/reality-doc genre. In 2 weeks:

  • Tiger King
  • To Hot Too Handle (international cast)
  • The Circle (4 international versions already released)
  • Love is Blind

I chuckled a bit when when Netflix remarked that they saw Fortnite as their biggest threat, but after 6 weeks in quarantine, they may have been on to something. Reality television is an awkward, but fun shot back at digital Travis Scott concerts.

The economics of reality shows have always favored cost. Casts cost $0, 1 or 2 filming locations, short shoot time (sub 6 weeks), and strong vitality potential that front loads views (no one is watching the back catalogue of Survivor). But why now, 7 years since House of Cards launched?

I think this is a simple growth parable: they’ve hit diminishing returns on drama, and this is the next highest expected marginal value item on the menu. In other words, they’ve picked the low hanging, high value fruit already. Widespread international distribution expands the watercolor effect and achieves economy of scale. Shows with live elements like Idol and Got Talent are forced in country specific renditions which limit audience reach. Subtitle support for international versions of shows like The Circle helps it reach English audiences and vice-versa. Living in Europe, you’d be surprised of the appetite for English content across non-native speaking countries.

They’re innovating reality game design as well (or at least the 3rd party studios are). Technology play a huge role in all shows, almost always as a form of communication (or lack thereof). To Hot Too Handle introduces of the most interesting examinations of communal property dynamics: a group prize is reduced when individuals act in their short-term private interest. The Circle asks how people make decisions on limited information – contestants can only communicate over text.

If the last last years have been golden age of TV dramas, maybe the next 10 will be the golden age of reality TV.

1950’s Peruvian Coke and Gacha

In the 1950’s, Peruvian inflation forced Coke to charge more per bottle of Coke. Unfortunately, their vending machines required updating to accept a new domination, a domination that was far too large of a price increase. Instead, Coke devised a probabilistic system: the machine would charge the same amount as before, but randomly refuse to give a bottle. This raises the expected price of a bottle Coke while forgoing any mechanical updating. But a miscellaneous software engineer has a better idea: raise the price of Coke, but instead randomly give the money back.

Our Surfer is ‘risk loving’

The increase in price for a ‘bottle draw’ would equal the expected payoff of of a lower ‘draw price’ of one that randomly refuses to give a bottle. This is an interesting experiment, as gacha is the number one player frustration in free-to-play games.

Anyone care to reckon which one would perform better?

Players go to their highest valued LTV: Ads are Beautiful Pareto Exchanges

Markowitz is not 5′ 9′ Milton Friedman and he resists your attempts to call him that.

Previously, I wrote about ads as a way to monetize non-payers, but there’s more to the ad exchange and what I’ll coin as ‘portfolio pumping’. It’s like portfolio theory, but not really.

These terms reference two growing phenomenon in F2P games. King is at the forefront of portfolio pumping, in which a given firm pushes a player from game to game within the firm’s portfolio.

Match-3  games in the portfolio and get $$$, ???.  Image credit: Eric Seufert

Unlike portfolio pumping, ad exchanges push players to another firm’s games. Companies like Scopely are more fond of ad exchanges.

Walking Dead ad in Yahtzee, interesting given Scopely has competing Walking Dead game.

Frequently, the ads being served are for competitor games. Why would a company show ads for its competitors? In addition, why would firms want players to move from one game in their portfolio to another? I argue the underlying explanation is Pareto Efficiency which is just a fancy term for trade.

Ads for competitor games only make sense to the ad-server if

churned player LTV < ad revenue
and to the advertiser if
acquired player LTV > ad cost

It tends to be the case that a given company will engage in both ad buying and selling. The outcome of these ad exchanges are migrations of players to the games in which they have the highest LTV; the initial allocation doesn’t matter. This process takes place in high-speed auctions where firms are constantly in the search for the maximizing the equations outlined above. The decision rule for portfolio pumping is similar, but we add some special conditions, mainly the probability of simultaneous play.

P(rLTV_{i} + nLTV_{i}) + P(nLTV_{i}) > rLTV_{i}

Where,
P(rLTV_{i} + nLTV_{i}) is the probability of playing both games simultaneously. We add up both of the LTVs in this case.
rLTV_{i} is the remaining LTV in the old game for the ith player, while nLTV is the LTV for the new game for the ith player.

This must be bigger than rLTV_{i} for profitability.

Of course, there are ways to play with this. Wooga tried altering portfolio game prompts during a player’s lifespan but found no effect.1 King continues to portfolio pump but dropped ads in Candy Crush Saga.

It’s a goddamn gorgeous process that should litter econ textbooks like lighthouses and lemons.

Re-rewriting Economic History

ediwm_3_5x5postcardfront_outlines-400x280

Will Luton argues on the dangers and solutions to F2P inflation over at gameindustry.biz.

While there are some missteps in the opening of the article, Will makes a powerful and elegant point:

…a sale can only be considered profitable if the net revenue from the start of the sale until resource equilibrium, and so demand, is restored is more than if the sale hadn’t been run. For well run sales in games with well balanced economies this should always be true.

Sales flood the economy with resources via shifts along the demand curve. Holding all else equal, this is modeled as a move from P1 to P2.

main-qimg-effd317e0b3c917bda803156142f38d6-c

The tricky part, not found in the textbook model, is time. Unlike say, refrigerators, a durable good, virtual currency is a consumable good. This means we expect repeat purchases, similar to say, gasoline. Sales in this sense pull revenue forward by changing purchase ‘schedules’ more so then a durable good. The sales are only profitable if the sale sinks resources players would have never sunk otherwise (net positive sink). In games, this is achieved this achieved via live ops. This model explains how Supercell runs their games; it’s no coincidence that Clash Royal is the first Supercell game to have sales and real live ops while their other titles have little of either. Introducing one without the other keeps net sink flat in the long run by shifting intertemporal time preferences rather than increasing the size of the ‘sink pie’ so to speak.

Progression is another confounding variable. Holding all else constant, a given item is worth less for each additional level a user is at. This is simply an artifact of rising difficulty (in the form of stronger enemies, more experience to level up etc). As a result, sales make late game players in different while making early game players better off.

Reproduced from the original article. Left: a game with constant difficulty. Right: a game with increasing marginal difficulty.
Reproduced from the original article. Left: player progression on a game with without sales. Right: a game with sales.

The insight Will offers is that sometimes this is an advantage by changing the progression path of newer players to a higher equilibrium then current late game players previously had.  This allows new players to ‘catch-up’. This sounds a lot like the Solow model. Yes, that Solow model 1. I don’t think Will models this correctly, however, as each player is not on a discrete curve as his graph on the left depicts. Even without inflation, the graph on the right is an accurate picture of a given game economy.

Consider two possible goods that could be put on sale (and thus inflated) from Clash of Clans: a builder or gold. The builder is a dramatically better purchase because it allows for more output per unit of gold or elixir (increase in technology). This shifts the growth rate of a given player up. On the other hand, the gold is a small one-time increase in capital stock that won’t scale with the game. For designers, this offers the chance to use sales as strategic instruments to alter the metagame. By offering Clash of Clan players discounts on a builder, players converge and then exceed the GDP of elder players. A sale of gold, however, merely ‘jumps’ the GDP of players without changing the long-run growth rate. This means designers can either jump the point along which new players are on the progression curve or they alter the new player curve entirely.

Sales jumps players along the curve, while a sale with 'technology' creates an entirely new curve. Also I have an S pen. Watch out art teams.
Sales jump players along the curve (A -> B), while a sale with ‘technology’ creates an entirely new curve. Also, I have an S pen. Watch out art team.

Back again

Known as 'Destroyer of Toy Islands' on Server 54G, Robert Lucas knows no mercy.
Known as ‘Destroyer of Toy Islands’ on Server 54G, Robert Lucas knows no mercy.

Unfortunately, this can deter some investment by changing inflation expectations. If players know a given dollar will have increased purchasing power later on, why make the investment now? Indeed, a 30+ paper written by Game of War players and subsequent boycotts attest to the negative side effects of perpetually trying to catch players up.

Careful consideration and analysis can make sales a valuable gameplay tool as much as they are a business one.

Eric Seufert’s best F2P blog post isn’t about F2P

screenshot-at-sep-17-20-28-48Everyone’s favorite former Rovio employee is a prolific writer on F2P games; the closest we have to a Fukuyama. Seufert has covered a range of topics, but none more important than internal organization.

Seufert argues for a number of institutional policies to surround analysts with within an organization. Frequently, analytics and data are as much about the appearance of sophistication as they are actual value adds. This need not be the case. The confusion arises over where the value of data lies. Perhaps ironically, data’s value doesn’t lie in the data, but rather in the data analyst.

In most organizations, analytics reports to product teams, a mistake, Eric argues. Often product managers face the principal – agent problem: their incentives and the companies do not align. Product managers want to successfully manage products and will present the narrative they are doing so. This is inefficient for companies who often wish to assess the true performance and trajectory of a portfolio. When an analyst’s career path depend on a product manager their narratives will often match. With organizational independence from product teams, analyst’s incentives align closer to the companies, providing more objective analysis.

Not just an accountability watchdog, real analyst value revolves around the ability to drive product roadmaps. At it’s highest order, analytics is a forward looking discipline, not a backward looking one. By experimenting and studying human behavior, analysts find levers that pull certain responses.  This creates opportunities to exploit these levers. Do currency pinches increase monetization? Are new gotcha characters or new levels driving revenue? Should we invest more in reducing load times or UI changes? Using theory driven empirical investigation analysts can move companies towards better outcomes than competitors. If organizations don’t allow analysts to pursue these questions, they’ll become cheerleaders for product teams. On the other hand, if first order information (RR, ARPU) is not accessible or automated, analysts will forever be running the hamster wheel of reporting. This is one of the more overlooked points Eric argues for.

I think this suggests a dual mandate of analysts: (1) accountability of features and (2) what features are worth developing. This creates a natural tension of not only playing the role of watchdog to product managers but partners as well. It is the duty of good analysts to navigate this relationship successfully.