A few months back, a member of Biden’s campaign team appeared on CNN. The team member describes how the campaign managed to stay politically center: staying off social media. I tweeted at the time this was great advice for game devs.
The vocal members of any community tend to dominate the feedback. And as we know, this minority is not representative of the whole. Most anyone I’ve talked to will freely admit this, but we commonly ignore it when referring to the “community” of a game. For example, job postings for Community Managers emphasize social media management.
The overwhelming supply and velocity of social media mean it dominates qualitative feedback. Yet, the community shouldn’t only refer to Reddit posters or Facebook commenters. Instead, a community is composed of everyone who plays the game. The Community Management role ought to solicit, organize, and describe the silent majority’s viewpoints. Unfortunately, firms relegate UXR divisions to the task. As the name might suggest, UXR divisions are busy with UXR studies. Too often, it’s only large firms that can support multi-member “Consumer Insights” divisions. These divisions conduct surveys and focus groups, gathering “pull” feedback (reducing the barrier to express an opinion) rather than “push” feedback (social media).
Speed and Feedback Loops
Peer-reviewed research on Twitch streamers highlights the feedback loop that makes streaming so compelling. Streamers report the near-instantaneous feedback as a primary motivation:
However, there is a second part to these experiences, in which the “chat goes crazy.” It rapidly fills up with messages like “LOLOLOL” or other humorous phrases and emoticons specific to the game or the stream. For instance, “RAISE UR DONGERS” is a popular phrase on the streamer Imaqtpie’s stream. This feedback lets everyone share in the emotional high of the moment. In addition, it reminds everyone that they are part of a unique group of people that saw something special as it happened. Streamers also reported that viewers expected them to have a webcam so that they could share their emotional reactions to these events through facial expressions.
Any social media user can empathize with the experience of a comment or like lighting up the notification bin. It’s a minor hit of endorphins at variable intervals; short feedback loops are divine, especially with other humans. I always end up revisiting Aristotle’s original claim that we are socio-political beings as a motivating factor. It’s no wonder then you’ll find nearly all developers alt-tabbed into their game’s subreddit on a studio floor. The viral cause-effect loop means social media dominates how studio members convince of qualitative feedback. Unfortunately, delivering the exact velocity of input for “silent majorities” remains expensive. Vocal communities push feedback; for silent majorities, it must be “pulled.”
Building Strong Priors
I can’t underscore how minuscule the Reddit audience is for notable titles. Consider the below sample using publicly available Steam and Reddit PSU data. When factoring in PS4 and Xbox players, Reddit represents anywhere from .5% – 4% of players. If we scaled to all every social media platform, it likely represents less than 10% of players.
It’s Also a Biased Sample
Even if Redditors were independently drawn and distributed (i.i.d), the sample is small enough to be susceptible to sampling error. It’s not that vocal communities are useless, far from it. Instead, the developers need to understand it represents a data point rather than the data. Each data point should be weighted relative to the strength of the evidence. This starts with more precise language. For example, the “Community” didn’t say x or y. Instead, “x posts with y upvotes said this or that. This represents x% of our total players.” Context and scale matter. Generalizations can sweep this under the rug. Investing more in in-game surveys or clever ways to poke at silent majority views is desperately needed as well.
In-game data, social media, CI, UXR, product management, design intuition, benchmarking…all help paint a picture of what’s going on and what to do next. If we want to make better games, we have to be vigilant in sharpening and understanding the tools at our disposal.