How Games Will Play You

JULIAN TOGELIUS
Associate Professor, Department of Computer Science and Engineering at the New York University Tandon School of Engineering

It may be an overstatement to say that everyone plays video games, but it’s probably fair to say that most of us play them [1]. At least in rich countries. Even if you’re part of the diminishing population that doesn’t play video games for entertainment, you may be engaging with them for other purposes, especially as “serious games” are increasingly being used for training, education, simulation, and even advertising. But no matter your opinion of video gaming on the whole, here’s the thing: video games will likely have their greatest impact on society not in how they entertain or teach us, but through how they spy on and adapt to us. Let me explain.

If you, like me, played video games in the eighties and nineties, you will have noticed that much has changed in today’s games. Most obviously the graphics are more detailed, games are more complex, and frequently (but not always) they’re better written and produced. What is subtler and perhaps harder to notice is that almost all of today’s video games make use of the internet to “phone home” to their developers—sending back data about how they are being played. This data provides developers with feedback about design aspects that work and don’t work, which helps them to improve their games. Note that “improve” here may mean both making the game legitimately better, and figuring out ways to make more money, perhaps including requiring users to pay more through in-game purchases. While some games simply send back basic information about how long you played and whether you bought anything, others collect much more detailed information—sometimes down to the level of every button you pressed.

Something else that is not obvious to the casual observer—or player—is exactly how much information we give away about ourselves when we opt in to play. Interaction with online games is a high-bandwidth interaction. A single game session might involve thousands of button clicks or controller or mouse movements, each expressing a decision of some kind. Most of these decisions are minor, such as whether to go out of the way to pick up a coin or not; but others are major, maybe involving whether to help a game character in need, or what kind of taxation policy to employ for the simulated country you lead.

jt1.jpeg

It is common belief that we somehow become different people when we play games. It is certainly true that we can behave differently. For example, the vast majority of people who play first-person “shooters,” as Call of Duty, thankfully don’t run around in their real lives shooting people. Meanwhile, introverted people who stay away from real-world parties may happily join 50-person raiding parties in multiplayer role-playing games such as World of Warcraft. Games are, to a large extent, a space where we can live out our fantasies.

That being said, when we enter a game world, we still bring our aptitude, reaction speed and dexterity, general physical and mental health, personality, fears, ideology, and everything else that defines us. We are still ourselves, even when playing the role of a game character.

Therefore, since we constantly hand over implicit information through the hundreds or thousands of little decisions we make when playing video games, it would seem that we could learn a lot about people from analyzing how they play games. And as it turns out, we can. We don’t yet know how much, but what we can learn seems to depend on the game.

Nick Yee, formerly of Stanford, and his colleagues investigated the relationship between personality and how players played World of Warcraft [2]. This is a role-playing game where social interaction is an important component, and players can choose between many different character classes (such as Paladin, Mage, or Warrior) and customize them in complex ways by equipping them with weapon, armor and special items. Using the OCEAN (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism) taxonomy, a widely accepted model in personality psychology, Yee and colleagues found that the character class that players chose in the game, the  equipment they acquired, and the quests they performed all correlated with real-world personality traits. For example, people who are extraverted tend to engage in group activities and participate in raids more often; introverted individuals prefer more solitary activities, including solo quests but also cooking and fishing. Conscientious people tend to be hoarders in-game, and those who are less conscientious often die from falling. Those who are more open to experience, which basically means they are more curious, tend to have lots of different in-game characters and spend more time doing off-beat activities rather than the main quest of the game.

Partly inspired by this work, my colleague Alessandro Canossa and I, together with a student of ours named Josep Martinez, decided to investigate how in-game behavior reflects a person’s life motives. Psychologist Steven Reiss has developed a theory of 16 “basic desires” in life, such as acceptance, order, curiosity, and family, and a questionnaire to assess these [3]. We had more than 100 individuals complete this questionnaire, and then correlated their answers with the records of their Minecraft playing. Minecraft is an extremely popular, open-world game that centers around mining the ground for materials, which you can then use to craft items and construct buildings. While the game features combat and a system of quests, it gives the player almost complete freedom in choosing what they want to do in the game. Analyzing the data with both classical statistics and modern machine learning methods, we found that it was replete with correlations between things done in the game and players’ real-life motives according to the questionnaires. Often it made sense: people who valued independence would rarely finish the game, those who were most curious in the real world would do lots of exploration, people who valued family would build nicely protected homes and avoid crafting extravagant items such as gold swords. Other times the correlations made no clear sense. But by putting all the variables together and using machine learning algorithms such as neural networks or support vector machines, we could predict some life motives (such as honor, romance, and independence), as well as gender and age, with good accuracy. No individual in-game behavior was a spectacularly good predictor on its own, but taking them all together gives us some very good predictions.

Both Minecraft and World of Warcraft are games which allow a high-degree of self-expression, in the sense that there are many ways of playing them. But what about games which are usually considered less expressive, such as first-person shooters? What can we learn about a person from analyzing the way the play a game which is mainly about running around and shooting others? Quite a lot, apparently. Shoshannah Tekofsky and Pieter Spronck, along with their collaborators at the University of Tilburg, studied a population of more than 10,000 players of the popular first-person shooter Battlefield 3 [4]. They found correlations between play style and age, gender, national origin, and even personality. In most cases the individual correlations are rather weak: Germans are more collaborative than Americans, though the difference is minor. Even so, by using machine learning it is possible to combine the various features to infer facts about the player with much higher precision.

This is almost certainly just the tip of the iceberg.  Could we predict political orientation, sexuality, ethnicity, and health status from how you play in games? I have not done any research on this (and I do not intend to), but it seems plausible to me. And it is almost certain that others are doing this research, though we are unlikely to see it published as those who would be interested in developing such models (from game developers to intelligence agencies) would probably not want to disclose their existence.

jt2.jpg

We already know that similar things can be predicted from what we post on social media, or our mobile phone usage patterns. But from a privacy perspective, a key difference is that when we post to Facebook or text our friends we are mostly aware of what kind of personal information we are handing over to the powers that be (network providers, Facebook, national security agencies, etc.) for their data mining perusal. When we play games, we may not be aware that we are revealing information about ourselves; we think we are simply acting in a made-up world, inside the magic circle of play. But in reality, we are letting the game spy on us through the magnifying glass of machine learning.

In my view, this is concerning. I think we should have the right to know what game developers know about us. Maybe, though I remain conflicted about this, we should also be able to force them to erase whatever data they hold about us. But this may not be easily achievable, as the models about you and many other may have been integral to how the game function.

Yet another issue is what game developers (or whoever has access to the game data) should be able to do with the models they created based on player data. We are already accustomed to search platforms and social media companies serving us advertisements based partly on what they think we like, and it’s very likely that we’ll see this inside games as well. But should the game developer also be able to sell these models to search engines, for example, to serve you ads outside the game? Or share them with the police, or with your potential employers?

In another case altogether, it is also possible to use machine learning to build generative models of your behavior, in other words to automatically create bots that play just like you [5]. It’s worth asking if the game developer should have the right to use your playing style in the game. If the game developer wants to offer other players the opportunity to play against (or with) a digital “you,” complete with your signature moves and typical quirks, even when you’re not playing, is that within their rights?

One of the reasons game developers (and game AI researchers, such as me) are so eager to investigate various ways of modeling players is that it makes adaptive games possible. Developing large games is extremely expensive, and the gaming demographic is more diverse than ever in terms of background, skills, and aesthetic preferences. It therefore makes sense to create games that can adapt automatically to players, selecting what challenges and content to present to particular players based on their best models of those players [6]. This could genuinely lead to better games. But there’s also the danger of importing the “filter bubble” effect we see so clearly in social networks, where we only hear from people who think like us and never breach the confines of our comfort zone. This might lead to us only being presented with the kind of game content and challenges that we already know and like, discouraging us from experiencing the wider spectrum of interactions.

You may think that, in the greater scheme of things, it’s not very important what happens inside games. But for young people today, video games are one of the most important forms of media, if not the most important. And, as pointed out earlier, games are increasingly used for non-entertainment purposes such as training, advertising, and simulation. In either case, what happens in games can have very real repercussions for the world outside them, particularly as modern machine learning methods give us the ability to infer so much about the person behind the keyboard or joystick. Please do not let this stop you from having fun next time you sit down to play your favorite game, but do think about how the game might play you as you play it.



Endnotes 

1. Entertainment Software Association (2018). Essential Facts About the Computer and Video Game Industry. http://www.theesa.com/about-esa/industry-facts/

2. Yee, N., Ducheneaut, N., Nelson, L., & Likarish, P. (2011). Introverted elves & conscientious gnomes: the expression of personality in world of warcraft. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 753-762). ACM.

3. Canossa, A., Martinez, J. B., & Togelius, J. (2013). Give me a reason to dig Minecraft and psychology of motivation. In 2013 IEEE Conference on Computational Inteligence in Games (CIG). IEEE. The underlying theory by Reiss is described in: Reiss, S. and Haverkamp, S. M (1998). Towards a Comprehensive Assessment of Fundamental Motivation: Factor Structure of the Reiss Profiles, Psychological Assessment 1998, Vol. 10, No. 2, pp. 97-106.

4. Tekofsky, S., Van Den Herik, J., Spronck, P., & Plaat, A. (2013). Psyops: Personality assessment through gaming behavior. In Proceedings of the International Conference on the Foundations of Digital Games.
Tekofsky, S., Spronck, P., Goudbeek, M., Plaat, A., & van den Herik, J. (2015). Past our prime: A study of age and play style development in battlefield 3. IEEE Transactions on Computational Intelligence and AI in Games, 7(3), 292-303.

5. Holmgård, C., Green, M. C., Liapis, A., & Togelius, J. (2018). Automated playtesting with procedural personas through MCTS with evolved heuristics. IEEE Transactions on Games.

6. Yannakakis, G. N., & Togelius, J. (2018). Artificial intelligence and games. New York: Springer.

Dipayan Ghosh