In December Centigrade carried out an evaluation of the racing game “Need for Speed: Rivals” for Electronic Arts – one of the biggest publishers and developers of computer- and videogames. Focus of the evaluation was the recording and analysis of the game experience under consideration of different situations in the game.
Based on their vast experience regarding the evaluation of computer and videogames, the pilot study was conducted by the Centigrade team of the North-Western branch under direction of Joerg Niesenhaus in close collaboration with the Deutsche Sporthochschule in Cologne. The Deutsche Sporthochschule runs a state-of-the-art interaction lab and contributed expertise in the evaluation of interactive entertainment via Dr. Carsten Moeller.
Goal of the study
The goal of the study was to determine how different game modes (singleplayer, multiplayer) and game elements (e.g. car crash, pursuit) of “Need for Speed: Rivals” affect the player and to establish and test a setup for the evaluation of this and similar games. Electronic Arts set value on tracking the game experience with all interactions in order to also register behavioral patterns that participants are unaware of. This makes it possible to identify which aspects of “Need for Speed: Rivals” particularly affect the player according to his self-report and how this perception relates to the physiological reactions of the player.
Every player participated in three different experimental conditions:
- First, players watched a gameplay video of the game to track their reactions to passive consumption of the game.
- In the second step players were asked to race against an AI competitor in singleplayer mode.
- In the third condition driving against a human competitor sitting next to the player increased competition.
Figure 1 shows the experimental setup in the interaction lab of the Deutsche Sporthochschule Cologne, which was installed by the game evaluation experts.
Next to the typical setup of a PlayStation 3 game console (B) and a screen for the game visualization (D) some additional technical equipment was needed. This included a computer to record the eyetracking data (A), a Tobii TX300 eyetracker (C), a camera to track the player’s behavior (E), a Varioport biosignal recorder (F) and a system to visualize the electrodermal activity (G).
The eyetracking system records the gaze behavior of the players through the Cornea Reflex Method. This method captures the reflections of several infrared light sources on the cornea and the position of the pupil to each other with a camera in order to determine gaze direction. In the recordings gaze movements (saccades) are visualized through lines. Fixations – the observation of a fixed point on the screen – are represented by circles, which expand with increasing observation length (see figure 2).
For the analysis of gaze data it can be assumed that the eyes focus on the object, which is processed by the mind (eye-mind assumption). In addition, one can use the gaze data to draw conclusions about the time needed to process the fixated object (immediacy assumption). Eyetracking is used e.g. in market research, to evaluate advertisement; in software engineering, to optimize user interfaces and in media research to gather information about the usage of media and the reaction to stylistic devices.
In the study the eyetracking method was used to determine in which situations players focus on which game elements (e.g. on their own car, a pursuer, obstacles or the route on the horizon). On this basis a connection between the current situation in the game and the activation of the player, his visual attention and the emotional reaction through the change of the pupil diameter can be found.
Measurements of the electrodermal activity (skin conductance) allow further conclusions about the degree of the inner stress and the willingness to absorb information. For this reason the skin conductance has a fundamental meaning for phenomena like stress and emotional reactions. In these moments perspiratory glands increase their activity and emit more sweat, which then lower the resistance of the skin. The perspiratory glands are controlled by the vegetative nervous system, which cannot be controlled deliberately. Therefore the measurement of the skin conductance counts as an objective procedure to trace states of activation.
In order to track the skin conductance, two electrodes were placed in the neck of the test subject and fixed with patches (see figure 3). A Varioport recorder converted the biosignals into computer-readable data. The synchronization was carried out with a marker signal on the monitor of the test subject, which was recognized by the Varioport.
In addition to objective data the test subjects were asked about their current states like activation, emotions and dominance through an image-based self-assessment scale (Self-Assessment Manikin, SAM), which was used before the start of the experiment and immediately after each experimental condition.
Questions and findings of the study
Together with Centigrade and the Deutsche Sporthochschule Cologne Electronic Arts lays the foundation for the research of conscious and unconscious perception of situations in games and the correspondent behavioral patterns of players.
The underlying questions for Electronic Arts and several other game development companies go even further: What makes digital games so attractive and distinguishes them from other forms of entertainment? How do players feel while playing a game? What differentiates successful from less successful players? Which lessons for the development of future games can be drawn from these experiments?
These questions can only be answered through further research by isolating different aspects of games step by step and putting them to the test in experiments.
The comparison between different situations in games can provide insights about which aspects of a game are of specific interest to entertain players. In the case of “Need for Speed: Rivals” we found objective proof that the competition against another human player triggers stronger emotional reactions than against an AI competitor. Further observations showed that a car crash not necessarily generated the strongest reactions. Especially the pursuit between human players in the roles of cop and racer showed particular strong psychophysiological reactions.
In addition, we discovered that players often reacted to varying degrees to typical game events like overhauling, pushing aside or crashing. These results are consistent with other experiments and research findings. For future work it would be interesting to identify and classify player archetypes (play personas) for specific games and game genres.
As already mentioned, Electronic Arts enters new territory with this experiment about “Need for Speed: Rivals”. Nevertheless, there are several international teams of scientists who also investigate the attractiveness of and reactions to digital games.
In several studies the scientists Lennart Nacke and Craig Lindley explored physiological reactions of players playing first-person shooter games and compared the biofeedback with the results of a game experience questionnaire.
Racing games were also scientifically investigated: Richard Hazlett observed teenagers during play and gathered measuring data of the activity of the electrical potential of facial muscles to relate specific game events to positive and negative emotions.
The psychologist Richard Bartle developed a test, which classifies players of massive multiplayer online role-playing games (MMORPGs) into several player archetypes based on a questionnaire about their gaming habits. Alessandro Canossa and Anders Drachen developed several methods to determine and visualize player archetypes (play personas) based on the earlier findings of Bartle.
The results of the studies conducted by Centigrade flow back to the development teams in order to use the findings to create better games. Based on these insights game developers can consider different player archetypes and play styles and generate as tense and entertaining moments as possible through well-directed use of specific game elements.
Furthermore, it is of definite interest for Centigrade to identify particular game elements and mechanisms, which can be applied in non-gaming contexts. With the gamification services offered by Centigrade, processes, e.g. in the industrial production, are enriched with playful elements in order to increase the motivation of the employees and to optimize the production efficiency. Experiments in the area of game experience research do not only have an impact on the specific game but also help to increase our knowledge about the outcome of game mechanics in games in general and other possible application areas.
Bernhaupt, Regina. Evaluating User Experiences in Games: Concepts and Methods, Springer, 2010. De Kort, Y.A.W., IJsselsteijn, W.A. and Poels, K., Digital games as social presence technology: Development of the Social Presence in Gaming Questionnaire (SPGQ). In: Proceedings of PRESENCE 2007: The 10th International Workshop on Presence, (Barcelona, Spain, 2007), 195-203.
Drachen, A., Canossa, A. and Yannakakis, G.N., Player Modeling using Self-Organization in Tomb Raider: Underworld. In Proceedings of the IEEE Symposium on Computational Intelligence and Games (CIG2009), (Milano, Italy, 2009), IEEE Computational Intelligence Society.
Hazlett, R.,Measuring Emotional Valence during Interactive Experiences: Boys at Video Game Play in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM Press, New York, 2006.
IJsselsteijn, W., de Kort, Y., Poels, K., Jurgelionis, A. and Bellotti, F. Characterising and Measuring User Experiences in Digital Games International Conference on Advances in Computer Entertainment Technology, ACM Press, Salzburg, Austria, 2007.
Isbister, Katherine and Schaffer, Noah. Game Usability: Advancing the Player Experience , Morgan Kaufmann, 2008.
Nacke, L. E., Drachen, A., Korhonen, H., Kuikkaniemi, K., Niesenhaus, J., van den Hoogen, W. Poels, K., IJsselsteijn, W., and de Kort, Y. Playability and Player Experience Research. In Proceedings of DiGRA 2009: Breaking New Ground: Innovation in Games, Play, Practice and Theory, Brunel University, West London, UK, 2009.
Nacke, L. and Lindley, C.A., Flow and Immersion in First-Person Shooters: Measuring the player’s gameplay experience. In Proceedings of the 2008 Conference on Future Play: Research, Play, Share, (Toronto, Canada, 2008), ACM, 81-88.