Even in 2024, there are still companies that believe they don’t need user research to develop new digital products because “we know our customers and know best what they need”. Most user researchers have probably come across this or similar statements and have pulled their hair out over them. Such assertions often lead to lengthy discussions and it takes a great deal of persuasion to convince sceptics of the necessity and positive impact of user research. Couldn’t it be easier? Instead of going round in circles with the usual arguments, it’s time to playfully familiarise the critics with the problem of the lack of user research. Studies show that learning through play is not only effective for children [1][2]. This is precisely why we developed the interactive user research game “You Don’t Know Your User”. By skilfully applying various user research methods, such as interviews or observations, participants learn in an entertaining way how user research can contribute to a deeper understanding of users’ real needs.
Structure of the user research game
In the game, players immerse themselves in the work of a user researcher in a UX company. They are tasked with analysing the needs of users. The fictitious client provides an actual scenario – i.e. the current state – of the process and the needs of the users for the respective product. But beware! This scenario is only based on the client’s assumptions. It is now up to the players to find out whether the users actually have the assumed needs.
The aim of the game is to collect as many points as possible by trying to verify or falsify the users’ needs or identify additional needs. It is particularly important to identify the core user need, as this serves as the basis for the subsequent design solution and therefore scores the most points. However, as in real life, there are not unlimited resources available for this. It is therefore important to choose which research method makes sense when in order to maximise the knowledge gained and use resources efficiently.
The possible user research methods
There are a variety of methods that can be used in user research. However, as the game is only intended to provide a first insight into the field of user research and to illustrate the necessity of research, we have limited ourselves to three methods that are frequently used and that we ourselves also use frequently in our daily work at Centigrade:
Monitoring
This method provides information that can be collected through observations in the real context of the target group, such as a person’s behaviour or environmental factors. It is very time-consuming and therefore also the most expensive method in the game. However, this does not automatically mean that you will obtain the most information. Thoughts and feelings, for example, can only be revealed indirectly using this method.
Interview
This method provides statements about a situation that typically occur in a conversation. It often requires less time than an observation and is therefore less expensive. However, you have to rely on the statements of the users and on their ability to articulate their motives.
Survey
This method involves short statements about a situation, as they might appear in a questionnaire. A questionnaire is generally less time-consuming and, above all, more cost-efficient than an observation or an interview. However, with this method it is not possible to ask follow-up questions, which can limit the interpretation and information content.
Game procedure
The game is played in rounds. The person who last took part in a research begins in the role of the researcher. Based on the scenario at hand, they choose which method(s) should be used for the individual steps. The corresponding units are deducted from the researcher’s budget and the other person – depending on the method – reports on the respective situation in the role of the user. Once the budget has been used up or the researcher is of the opinion that all the important information has been collected, the roles are switched. The other person then tries to check their respective scenario. At the end, we jointly analyse which user needs have been correctly identified. Points are awarded for each identified user need and the person with the most points wins the game – and hopefully now appreciates the value of user research (even) more.
Game development
After the initial idea, a lot of work has been done over the past two years to optimise the game for as broad a target group as possible. The scenario for “You Don’t Know Your User” is a central element of this. It forms the basis for the researcher’s approach, making it all the more important to find a suitable one. However, this proved to be more difficult than initially thought. Usage scenarios in the context of digital product development are often very specific and generally not intended for use outside of the project. In addition, the scenarios are usually only relevant for a specific group of people and are difficult for outsiders to understand. What is needed are scenarios that are generally known or that everyone can at least quickly empathise with.
Initially, there were several internal test runs with a wide variety of scenarios, from self-developed scenarios to scenarios generated entirely with ChatGPT. However, the use of AI in the creation of scenarios turned out to be less helpful than expected. The AI-generated scenarios were often too generalised. The internal tests showed that while AI can provide good approaches, it needs human fine-tuning before the scenarios are really playable.
In September last year, our first public test run took place at Mensch und Computer 2023 with two scenarios from the film industry. The game was well received and we received a lot of constructive feedback. One of the suggestions was to adjust the costs of the individual research methods and the idea of playing the game in blocks instead of swapping roles after each question. This is exactly the kind of feedback we wanted to achieve with the test runs – user research on the user research game, so to speak. We took the comments to heart and, in addition to a few rule and process adjustments, we also developed two completely new everyday scenarios that anyone can easily imagine: a trip on holiday and remote working, which has become popular due to the coronavirus pandemic. These scenarios have already been tested again and made their debut at the DD & UX Next conference in April. However, there is always room for improvement, which is why we take the feedback from DD & UX Next participants very seriously and will continue to improve the game. After all, “You Don’t Know Your User” should convince all critics of user research once and for all.
„You don’t know your AI“ – The future of „You don’t know your User“
The current version of the research game can be played both as a classic pen & paper version and on a PC by filling out PDFs. However, one requirement is to have a counterpart who can demonstrate their acting skills as a user. To make the game independent of human players and their acting talents, the next goal for us is digitalisation: we plan to develop a chatbot that uses AI to realistically answer the questions asked by the researcher. This will allow anyone to enjoy the game at any time and gain an even more realistic insight into the world of user research. We will incorporate the feedback we collect and take the next development step from “You Don’t Know Your User” to “You Don’t Know Your AI” – and of course report on this in a blog article.
[1] Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis. Review of Educational Research, 86(1), 79-122. https://doi.org/10.3102/0034654315582065
[2] Karakoç, B., Ery?lmaz, K., Turan Özpolat, E. et al. The Effect of Game-Based Learning on Student Achievement: A Meta-Analysis Study. Tech Know Learn 27, 207–222 (2022). https://doi.org/10.1007/s10758-020-09471-5
We have aroused your interest? Take a look at our services!
