Introduction
One, in my opinion positive trend is that companies increasingly want to involve their potential users in the development. This can happen in many different ways: The spectrum ranges from market research interviews to data analysis of websites and various usability test methods. Testing is particularly important in order to obtain more intensive and detailed information about the problems and, above all, the user needs. However, especially with usability tests it can often be observed that they are only used when prototypes are already very mature. These are then often heavyweight tests in which everything is to be checked with as many participants as possible. The late timing of the tests leads to the fact that resulting adaptations involve high development efforts, since the majority of the application has already been implemented. Changes to concepts, on the other hand, are less complex. Especially in pure screen-based testing, where the user needs are left out, it can happen that the development has to be started almost from scratch. A complete misalignment of the application with the right user-needs should not occur at all. Companies are therefore faced with a whole range of challenges if the user is not to be left out in UX.
In this article, I would like to show how we solve these challenges and involve potential users early on in the project using various test methods, in order to create a basis for further development.
Challenges in Usability Testing
Challenge 1: Sample size
Many customers believe that it is only possible to sell the results to management by increasing the number of participants and the resulting supposedly higher significance of usability tests. Most of them refer to their knowledge of statistical significance and that it requires a large number of participants to prove it. Another reason why large samples are desired is the conviction that more people automatically lead to more knowledge. However, this is not necessarily the case, as I will explain further. Due to their size, larger samples lead to greater effort in planning, preparing and executing the testing. A larger sample also means a higher cost factor, which results from, among other things, the compensation of the test persons and the increased working time of the researcher. Jakob Nielsen already showed in 2000 that larger samples do not reveal more usability problems than small ones.
Small sample, large impact
In his article “why you only need to test with 5 users” Nielsen shows that 85% of all usability problems can be identified with a sample of 5 people.
Why You Only Need to Test with 5 Users – https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
According to Nielsen, for tests in the initial stages of development, even 3-5 people are sufficient to reveal problems in the navigation structure or preferences for a procedure. Even if several target groups have been defined for a product, it is sufficient if each of these target groups is represented by 3-5 people in the test. We also try to keep the testing as lightweight as possible and test with the target group of appropriately selected persons.
A small sample not only facilitates the acquisition, but also reduces the time aspect of the tests to a minimum. This in turn has the advantage that we can carry out tests with potential users significantly more often (at different stages of development) and faster.
Challenge 2: The test scope
The further a product is developed, the more functions need to be tested. But more functions also mean a longer test duration and preparation. As a result, test subjects may become less concentrated and the results may be distorted. Our experience with usability tests shows that a test should not take much longer than 30 minutes. Another consequence of complex test setups is that the researcher receives a lot of results which have to be processed all at once. In addition, the implementation of changes to the product becomes more complex.
Concept test with a paper prototype
Concept test
In order to avoid these problems, we perform concept tests in early project phases. For this purpose, click dummies are created with the help of previously defined user needs, for example in paper prototypes or in PowerPoint, and the implementation is tested. With little effort, we can quickly see whether the concept is understood by the users and where there is still room for improvement.
The goal of such a concept test is to methodically check a low fidelity prototype for strengths and weaknesses. The focus is not so much on the visual implementation but rather on the navigation and completeness of the application. Based on user stories, we develop a test scenario that checks whether the implementation of the previously identified user needs is appropriate or whether there are usage barriers that need to be removed.
But even if it cannot be avoided to implement a usability test late in the development process, there is still the possibility to test different functions in several concept tests. In this way it can be avoided that the test persons tire quickly and more concrete insights can be gained. The individual test sections must be well coordinated to avoid unnecessary overlaps or functions and interrelationships not being adequately covered.
User Acceptance Test
In addition to the concept test, we conduct a user acceptance test. Here the user friendliness and acceptance of a finished application is tested by the user. It is checked whether the application works for a user and is also perceived positively from a visual point of view. However, it is not so much a question of whether a software (technically) works or the elimination of serious usability problems. This should have been done before concept and function tests. The user acceptance test is rather a final check of a finished product or shortly before the launch of a product, where the overall experience of the product is in focus. For this reason, user acceptance tests are conducted late in the development process.
Challenge 3: The laboratory
In connection with usability tests, we often speak of test settings in specially equipped laboratories. These laboratories are equipped with expensive equipment and allow tests to be carried out under exactly the same conditions. The advantage of these laboratories is seen in the fact that user tests are standardized and many disturbing factors can be excluded. Once the appropriate technology is installed, time can certainly be saved in test preparation. A major disadvantage of such laboratories is the cost factor. You need a special equipped room that is only used for this purpose, the test users have to travel extra and are in an artificial test climate. Therefore, we work on site at the users premises or perform remote usability tests. Both variants offer the advantage that the user can use the concept or the software in its real usage context. This allows, for example, to uncover correlations with external disturbance factors or problems in the application caused by the environment (e.g. dirt on the display, different lighting, noise), which might not have occurred in the laboratory. UX researchers also get a better impression of the conditions under which the user actually uses the application. In the case of a remote test, the time flexibility is added and offers the possibility to reach even distant users.
Challenge 4: Results of usability test methods are ignored
Another reason for a distorted picture of usability test methods is that the results of the tests are simply ignored by the designers or decision makers. There are several reasons for this. One of them may be that the cost and effort of implementation would simply be too high. The decision makers want to work cost-optimized and therefore do not implement the usability recommendations. The solution here lies in the early application of concept tests as described above.
Another reason may be the preparation of the test results. Often, a lot of information is generated by these usability test methods, which do not all have the same relevance for the product. The best usability test is useless if all information is presented unfiltered in charts. Even if all team members involved are able to interpret the charts, they alone only give an indication of where problems may exist, but not yet how to solve them. Especially with qualitative results, it is even more important to prepare the results of a usability test in such a way that the most important/biggest pain points are shown and recommendations for action are given on how to improve the product based on the results. If the results of a usability test method are prepared in a way that is understandable for all team members, it is relatively easy for them to make the proposed changes to the product. Working with user-need based usability tests also helps here. User Needs not only help to create the concepts and test the implementation, but also to evaluate the results. Based on the User Needs, recommendations for action can be created, which always have their focus on the user.
Fazit
The different usability test methods offer a broad repertoire that goes far beyond testing in the laboratory. There are suitable usability test methods for all project phases and development stages of a product. Unfortunately, these are often rejected from the outset due to the prejudice of heavyweight usability testing. To counteract this prejudice, we work user-need based, in order to develop great products as light-weight as possible, where the user is the focus of attention. Even with smaller budgets, companies should not lose sight of the user, because even with simple methods, important insights can be gained in a very short time. Even if we would naturally wish that a product is developed user-centered from the very beginning, this does not mean that tests at a later stage are no longer worthwhile. Due to the diversity of usability test methods it is possible to test a product at any stage of development. The tests do not have to take place in an expensive laboratory, on the contrary, it often makes much more sense to test on site. Remote usability testing is also a good alternative. Here test persons can test a prototype comfortably from their own computer.
Analysis methods can also be used profitably in beta tests or during operation. For example, we can evaluate automatically generated data with the help of a usage data analysis specially tailored to user stories. This data can then be used as preliminary work for further testing.
We have caught your interest? Take a look at our services!
