With all the “advanced topics” on usability engineering floating around in the blogosphere it may sometimes be hard to find information on the fundamentals, so we are providing this primer on the involvement of end users in the development process. This is basic information, but for some readers it may also be information they always wanted to know but were afraid to ask for. After all, you have to start somewhere.

Motivation for Including End Users

The basic motivation for including end users is that the benefits of this inclusion far outweigh its effort in terms of resources spent.

Including users in the right way can free resources by

• significantly shortening design discussions and even
• eliminating the need to discuss certain design issues at all.

In addition, user research can provide information about the functional scope of a system, either by

• showing that certain features can be de-scoped without impairing the user experience or by
• providing insight concerning features that would significantly enhance the system but that had not been brought up during “theoretical” discussions or “classical” requirements gathering.

All in all, the inclusion of a variety of representative end users can significantly improve the overall efficiency with which a project is executed.

Overview

User Task Analysis / Contextual Analysis

User Task Analysis aims at understanding relevant user characteristics and user workflows. An important source of information is observing several key users (of different types) in their real-life work context. That way, insight can be gained into how workflows are performed as opposed to how they should be performed. User Task Analysis usually results in rich information about how a system can be adapted to users’ requirements in an optimal way, which cannot be elicited through questionnaires or interviewing alone. During interviews or in questionnaires, the danger exists that a workflow described by users is regarded as “standard” or “desired” workflow even though it is forced upon the users because of the limitations of current system design. User Task Analysis yields a richer picture: in such a case it usually becomes clear that the ideal workflow would be different. Thus, User Task Analysis prevents a user interface design to be based on the erroneous assumption that a workflow, which has only been described verbally by users, is the ideal workflow, when in practice they need something different.

Contextual Analysis extends the focus of observation to the users’ working context and the aspects that are likely to have an impact on system usage. This can, for example, concern work interruptions due to other unrelated tasks that need to be performed or also interaction with colleagues that might be relevant for the overall system usage. Knowledge about the context of work can ultimately result in features that enhance the system and provide the users with added value, leading to more profound user satisfaction.

Formative Usability Evaluation

The goal of the Formative Usability Evaluation is to validate decisions concerning the user interface design and to provide information on how to improve / fine-tune the design. Users interact with the interface (or prototypes, respectively) by going through key scenarios (that, for example, have been identified during User Task Analysis). Critical incidents and observations as well as feedback by the users are collected to be fed back to the development process. This means that Formative Usability Evaluation is a tool for steering the processes of user interface design and system development, which provides relevant information when it is needed.

Evaluation of Workflows

Formative Usability Evaluation can be done as soon as early prototypes (paper, PowerPoint, Visio etc.) have been created. These prototypes are suited, for example, for validating workflows by asking several users to “perform” representative tasks with the prototype (for example by presenting them screens and asking them to describe how they would go about performing the task. Depending on their choices, the sequence of screens is unfolded).

In-Depth Evaluation

During later phases more interactive prototypes (for example HTML) can be used to perform a more in-depth evaluation that examines user-interface interaction in more detail. Since the development of more refined prototypes means more effort, the potential gains should justify the effort. This is, for example, the case when a decision on design alternatives is crucial and cannot be made on the basis of other information or when a new control/widget shall be offered in the user interface.

Some Tips

• Analyze and aggregate user requirements. Do not try to incorporate each and every requirement with equal weight into the user interface design, because this might well result in a user interface that will suit no one’s requirements. Workflow analyses based on observations provide hints on how to aggregate and prioritize requirements to arrive at a user-friendly interface.

• Be aware of the pitfalls of direct inquiry. In some cases, it might be better to observe users’ work or inspect their work products instead of asking them directly about feature requests. Some users tend to over-estimate their requirements, for example when requesting a rich-text editor when a plain-text editor would do.

• Choose the appropriate approach to evaluation. Make sure that the evaluation method is suited for the questions to be answered. To validate basic workflows, for example, paper prototypes can be sufficient and resources to create more advanced interactive prototypes might not be needed.

• Communicate findings efficiently. A short and well-prepared workshop might be more suitable for communicating what has been found out during evaluations than trying to feed information back into the development cycle by assembling extensive written documentation.