|Applicable stages: design, code, test, and deployment.
Personnel needed for the evaluation:
|Can be conducted remotely: No
|Can obtain quantitative data: No
Cognitive walkthrough involves one or a group of evaluators inspecting a user interface by going through a set of tasks and evaluate its understandability and ease of learning. The user interface is often presented in the form of a paper mock-up or a working prototype, but it can also be a fully developed interface. The input to the walkthrough also include the user profile, especially the users' knowledge of the task domain and of the interface, and the task cases. The evaluators may include human factors engineers, software developers, or people from marketing, documentation, etc. This technique is best used in the design stage of development. But it can also be applied during the code, test, and deployment stages.Procedure
Defining the Input to the Walkthrough
- Who will be the users of the system? This should include specific background experience or technical knowledge that could influence users as they attemp to deal with a new interface. The users' knowledge of the task and of the interface should both be considered. An example user description is "Macintosh users who have worked with MacPaint".
- What task(s) will be analyzed? In general, the system should be limited to a reasonable but representative collection of benchmark tasks. Task selection should be based on the results of marketing studies, needs analysis, concept testing, and requirements analyses.
- What is the correct action sequence for each task? For each task, there must be a description of how the user is expected to view the task before learning the interface. There must also be a description of the sequence of actions that should accomplish the task with the current definition of the interface. Example actions are: "press the RETURN key", "move cursor to 'File' menu". It can also be a sequence of several simple actions that a typical user could execute as a block, such as, "Select 'Save' from 'File' menu".
- How is the interface defined? The definition of the interface must describe the prompts preceding every action required to accomplish the tasks being analyzed, as well as the reaction of the interface to each of these actions. If the interface has been implemented, all information is available from the implementation. Earlier in the development process, the evaluation can be performed with a paper description of the interface. For a paper description, the level of detail in defining the interface will depend on the expertise that the anticipated users have with existing systems.
Walking Through the Actions
The analysis phase consists of examining each action in the solution path and attempting to tell a credible story as to why the expected users would choose that action. Credible stories are based on assumptions about the user's background knowledge and goals, and on an understanding of the problem-solving process that enables a user to guess the correct action.
As the walkthrough proceeds, the evaluators ask the following four questions:
The evaluator(s) will try to construct a success story for each step in the task case(s). General conditions where a success story can be told is given next in "common features of success". When a success story cannot be told, construct a failure story, providing the criterion (one or more of the four questions above) and the reason why the user may fail.
- Will the users try to achieve the right effect? For example, their task is to print a document, but the first thing they have to do is select a printer. Will they know that they should select a printer?
- Will the user notice that the correct action is available? This relates to the visibility and understandability of actions in the interface.
- Will the user associate the correct action with the effect to be achieved? Users often use the "label-following" strategy, which leads them to select an action if the label for that action matches the task description.
- If the correct action is performed, will the user see that progress is being made toward solution of the task? This is to check the system feedback after the user executes the action.
Common Features of SuccessUsers may know "what effect to achieve":
Users may know "an action is available":
- Because it is part of their original task, or
- Because they have experience using a system, or
- Because the system tells them to do it
Users may know "an action is appropriate" for the effect they are trying to achieve:
- By experience, or
- By seeing some device (like a button) or
- By seeing a representation of an action (line a menu entry)
Users may know "things are going OK" after an action:
- By experience, or
- Because the interface provides a prompt or label that connects the action to what they are trying to do, or
- Because all other actions look wrong
- By experience, or
- By recognizing a connection between a system response and what they were trying to do