Applicable stages: test, and deployment.
Personnel needed for the evaluation: Usability experts: 1 Software developers: 0 Users: 6	Usability issues covered: Effectiveness: Yes Efficiency: Yes Satisfaction: No
Can be conducted remotely: Yes	Can obtain quantitative data: No

 Overview

Logging involves having the computer automatically collect statisctics about the detailed use of the system. It is useful because it shows how users perform their actual work and because it is easy to automatically collect data from a large number of users working under different circumstances. Typically, an interface log will contain statistics about the frequency with which each user has used each feature in the program and the frequency with which various events of interest (such as error messages) have occured. Statistics showing the frequency of use of commands and other system features can be used to optimize frequently used features and to identify the features that are rarely used or not used. Statistics showing the frequency of various error situations and the use of online help can be used to improve the usability of future releases of the system by redesigning the features causing the most errors and most access for online help. This technique can be used at the test or deployment stages of software development.

Logging is usually achieved either by instrumenting low-level parts of the system software, such as keyboard and mouse drivers, or by modifying the software of interest. The latter is much preferred, since it makes it easier to log events of interest. If the only available data is raw input and output, it becomes much harder to analyze the higher-level events of interest for system usability, such as feature use or error situations.

If the instrumented system runs on a mainframe or on workstations with a shared file space, it is easy to collect logging data by simply copying the log files from each user at regular intervals. Otherwise, it may be necessary to collect log data through electronic mail -- either automatically or by asking the users to periodically run a small script that sends off the log file. Or, the logs may be collected by going around to the users and copying floppies.

In addition to statistical use of logging data, it is also possible to log complete transcripts of user sessions either for use in later playback or for analysis of patterns of use, such as what commands are issued next after an error situation.

A final use of logging data is to study the user's detailed use of a user interface to find usability problems that may not be apparent when observing users. The logging data can vividly show the statistics of one action (e.g. clicking on an object) from many users by integrating the logging data with the user interface.

Since the logging data only shows what the users did but not why they did it. It would be better to combine logging with other methods such as interviews, where users are shown data about their own use of the system and asked to explain their activities.

1. J. Nielsen "Usability Engineering", pp.216-220, Academic Press, 1993.