Performance Measurement


Applicable stages: design, code, test, and deployment.
Personnel needed for the evaluation:
Usability experts: 1
Software developers: 0
Users: 6
Usability issues covered:
Can be conducted remotely: No Can obtain quantitative data: Yes


This technique is to used to obtain quantitative data about test participants' performance when they perform the tasks during usability test. This will generally prohibit an interaction between the participant and the tester during the test that will affect the quantitative performance data. It should be conducted in a formal usability laboratory so that the data can be collected accurately and possible unexpected interference is minimized. Quantitative data is most useful in doing comparative testing, or testing against predefined benmarks. To obtain dependable results, at least 5 user participants are needed, while 8 or more participants would be more desirable. The technique can be used in combination with retrospective testing, post-test interview or questionnaires so that both quantitative and qualitative data are obtained. The technique can be used in the following development stages: code, test, and deployment.

Define the Goals

Conduct the Test

Make sure that there won't be unexpected interruption during the test. Conduct pilot test to make sure that the tools and the techniques for data collection work well. When possible, the test should be video-recorded to support data collection, so that some data can be collected or verified after the test by reviewing the video recording.

Even though this technique is aimed to collect quantitative data, it should be noticed that it's very important to collect qualitative data to uncover the user's mental process and other information behind the quantitative data and take them into account while drawing the conclusions.

Analyze the Data to Draw the Conclusions

To compare with a benchmark value (for ordinal, interval, or ratio data), mean or median can be calculated, together with standard deviation, standard error of the mean, and the confidence intervals (for details see reference 2).

To compare the data from different user interfaces, some kind of inferential statistics test can be performed (for details see reference 2).


  1. J. Nielsen "Usability Engineering", pp.191-194, Academic Press, 1993.
  2. N. Soken, B. Reinhart, P. Vora, & S. Metz "Methods for Evaluating Usability" (Section 5B), Honeywell, Dec. 1993.