Evaluation Method
|
Evaluation Method Type
|
Applicable Stages
|
Description
|
Advantages
|
Disadvantages
|
Think-aloud protocol
|
Testing
|
Design, coding, testing and release of application
|
Participants in testing express their thoughts on the application while executing set tasks
|
- Less expensive
- Results are close to what is experienced by users
|
- The Environment is not natural to the user
|
Remote Usability testing
|
Testing
|
Design, coding, testing and release of application
|
The experimenter does not directly observe the users while they use the application though activity may be recorded for subsequent viewing
|
- Efficiency, effectiveness and satisfaction, the three usability issues, are covered
|
- Additional Software is necessary to observe the participants from a distance
|
Focus groups
|
Inquiry
|
Testing and release of application
|
A moderator guides a discussion with a group of users of the application
|
- If done before prototypes are developed, can save money
- Produces a lot of useful ideas from the users themselves
- Can improve customer relations
|
- The environment is not natural to the user and may provide inaccurate results.
- The data collected tends to have low validity due to the unstructured nature of the discussion
|
Interviews
|
Inquiry
|
Design, coding, testing and release of application
|
The users are interviewed to find out about their experience and expectations
|
- Good at obtaining detailed information
- Few participants are needed
- Can improve customer relations
|
- Can not be conducted remotely
- Does not address the usability issue of efficiency
|
Cognitive walkthrough
|
Inspection
|
Design, coding, testing and release of application
|
A team of evaluators walk through the application discussing usability issues through the use of a paper prototype or a working prototype
|
- Good at refining requirements
- does not require a fully functional prototype
|
- Does not address user satisfaction or efficiency
- The designer may not behave as the average user when using the application
|
Pluralistic walkthrough
|
Inspection
|
Design
|
A team of users, usability engineers and product developers review the usability of the paper prototype of the application
|
- Usability issues are resolved faster
- Greater number of usability problems can be found at one time
|
- Does not address the usability issue of efficiency
|