Module 5: Evaluation Methods (Test)
Evaluating solutions: does what we designed work?
Learning objective: At the end of this module, you will be able to conduct a usability test with a think-aloud protocol, evaluate an interface using Nielsen's 10 heuristics with severity and recommendations, and factor in accessibility and inclusion with a Latin American lens.
Estimated time: 2 - 2.5 hours
With Discover and Explore (Module 4) you collected and modeled the problem. Now you enter the Test phase of the cycle: a solution already exists —a prototype, a flow, a live product— and it's time to ask reality whether it works. Evaluation doesn't create new ideas; it puts the existing ones to the test, before getting it wrong gets expensive.
5.1. Usability Testing
Usability testing is one of the most important and frequently used methods in UX Research.
Definition and Purpose
Usability Testing is a collection of techniques used to measure the characteristics of a user's interaction with a product.
Main objectives:
- Evaluate how well users can complete specific tasks
- Identify problems they encounter when interacting
- Validate the effectiveness of design decisions
Important: Usability tests serve to evaluate, not to create. They evaluate the effectiveness of ideas already conceived.
Types of Usability Tests
| Type | Characteristics | When to use it |
|---|---|---|
| In-person moderated | Researcher and user in the same space | When depth and detailed observation are needed |
| Remote moderated | Via video conference in real time | Geographically distributed users |
| Remote unmoderated | User completes tasks alone, recorded | Large samples, quantitative validation |
| With prototype | Evaluates a design in development | Early stages, low cost of change |
| With final product | Evaluates the product in production | Post-launch, continuous improvement |
Step by Step: Conducting a Usability Test
1. Planning
- Define objectives (aligned with research question)
- Select tasks to evaluate (typically 5-7 tasks)
- Recruit representative participants (5 users detect ~85% of problems, according to Nielsen)
- Prepare prototype or product
- Create session script
2. Task Design
Tasks should reflect the user's real objectives and goals.
Fundamental principle: Tell the user the objective they have to achieve, not the steps to follow.
| ❌ Poorly designed task | ✅ Well designed task |
|---|---|
| "Click on the menu and select 'My orders'" | "You want to know what status the order you placed yesterday is in" |
| "Find the help button" | "You have a question about returns, how would you resolve it?" |
Write a brief scenario that gives context to each task:
"Imagine you bought a product 3 days ago and it hasn't arrived yet. You want to know where it is."
3. Think-Aloud Protocol
The Think-Aloud Protocol requires participants to verbalize what they're thinking while completing a task.
Instruction to participant:
"While you perform the tasks, I'd like you to think aloud. Tell me what you're seeing, what you're thinking, what confuses you, what you expect to happen. There are no right or wrong answers, we're evaluating the design, not you."
Advantage: Captures the thought process and decision-making.
Variant - Retrospective Think-Aloud (RTA): The participant performs the task silently and then reviews the recording commenting on their thoughts. Useful when precise time data is needed.
4. During the session
- Don't guide or help (unless the user is completely stuck)
- Observe non-verbal behaviors (frustration, confusion)
- Take notes of specific problems and verbatim quotes
- Ask at the end about the overall experience
5. Analysis and Reporting
- Identify problem patterns among users
- Prioritize by severity and impact
- Document with evidence (videos, quotes, metrics)
5.2. Heuristic Evaluation
Heuristic Evaluation is a usability inspection method that doesn't require users. Experts review the design comparing it with established principles.
History and Origin
The method was developed by Jakob Nielsen and Rolf Molich in 1990. Nielsen refined the list in 1994 in the paper "Enhancing the Explanatory Power of Usability Heuristics" presented at the CHI conference.
Advantages
- Fast and relatively inexpensive
- Doesn't require recruiting users
- Can be done at any stage (even with low-fidelity prototypes)
- 3-5 expert evaluators can find most problems
Nielsen's 10 Heuristics
1. Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback in reasonable time.
Example: Show a progress bar during loading, indicate how many steps remain in a form.
2. Match between system and the real world
The system should speak the users' language, with words, phrases and concepts familiar to the user. Information should appear in a natural and logical order.
Example: Use "Shopping Cart" instead of "Acquisition Basket".
3. User control and freedom
Users need a clearly marked "emergency exit" to leave unwanted states without having to go through an extended process.
Example: "Undo" option after deleting an item, visible "Cancel" button.
4. Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Example: If "Save" is used in one section, don't use "Store" in another.
5. Error prevention
Even better than good error messages is a careful design that prevents problems from occurring in the first place.
Example: Disable "Submit" button until all required fields are complete; confirm before permanently deleting.
6. Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another.
Example: Show recently viewed products; autocomplete in search fields.
7. Flexibility and efficiency of use
Accelerators (invisible to the novice) can speed up the interaction for the expert user. Allow customization.
Example: Keyboard shortcuts, advanced configuration options.
8. Aesthetic and minimalist design
Dialogues should not contain irrelevant or rarely needed information. Every extra unit of information competes with relevant units.
Example: Remove unnecessary text, prioritize primary actions over secondary ones.
9. Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), indicate the problem precisely, and suggest a constructive solution.
Example: Instead of "Error 404", say "We didn't find this page. You can return to home or search for what you need."
10. Help and documentation
Even though it's better if the system can be used without documentation, it may be necessary to provide help. This should be easy to search, focused on the user's task, and offer concrete steps.
Example: Contextual FAQs, tooltips, help chat.
Step by Step: Conducting a Heuristic Evaluation
- Select evaluators: Ideally 3-5 usability experts.
- Define scope: Which flows or sections will be evaluated?
- Each evaluator reviews independently: Avoid group biases.
- Document each problem found:
- Heuristic violated
- Location of problem
- Problem description
- Severity (scale 0-4)
- Consolidate findings: Combine individual evaluations.
- Prioritize by severity and impact.
Severity Scale
| Level | Description |
|---|---|
| 0 | Not a usability problem |
| 1 | Cosmetic problem - fix if there's time |
| 2 | Minor problem - low priority |
| 3 | Major problem - high priority |
| 4 | Usability catastrophe - must fix before launch |
Heuristic Evaluation Template
HEURISTIC EVALUATION
Product evaluated:
Evaluator:
Date:
| # | Location | Violated Heuristic | Problem Description | Severity (0-4) | Recommendation |
|---|----------|-------------------|---------------------|----------------|----------------|
| 1 | | | | | |
| 2 | | | | | |5.3. Inclusion, Accessibility and Latin American Context
Web Accessibility
Web accessibility ensures that products and services can be used by people with the widest range of characteristics and capabilities.
Why does it matter?
- Inaccessible products exclude a significant percentage of the population
- Accessibility is usability for users with disabilities
- In many countries it's a legal requirement
WCAG Standards
The Web Content Accessibility Guidelines (WCAG) are the international standard. They are organized into three conformance levels:
- Level A: Minimum required
- Level AA: Recommended (common goal)
- Level AAA: Optimal
Recommendation: Include users with disabilities in usability tests when possible.
Regionalization in Latin America
Cultural and social context influences design practices. There are no universal "best practices".
Aspects to consider in Latam:
Communication:
- Formal vs. informal treatment varies by country (tú/usted/vos)
- Local idioms and expressions
- Sensitivity to cultural topics
Payment Processes:
- Great diversity of methods (local cards, transfers, cash payments in stores)
- Variable penetration of digital money
- Different legislations by country
Example of cultural adaptation:
A travel site can promote the same destination (Santiago, Chile) differently depending on country of origin: highlighting shopping for Argentinians vs. snow and wines for Brazilians.
Practical Exercise - Module 5
Evaluate the corner store's site
Pick up the corner store case you've been working on since Module 3. You already planned the research (M3) and understood its customers (M4). Now evaluate the site it has today.
Part 1: Heuristic evaluation
Walk through the purchase flow and identify 5 problems using Nielsen's 10 heuristics. For each one note: heuristic violated, location, severity (0-4), and a recommendation. Use the template from section 5.2.
Part 2: Mini usability test
Ask 2-3 people to complete a real task ("buy two products and get all the way to the end") while thinking aloud. Remember: you give them the goal, not the steps. Don't guide.
Part 3: Contrast the two methods
Do the problems you found on your own with the heuristics match the ones that showed up when people actually used the site? That difference is exactly why you need both methods: one anticipates, the other confirms.
Keep your raw findings. In Module 7 you'll analyze, synthesize, and communicate them.
Module 5 References
- Nielsen, J. (1994). Enhancing the Explanatory Power of Usability Heuristics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94).
- Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
- Krug, S. (2014). Don't Make Me Think, Revisited. New Riders.
- W3C. (2018). Web Content Accessibility Guidelines (WCAG) 2.1.
Evaluation methods in detail
Evaluation without users
Evaluation with users