A4: Show Your Flow: Get Feedback

← A3    A5 →


In this assignment, you will conduct heuristic evaluations (HEs) of another team's paper prototype. The heuristic evaluations will be a way to highlight usability issues in your rapid electronic prototypes. Heuristic evaluations will follow the "How to Conduct a Heuristic Evaluation" and "Ten Usability Heuristics" readings by Jakob Nielsen. For a thorough explanation of the heuristic evaluation process, see the course videos.

Note: You will be given time both in studio (week 3) and in class on Tuesday to do Steps 1 and 2, but it is also recommended that you complete this over the weekend, if possible.

Step 1: (Individual) Evaluate

This part will be completed individually. You will be assigned a group to evaluate. Meet with the group and go through their prototype (i.e. pressing buttons) in any order you prefer, jotting down as many problems as you can. Don't try to be "nice" by not reporting problems--everything you write will help the group improve their interface. Try to make comparative feedback as well by noting if certain problems are found across both prototypes or just in one. If a problem exists in only one prototype, note which one by using the labels Prototype #1 and Prototype #2. You should highlight enough similarities and differences between the two prototypes so that the group receiving your feedback will understand the advantages and drawbacks of each design and know which designs they should implement in the coming weeks.

Use Nielsen's heuristics as a guide and specify what heuristic(s) each problem is related to. If you come across problems that don't fall neatly into particular heuristics, mark that no heuristics apply. Be sure to also use Nielsen's Severity Ratings for Usability Problems to add a severity rating for every problem discussed in your heuristic evaluation.

As an evaluator, you should spend about 20 minutes reviewing both prototypes. Do not wait until the last day to do your evaluations. You submission will not be accepted if you fail to find a team to evaluate by the deadline.

Step 2: (Team) Conduct

This part will be completed as a team, though only two team members need to be present. Your team will conduct three Heuristic Evaluation sessions with three different assigned evaluators who will test out your prototypes. Before meeting the evaluators, your team should know the flow of your prototype and be able to quickly find/move the pieces of your prototype to simulate the experience of using an app.

For each HE session, your team members should have specific roles:

  1. Facilitator: This team member will greet the evaluator, explain how the session works, and give a brief introduction to your prototypes. Once the session starts, this person will observe and take notes/pictures.
  2. Computer: This team member will be the computer that manipulates the prototype pieces. (i.e. When the evaluator presses a button on the prototype, the computer will change the prototype pieces to show what the app should do next)

Have a copy of Nielsen's heuristics ready for the evaluator to refer to. You may provide evaluators with this worksheet to fill out while evaluating. You may also want to create a Google spreadsheet containing the heuristics with three tabs (1 for each evaluator) so the evaluator can write their feedback directly to you. A spreadsheet shared with your team and the evaluators is recommended because it makes it easier for the evaluator to do their assignment and for you to improve your design.

Each HE session, where one evaluator evaluates both prototypes, should take 20 minutes. Therefore, the three HE sessions from three evaluators should take about a total of one hour.

Step 3: (Individual) Consolidation

Think about the general characteristics of the UI you evaluated, and suggest potential improvements to address major usability problems that you identified. Decide which problems are most important, then brainstorm potential solutions for the individual you evaluated.

Finally, distill all of this down to one paragraph where you address the major problems you all identified, as well as the potential solutions. Include a sentence or two reflecting on what kinds of things you found heuristic evaluation valuable for, and what kinds of things it's not very useful for. This is done individually.

In addition, write another paragraph, or make a list of changes, that you would make to your prototypes based on your experience evaluating another team's prototypes, and the feedback you received from other teams evaluating your prototypes. Each individual submits a paragraph, but also take the opportunity to discuss your findings with your team.

Step 4: (Team) Create a Video Prototype

Create a 1-minute video prototype that demonstrates a scenario where your app might be used and the potential users that might use it. The video should illustrate the features that your team plans to implement and how those features will be used. We'll play each group's video prototypes during studio next week!

The Walkabout video is a good example of a video prototype that highlights all of the team's intended features and the ways users can interact with those features.

Student Examples

Examples are a great way to learn, but they are not a gold standard for this class, especially as it evolves over the years. Use these examples as a reference, see where they succeeed/breakdown, and make sure your final submissions adhere to the rubric items.

Example 1 - This student did a thorough heuristic evaluation with comparitive feedback.

Example 2 - This is a weak example of a heuristic evaluation, as it wasn’t very thorough. Problems are listed for each prototype, and the heuristics and severity scores are identified. This example would have benefitted more from comparison of the prototypes.


This assignment will be submitted individually, in a single formatted pdf file with the following items:

  • A typed heuristic evaluation of another group's two paper prototypes. This comprises a list of usability issues you found, along with their severity, for each prototype.
  • A paragraph addressing the major problems identified with the prototypes and potential solutions, as well as thoughts about heuristic evaluations.
  • A paragraph/list of changes that your team plans to implement based off evaluator feedback and experience from evaluating others. Each member of your team could submit the same list in their own submission.
  • A link to the team's video prototype. Each member of your team should submit the same link in their own submission.

Submit your formatted pdf here

Evaluation Criteria & Grading Rubric

The rubric below contains criteria that are worth one point each and will be graded independently and in a binary fashion.

  1. Analysis and relations to Nielsen's heuristics aren't overly vague or tough to understand.
  2. Evaluation applies all 10 heuristics across the two prototypes, with up to 3 sections being positive findings if no violations are found.
  3. Evaluation applies heuristics to partner's prototype in a useful, organized way.
  4. Violations listed each have an appropriate severity rating that will help teams prioritize their changes.
  5. Feedback following each heuristic applied was specific and insightful in explaining the violation and how it can be improved.
  6. Feedback draws a useful comparison between the two prototypes.
  7. List of changes to prototype is provided.
  8. Relationship between list of changes and feedback from the heuristic evaluations is clear.
  9. Video prototype submitted is about 1 minute long.
  10. Video prototype clearly demonstrates the goal of the app.
  11. Video prototype clearly demonstrates different features that are meaningful to the overall goal.
  12. Video prototype shows who the potential users may be.