A8: Test your prototype Team submission

In this assignment, we test our prototypes with real users. We capture our observations and reflect on our findings. As a result, we further streamline, debug, and expand our app. Identify and justify a design choice that A/B testing could help make, and is suitable for our statistical test of choice. Implement both A and B versions of our design choice. (In A9, we randomly assign testers in an online experiment.) Interpret potential results of the experiment before collecting analytics data. Assess our progress by updating our development plan.

The following rubric items are independently assessed. In general, evaluation works such that if the student/team meets, for instance, 9 out of 10 rubric items, then they have earned a grade of 90%. The assignment will not be graded unless it is submitted on a single, well-formatted, and easily readable PDF.

Evaluation Rubric

Note: if you are a rare group of 2: pursue a dev plan that meets the rubric with a reasonable workload. If you are unable, you may propose an accommodation to this on Piazza; include a link to your dev plan.

    User Testing
  1. Submit updated protocol. (Avoid ”our”.) Test at least three people on a mobile device (not a laptop) using a protocol with clear, specific steps.
  2. Write a 2-3 sentences justifying why you chose these people to test, e.g. are they from your user base?
  3. Submit a digital copy of your consent forms for all testers, signed prior to testing
  4. Take a photo or draw a sketch of each tester using your prototype, preferably over-the-shoulder shots of the interface
  5. Write a detailed record of your observations for each test. Like for needfinding, write captions for the photos, identifying breakdowns and successes, as well as tester actions
  6. After each test, debrief with your team and write a few detailed sentences summarizing and reflecting on your findings. Compile a list of changes as a result of your debriefing and, when possible, modify/update your prototype before running the next test
  7. After all tests, summarize three general behavior patterns you observed in 1-2 sentences apiece.

  8. Prepare for A/B Testing
  9. No usability bugs or console errors when reviewing the application.
  10. Repeated header content included using partials rather than copy & paste.
  11. Write 2-3 sentences identifying one component on one page of your prototype that you want to redesign, ideally you're uncertain if the redesign is better and an online experiment could inform your final choice
  12. Write 2-3 sentences justifying your choice of component
  13. Implement both A (original) and B (redesigned) versions of your design in the same repo, accessible through different routes, e.g. "/page_A" and "/page_B"
  14. Write 2-3 sentences justifying A and B designs are different enough to test, as superficial changes are not accepted
  15. Write 2-3 sentences how the results of these changes will be measured in your online test next week, e.g. clicked or didn't click a button
  16. Write 2-3 sentences explaining why the measure is appropriate for a chi-squared test -- see video here for help . If you plan to use a different statistic, explain why it's appropriate
  17. Write 2-3 sentences suggesting why the A design might plausibly turn out to be better
  18. Write 2-3 sentences suggesting why the B design might plausibly turn out to be better

  19. Development Plan
  20. A clickable or easily typed link, or a readable, properly oriented, and complete snapshot of your dev plan. Make sure the grader has access by the deadline. All tasks are actionable, prioritized, assigned an owner, and given a time estimate. In your comments column, identify tasks that were newly added. Includes a sum of the number of hours for each teammate. Outliers should be justified

  21. Submit
  22. Add IxD teaching Heroku account ixd@ucsd.edu to your Heroku app as collaborator (Under “Access” tab in your app settings). username: ixd@ucsd.edu Invite: ixd@ucsd.edu
  23. Submit a clickable or easily typed link to your app of the form “a8-projectname.herokuapp.com”. Submit a clickable or easily typed link to your A design app of the form “a8-projectname.herokuapp.com/path/to/page_A” and B design app of the form “a8-projectname.herokuapp.com/path/to/page_B”. If your app is changed before grading is completed, you will receive no credit for the assignment
  24. Submit a clickable or easily typed link to your project repository on GitHub. Make sure the grader has access

Student Examples

Here are some examples from prior years. Note assignments change from year to year, so use these examples as a reference, see where they succeed/breakdown, and make sure your final submissions adhere to the rubric for this year.

User Testing and A/B Prep: (1) (note: paper prototypes not required) (2) (3)

User Testing Plans: (1) (2)

Apps: (1)

App Demos (not required for this assignment): (1) (2)