A8: Test your prototype
← A7 A9 →
In this assignment, we test our prototypes with real users. We capture our observations and reflect on our findings. As a result, we further streamline, debug, and expand our app. We identify and justify a design choice that A/B testing could help us make, and is suitable for our statistical test of choice. We implement both A and B versions of our design choice to which we randomly assign testers when we run our online experiment. We interpret the potential results of our experiment before doing any data collection. We assess our progress by updating our development plan.
The following rubric items are independently assessed. In general, evaluation works such that if the student/team meets, for instance, 9 out of 10 rubric items, then they have earned a grade of 90%. The assignment will not be graded unless it is submitted on a single, well-formatted, and easily readable PDF.
This assignment requires one submission per: team
- Improve the protocol that you developed in A7 if necessary, and use it to test at least two different people
- Write a few detailed sentences justifying why you chose these people to test, e.g. are they from your user base?
- Submit a digital copy of your consent forms for all testers, signed prior to testing
- Write a detailed record of your observations for each test
- Take a photo or draw a sketch of each tester using your prototype, preferably over-the-shoulder shots of the interface
- Like for needfinding, write captions for the photos, identifying breakdowns and successes, as well as tester actions
- After each test, debrief with your team and write a few detailed sentences summarizing and reflecting on your findings
- After each test, compile a list of changes as a result of your debriefing and, when possible, modify/update your prototype before running the next test
- After finishing all testing, write a few detailed sentences for three general patterns you found in tester behavior and speculate why they had these patterns
Prepare for A/B Testing
- Identify and fix remaining bugs that are either small and easy to fix, or too severe to ignore
- Write a few detailed sentences identifying one component on one page of your prototype that you want to redesign, ideally you're uncertain if the redesign is better and an online experiment could inform your final choice
- Write a few detailed sentences justifying your choice of component
- Implement both A (original) and B (redesigned) versions of your design in the same repo, accessible through different routes, e.g. "/page_A" and "/page_B"
- Write a few detailed sentences justifying A and B designs are different enough to test, as superficial changes are not accepted
- Write a few detailed sentences how the results of these changes will be measured in your online test next week, e.g. clicked or didn't click a button
- Write a few detailed sentences explaining why the measure is appropriate for a chi-squared test -- see video here for help . If you plan to use a different statistic, explain why it's appropriate
- Write a few detailed sentences suggesting why your A design might turn out to be the better design
- Write a few detailed sentences suggesting why your B design might turn out to be the better design
- Verify all tasks are actionable, prioritized, assigned an owner, given a time estimate, and have actual time spent and completion date recorded
- In your comments column, identify tasks that were newly added or removed/updated
Links to Submit
- Submit a clickable or easily typed link to your app of the form “a8-projectname.herokuapp.com”. If your app is changed before grading is completed, you will receive no credit for the assignment
- Submit a clickable or easily typed link to your A design app of the form “a8-projectname.herokuapp.com/path/to/page_A” and B design app of the form “a8-projectname.herokuapp.com/path/to/page_B”
- Submit a clickable or easily typed link to your project repository on GitHub. Make sure the grader has access
- Submit a clickable or easily typable link to -- or a readable, properly oriented, and complete snapshot of -- your dev plan. Make sure the grader has access
Here are some examples from prior years. Note assignments change from year to year, so use these examples as a reference, see where they succeed/breakdown, and make sure your final submissions adhere to the rubric for this year.
User Testing and A/B Prep:
User Testing Plans:
App Demos (not required for this assignment):
Here is a cool video of the dynamic nature of implementation plans throughout the project.