The goal of this assignment is to test your app prototype with two people to further streamline your app and to inform the online A/B testing that you will do for A8.
Observe at least two different people testing your app prototype in-person. Try to find representative testers whom you would expect to use your app in real life.
One person will facilitate the test and interact with the user, and the rest of the team will be in charge of taking notes/photos/video/etc. The facilitator should use the protocol that you developed in A6. Don't forget to have your user sign a printed consent form before beginning the test (some user testing may be unethical without informed consent). Unlike in A3's heuristic evaluations, this time around your user will not be writing down the problems they find for you. It's your job to learn what the people testing your prototype are thinking, and what mental models they are forming; the feedback they provide you will be very valuable for your next iteration. Your main goal here is to find ways to improve your interface.
Submit your testing protocol and signed consent form for each participant. Immediately after each test, do a quick debrief with your team and write down any reactions or thoughts that came up during the testing session.
Take a photo or draw a sketch of each tester using your prototype. As with the A1 needfinding assignment, these photos with captions should show breakdowns and design opportunities. Contextualize them by capturing the action, e.g. by using over-the-shoulder photos to capture the user using your app, and by showing the setting. Look for other breakdowns and pain points in your interface and try to understand what the problems are and how you might fix them. When possible, modify/update your app prototype before testing on the next participant so that you can get fresher data.
Watch the In-Person Experiments lecture video for some practical tips about running these sorts of experiments.
After testing, take some time with your team to reflect on your findings. Discuss as a team and define some general patterns in people's behavior. When you identify some interesting points, talk deeply about them—ask each other questions, recreate the different tests, analyze the decisions people made, other paths they could have taken, and so on.
Submit a detailed and understandable list of changes that you will implement as a result of your testing and discussion, with justifications. Fix the bugs that are either small and easy to fix, or too severe to ignore. Make sure that you do this before moving on to the next step of this assignment.
Choose ONE non-trivial component of your prototype to redesign in order to either resolve a breakdown or provide a potentially better solution than what was created before, as informed by user testing. To do this, make a duplicate of the specific webpage where the change will take place and change the URL route. For example, if you are redesigning a component of the homepage, keep the current homepage ("/homepage") and create a second page with a route "/homepage2". Do not overwrite the current page and do not create an entirely new repository for this redesign! You will be submitting both the original URL and the redesign URL.
The redesign will be used for online A/B testing in the next assignment (A8), which requires the use of the chi-squared statistical test. Therefore, the aspect you choose to redesign must be testable using the chi-squared test, which usually tests an aspect that can be binarily classified (i.e. user clicked or didn't click a button). The redesigned component must also be noticeably different from the original design in a substantial way—changing the background color as your redesign is not enough. We highly suggest that you read A8 in detail to understand what kind of redesign can be measured by chi-squared tests during online A/B testing. See the lecture video on "comparing rates" for more information.
Submit a description of the online A/B test that you plan to run for next week's assignment (A8); you do NOT need to run the online A/B test yet. The description should explain what the differences between your original and redesigned versions of the component are (from Step 3), and how the results of these changes will be measured in your online A/B test next week. Also include your best guess of all possible outcomes and interpretations for next week's test, as interpreted with implications on the design of the prototype.
Just as we've been doing in the previous weeks, update your development plan. Add new tasks for this week and the following weeks while marking when existing tasks have been completed. Add stretch goals that you find feasible and adjust other tasks that may be out of reach.
Here are three student examples (note that the assignment might have differed slightly in prior years):
By Mike Davison, Community TA and UX Project Manager
Testing your high fidelity prototype with users closes the circle. It is vital to ensure your solution meets the needs identified during the first assignment, and that the agency has not simply spent months drifting further from the problem.
It also allows you third party reflection and suggestions for tweaks to the design. Everything we learn here and correct is a problem we don’t have to live with because it has already been coded and is too costly to change....it’s a valuable phase of the process.
Remember this - feedback is not criticism, feedback is not personal. User centered design works best when pride is left aside, and the feedback of others is incorporated into your design thinking!
For this assignment, ONE person will submit the assignment for their team, listing every team member's name and student ID number (PID) in the assignment submission.
Your write-up will contain the following in one single PDF:
Submit your single formatted PDF in Gradescope.
The rubric below contains criteria that are worth one point each and will be graded independently and in a binary fashion.