A9: Results Team submission
In this assignment, use Google Analytics to run an online A/B experiment that you planned out. Statistically analyze the data. Interpret the results to guide final design changes. Reflect on the validity of our experiment. Settle on our prototype's branding, and finalize our aesthetics to reflect the essence of our design. Update our dev plan, identifying new stretch goals that we may not have time for.
The following rubric items are independently assessed. The submission will only be assessed if it is a single, well-formatted, easily-readable PDF.
Note: if you are a rare group of 2: pursue a dev plan that meets the rubric with a reasonable workload. If you are unable, you may propose an accommodation to this on Piazza; include a link to your dev plan.
Compile and Analyze
Note: for this week's studio, also draft a 30 second elevator pitch, describing your point of view, user needs, and prototype
- Google Analytics screenshot shows data from at least ten users in each condition. (note: having 22+ provides wiggle room for slightly uneven assignment or tracking lost to an ad-blocker.)
- 2-3 detailed sentences summarize your recruitment process and justify a good-faith effort to obtain your target demographic.
- Present data in a table showing observed and expected values for each condition. (Or other values as appropriate, if using test other than χ2)
- Show the correct inputs to your chi-squared or other statistical test, your calculations, the correct resulting chi-squared value or test statistic, and resultant p value. Use 3 significant digits for all numbers. See another video for help
- Summarize all formal and informal, quantitative or qualitative, feedback you received from online testing.
Interpret and Implement.
- Write a few detailed sentences interpreting the results of your analysis, speculating why the results are as they are.
- Write a few detailed sentences describing the internal and external validity of the experiment.
- Write a short paragraph delineating changes you have made, will make, or would like to make to your prototype, and mark them as such, and justify these changes based on your results (it is very rare for a test not to suggest changes)
- Write a few detailed sentences discussing what you learned from A/B testing and any other data generated from using Google Analytics
- Implement feasible changes you identified as a result of testing
- No usability bugs or console errors when reviewing the prototype. Prototype is polished as a result of online testing
- Design parsimonious with POV: interface has what’s needed to accomplish POV goals w/o unnecessary steps or clutter.
Branding & Aesthetics
- Clearly identify your prototype's final brand name and point of view to be used during your final pitch
- Prototype's final brand name reflects its functionality and final point of view. You are allowed to continue to tweak your app.
- Prototype's visual design choices about color, typography, and layout reflect user needs and final point of view. Prototype has a clean look and is easy to read (i.e. no white text on light background or vice versa) on all pages; words/buttons are not cluttered and information is not presented in such large chunks that they can be overwhelming to a user
- Prototype is visually compelling and mobile-optimized. Visual design is consistent within and across pages
- A clickable or easily typed link, or a readable, properly oriented, and complete snapshot of your dev plan. Make sure the grader has access by the deadline. All tasks are actionable, prioritized, assigned an owner, and given a time estimate. In your comments column, identify tasks that were newly added. Includes a sum of the number of hours for each teammate. Outliers should be justified
- Add new stretch goals for your prototype and identify them in the comments column
- Submit a clickable or easily typed link to your final prototype of the form “a9-projectname.herokuapp.com”. Submit a clickable or easily typed link to your A design prototype of the form “a9-projectname.herokuapp.com/path/to/page_A” and B design prototype of the form “a9-projectname.herokuapp.com/path/to/page_B” If your prototype is changed before grading is completed, you will receive no credit for the assignment
- Submit a clickable or easily typed link to your project repository on GitHub. Make sure the grader has access by the deadline
- Make email@example.com a collaborator for your Heroku app so that we can verify the commit that has been deployed
Here are some examples from prior years. Note assignments change from year to year, so use these examples as a reference, see where they succeed/breakdown, and make sure your final submissions adhere to the rubric for this year.
User Testing and A/B Prep:
User Testing Plans:
App Demos (not required for this assignment):