HW 4 - Evaluation

From CS260Wiki
Jump to: navigation, search

Overview

Revisit the interaction technique you implemented for HW1. Run a small user study (either in person or on Mechanical Turk), analyze and write up the results.

This homework has a shorter deadline (next week Friday); you can work either individually or in pairs.

Instructions

This homework will ask you to do a small evaluation of the Bubble Cursor, the interaction technique (most of) you implemented for HW1. (If you implemented marking menus, please pair up with a student who has a working Bubble Cursor implementation.)

Review the evaluation conducted in the the original paper: <br /> Grossman, T. and Balakrishnan, R. 2005. The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area. In Proceedings of the CHI 2005, p. 281-290.

The authors conducted two evaluation procedures: one on single-dimension alternating pointing (i.e., the classic Fitts' setup), and a second on a more realistic 2D pointing task with distractors. You will gather performance data on pointing using this second setup (as developed in HW 1; do not worry about "object pointing" as described in the paper).

At a minimum, you should vary the following independent variables:

  • Cursor type (Bubble vs Normal)
  • Movement amplitude (distance between targets - at least 2 values)
  • Target Width (at least 2 values) and/or Effective Target Width (at least 2 values) (your pick)

Collect Data about the following dependent variables:

  • Movement Time
  • Error Rate

Recruiting Participants

You can either perform this study in person or attempt to crowdsource the study on Mechanical Turk.

If you decide to conduct the study in person, recruit at least 5 users. You may not count yourself or your group partner as a user. Asking other CS260 students to participate in your study (in exchange for your own participation in their study) is encouraged.

A within-subjects design (where each user goes through multiple different combinations of independent variables) is fine.

To conduct the study on Mechanical Turk, you can export your Processing implementation (if you used Processing) as a Java Applet and upload it to a web server (contact cs260@imail if you do not have access to a server to host these files). You will have to figure out how to get users to submit their experimental data collected on the client side back to your web server. You will be able to recruit a larger number of users, but might have to break tasks into smaller subsets to keep remote users motivated to complete your tasks. Leave yourself enough time to experiment with different approaches - getting good data out of Mechanical Turk is possible, but subtle design decisions can have large unforeseen impacts.

Analyzing Your Data

Once you have collected your data, produce appropriate graphs and descriptive statistics that compare movement time and error rate for the two cursor types. Also use the appropriate statistical tests to report inferential statistics: determine whether the observed differences are significant (i.e., whether we can expect them to generalize from your sample of users to the larger population).

For background on appropriate tests, consult David Martin, Doing Psychology Experiments:

Submission Instructions

You will submit your assignment on this wiki.

Create a Wiki Page for this assignment

Begin by creating a new wiki page for this assignment. Go to your user page that you created when you made your account. You can get to it by typing the following URL into your browser:

<tt><nowiki>http://hci.berkeley.edu/cs260-fall10/index.php/User:FirstName_LastName</nowiki></tt>

Replace FirstName and LastName with your real first and last names. This will take you to the page you created for yourself when you created your wiki account. If you have trouble accessing this page, please check that you created your wiki account properly.

Edit your user page to add a link to a new wiki page for this assignment. The wiki syntax should look like this:

<tt><nowiki>Homework 4</nowiki></tt>

Again replace FirstName and LastName with your name. Look at my user page for an example. Then click on the link and enter the information about your assignment. You should upload the files described below and describe any extra functionality you implemented and want us to review.

Upload Project

  • Submit both your raw data (as an upload) and your analysis writeup (as a wiki page - below).
  • Create a zip file of your raw experimental data. Rename the zip file to firstname-lastname-hw4.zip (e.g., bjoern-hartmann-hw4.zip)
  • Upload the zip file to the Homework4-FirstNameLastName page you just created:
    • Create a new file link like this: <nowiki>File:Firstname-lastname-hw4.zip</nowiki>
    • Save the page, then click on the File link you just created to upload the zip file.

Describe your analysis on the wiki

  • On the Homework4-FirstNameLastName page you just created, write up your method, results, and discussion
    • describe the experimental setup and your user population - any information that may help another researcher replicate your experiment
    • report summary statistics and appropriate test results
    • show graphs and figures for the most important results (upload as JPG, GIF, or PNG pictures to the wiki)
    • interpret the results in a discussion paragraph.

Add Link to Your Finished Assignment

One you are finished editing the page, add a link to it at the bottom of the page with your full name as the link text. The wiki syntax will look like this: <tt><nowiki>*FirstName LastName</nowiki></tt>. Hit the edit button for the last section to see how I created the link for my name.


Links to Finished Assignments

Add your submission below this line.