HW 2 - Depth Camera Interaction Technique

From CS260 Fall 2011
Jump to: navigation, search

Due: Friday, September 23, 5pm 10 points (up to 12 with extra credit)


In this homework assignment you will implement gesture-based menu navigation using the Microsoft Kinect sensor. You will submit your code, and a video that demonstrates the interaction. This assignment can be done in pairs or alone (pairs are strongly encouraged).


  • zip file of Source Code
  • Demonstration video: Narrated Screen recording
  • One paragraph writeup on Wiki that describes your implementation.


  • Each team will get a Kinect sensor in class on Monday, 9/12/11.
  • You can use the PCs in 125 Cory to work on this assignment, or your own laptop.
  • We strongly suggest you use the Official Kinect SDK from Microsoft. You are free to use open source drivers, e.g., for OpenFrameworks (C++) or Processing but we cannot support these in as much detail. The official Kinect SDK will only run on Windows 7 (bootcamp of Macs is ok; virtual machines do not work).
  • Plan to spend at least one day on getting your development environment up and running. Get started early!

Menu Navigation

Implement at least one of the following gesture-based menu navigation techniques:

At a minimum, your implementation should do the following:

  • Use the following menu hierarchy: Homework2-MenuHierarchy.
  • On Application start, show the root level of the menu.
  • Enable users to select one of the options; if this option opens a sub menu, show the submenu; otherwise, show a message that the option was successfully selected, then return to the root of the menu.
  • During navigation, show graphical feedback which item is currently in focus, if any. (This usually requires you to create three different graphics for each item: not in focus, in focus but not activated, and activated).
  • Enable users to go back a level in the hierarchy if they are in a submenu.

For extra credit (up to 2 extra points), you may implement the following advanced features:

  • Implement two different techniques, and compare (qualitatively) how easy, efficient, error-prone they are. You do not need to run a study, just show that both techniques work and report your impressions of their relative strengths and shortcomings.
  • Smooth animations and rich visual feedback.


For this assignment, it is important to show both the physical actions a user is taking and the result of those actions in the user interface. You can achieve this in at least three different ways:

  1. Recording a single live video over the shoulder of the user so you can see both the user's hands and the screen. Getting the lighting correct is tricky. Both screen and hands must be clearly visible. Turn down screen brightness and turn up room lighting if either the hands are too dark or the screen is washed out.
  2. Produce a picture-in-picture video where you show both a screen recording and a live video. This is more work, but the quality of the result will be significantly higher.
  3. Show the color video feed of the Kinect in a corner of your UI. This will give you picture-in-picture without having to edit two separate videos in post production.

You video must either be narrated with a clear, understandable audio track; or it must have complete subtitles that explain what is happening.

This video is more complicated to produce than the screencast of HW1. Plan to spend more time on it.

Submission Instructions

You will submit your assignment on this wiki.

Create a Wiki Page for this assignment

Begin by creating a new wiki page for this assignment. Go to your user page that you created when you made your account. You can get to it by typing the following URL into your browser:


Replace FirstName and LastName with your real first and last names. This will take you to the page you created for yourself when you created your wiki account. If you have trouble accessing this page, please check that you created your wiki account properly.

Edit your user page to add a link to a new wiki page for this assignment. The wiki syntax should look like this:

[[Homework2-FirstNameLastName|Homework 2]]

Again replace FirstName and LastName with your name. Look at my user page for an example. Then click on the link and enter the information about your assignment. You should upload the files described below and describe any extra functionality you implemented and want us to review.

Upload Project

  • Your submitted project must include both the full source code as well as the executable of the working application.
  • Create a zip file of your project tree. Rename the zip file to firstname-lastname-hw2.zip (e.g., bjoern-hartmann-hw2.zip)
  • Upload the zip file to the Homework1-FirstNameLastName page you just created:
    • Create a new file link like this: [[File:firstname-lastname-hw2.zip]]
    • Save the page, then click on the File link you just created to upload the zip file.

Create & Upload Video

  • What your screencast should contain:
    • Narrated walkthrough of the interface including all implemented interactions.
    • Be CONCISE. Your video shouldn't be longer than 90 seconds.
    • Be prepared to do multiple takes; plan and/or write out a script first.
  • Your file should be in AVI, WMV, OGV, or MOV format
    • Rename the wmv/mov file to firstname-lastname-hw2.mov (e.g., bjoern-hartmann-hw2.mov)
  • Upload the mov file to the Homework2-FirstNameLastName page you just created:
    • Create a new file link like this: [[File:firstname-lastname-hw2.mov]]
    • Save the page, then click on the File link you just created to upload the mov file.
  • Alternately, you may instead elect to upload the video to a video sharing service and include the link in your submission. On the upside, a video posted to Youtube may have greater visibility, but on the downside, the class record will no longer have an archival copy.

Describe your implementation

  • On the Homework2-FirstNameLastName page you just created, write one or two paragraphs about your implementation:
    • what platform, language, and tools you used
    • any additional features you implemented that we should consider for extra credit
    • what you learned from this assignment

Add Link to Your Finished Assignment

One you are finished editing the page, add a link to it at the bottom of the page with your full name as the link text. The wiki syntax will look like this: *[[Homework2-FirstNameLastName|FirstName LastName]]. Hit the edit button for the last section to see how I created the link for my name.

Links to Finished Assignments

Add your submission below this line.