Lab:Real-Time Audio Classification
Re-build acoustic bar codes with the equipment we gave you (contact microphone, mic preamp). You should be able to distinguish five different codes in real-time. For example, you can match them to an audio player: pause/play, volume up, volume down, next track, previous track. However, it's enough to show that you can correctly distinguish between the tags.
Fabricate the barcodes yourself - we suggest using acrylic on the laser cutter, as the contact mic will stick well to the smooth surface. Experiment with scoring (blue) versus rastering (other colors). Harrison's paper has useful measurements to get you started.
- I used PureData (also known as Pd), a real-time audio programming environment. You want the pd-extended distribution which comes with lots of useful extras.
- To map audio events to mouse/keyboard events, I used OpenSoundControl to send events between applications and OSCulator to generate the mouse/kbd events (free trial available).
- If you want to program in C++, RtAudio is a nice small cross-platform library.
- To take a look at your signals, Audacity is a free open source audio editor.
- Trigger an event (e.g., press a key) whenenver audio input exceeds a threshold (e.g., when you clap): File:01-clap.pd
- Map whistling frequency to mouse motion: by thresholding File:02-whistle.pd, or by linear interpolation File:02-whistle-continuous.pd (you also need File:Lerp.pd in the same directory)
- Output times between RMS peaks File:Time-taps.pd (started, not completed)
Pd and Python
Storing items in memory in Pd can be a hassle. There are various options for interfacing Pd with standard, procedural programming languages. py/pyext, a Python extension, has some momentum behind it but can be tricky to install. Here are some notes:
- Download PD-extended 0.42.5: Since there seems to be a problem with the Preferences menu in the latest Pd-extended 0.43, I downloaded 0.42.5.extended from http://puredata.info/downloads/pd-extended/releases
- Check out source for py (needed for script directory and examples): svn co https://svn.grrrr.org/ext/trunk/py/ pd-py
- Download the binary http://grrrr.org/data/dev/ext/macos/pd/py.pd_darwin-i386-x86_64-system-2.7.zip from http://grrrr.org/data/dev/ext/macos/pd/ and unzip
- Move py.pd_darwin from 2) into the script directory /pd-py/scripts from 1)
- Launch PD. Pd-extended->Preferences->Startup...->Add "py" to the list.
- Pd-extended->Preferences->Path...->Add the path to /pd-py/scripts from 1) to the list.
- Restart Pd. If successful, you'll see something like:
------------------------------------------------ py/pyext 0.2.2 - python script objects (C)2002-2011 Thomas Grill - http://grrrr.org/ext using Python 2.7.1 (r271:86832, Aug 5 2011, 03:30:24) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] Python array support enabled ------------------------------------------------
- To test, open /pd-py/pd/script-1.pd, which calls functions in /pd-py/scripts/script.py
You'll also need numpy for Python. I already had it through MacPorts. If someone can provide a concise explanation how to install it for the Mac OS X system Python interpreter, that'd be great.