AIR JAM!!!

Wearable Air Guitar

by Yuan Ning (yn47) and Adam Beece (apb36)

Introduction

Why a working, wearable air guitar? Bottom line, it's just that cool. Who doesn't want a machine that will take their jamming and turn it into rock they can hear? With just a glove, a pick and a little practice you can rock the synthetic acoustic guitar like no one else in the world.

High Level Design

While perhaps not the first people to have the idea of a air guitar to actually play the instrument, we were inspired more by the hardware available than any other incarnation of a working air guitar. It looks something like this:

There are three basic functional components of Air Jam: the glove, the accelerometer and the software, which implements the Karplus-Strong algorythim. Karplus-Strong uses a burst of white noise (generated in this case by the repeated use of the stdlib rand() function) that is repeatedly filtered and played back. The frequency of the delay controls the frequency of the plucked string with a simple formula:

Delay Length L = Fsample/Fout

Let us take a moment to consider whose toes Air Jam steps on. The Australian research company CSIRO have created a Wearable Instrument Shirt (WIS) that tracks arm position and uses it to calculated notes actually fretted. These i-Textiles are significantly more sophisticated technology than the simple flex sensors used in Air Jam's glove, not to mention the fact that CSIRO synthesizes an electric guitar, not an acoustic one.

Hardware Design

Speaking of flex sensors, how does Air Jam work? Each finger of the glove has sewn directly to it a flex sensor that is in fact a variable resistor. Wires run from the glove to our circuit board, where in sits the rest of the circuit

a voltage divider and an analog comparator implemented by an op-amp with the inverting input tied to a 3.33 V line. The voltage divider between the flex-variable resistance and R would bring the non-inverting line above 3.33 V when a digit is extended, and below that value when it is flexed. During such a flex, the sensors resistance grows from ~71 kΩ to ~310 kΩ, and via experimentation we determined the value of R that gave the best triggering from 0 V to 5 V was 400 kΩ for the pinky, ring, middle and index fingers, and 300 kΩ for the thumb, because it flexed it's sensor less than the other digits. The op-amp is a Linear Technology 1490, chose primarily because it was rail to rail.

The accelerometer is one of the ones generously donated by Freescale, whose output is an analoug value from 0 V to it's 3 V supply (achieved using a voltage regulator). This output was hooked up to the Mega32's A to D converter through a simple protective circuit (diodes to prevent any voltage less than 0 or getter than Vcc from being put on the ADC input pin). We tried to use a Kionex digital accelerometer, only to find the chips unable to function after they were exposed to multiple mis-wirings, possibly damaging the chips. Thank you, Freescale, for saving Air Jam.

The output of our circuit is a 3.5 mm stereo audio socket, an industry standard, an so you can use Air Jam with almost any set of speakers. We wanted to be able to use headphones, but the total output voltage was unfortunately too low so that without some sort of amplifier (meaning a volume adjust nob), the beautifully synthesized sound was inaudible.

Program design

How does Air Jam make that beautiful acoustic sound? Karplus-Strong, of course.

Math

In the Karplus-Strong algorithm, the pluck, which in real string can contain energy at any frequency, is simulate by filling the delay-line with noise at each note beginning. In a real string, vibration eliminates those frequencies introduce by the pluck that don't match the normal modes of the string. Friction and losses at end point will create the sound decay, with higher frequencies decaying faster than lower ones. At the end, the wave shape will almost be sinusoidal with a period corresponding to the fundamental mode of the string. The sound will finally decay to silence. In the Karplus-Strong model this can be achieve by the introduction of an averager in the loop

Math

Alex Strong device filters by simply adding the two last outputs and right shifting the result (divide by 2). The so create sample is then re-introduce in the loop. This and the length of the loop will cause self-cancellation of the "non-harmonic" partials. This will also cause higher frequencies to decay more and more at each delay-line trip, until only the fundamental remain. Like in the case of a real string, the delay-line content will finally decay to a constant value, which sounds like silence.

The period of the resulting signal is the period of the delay line plus the average group delay of the filter; the fundamental frequency, as usual, is the reciprocal of the period. The required delay D for a given fundamental frequency F1 is therefore calculated according to D = Fs / F1 where Fs is the sampling frequency.

One of the major problems of this model is that the delay-line length needs to be of an integer size. This limitation has for effect the quantization of the frequencies that can be precisely reproduced. As figure 6 illustrates, the effect is small for low frequencies (long delay-line) but can be major for higher frequencies where the difference between sr / (d + 1/2) and sr / (d + 1 + 1/2) is huge.

Since for our application the desired frequency was realatively low, this wasn’t a significant problem.

Implementation of the algorithm:

To make the algorithm extremely efficient we simply used 6 vectors of different length each representing the maximum length of the delay line for each string. To avoid shifting, we update two pointers for each vector to keep track of the elements of interest.

To avoid multiplies, we used an extremely simple low pass filter which is simply an averager that averages adjacent elements within the delay line.

We calculated the delay length which corresponds to desired frequencies of each note playable on an acoustic guitar and stored this information in 6 length 12 vectors which correspond to 6 strings and 12 frets (we ignored the higher frets for conciseness). To play a note or make a chord we simply looked up delay lengths within these vectors. This makes encoding the chords extremely simple and intuitive as most guitar tabs available online are in terms of strings and frets.

Our guitar is essentially a chord player. With 5 fingers of our flex glove we can have up to 32 different encodings of different chords we can play. For our demo, we encoded 24 major and minor chords built from each of the 12 notes of a chromatic scale within an octave. To make things simple we encoded the 7 chords of ‘Hotel California’ by the eagles for the first 7 chords. Following is a table of chord encodings for the flex glove. The chord numbers are binary. This means that the binary number you make with you hand correspond to the number of the chord you are playing. A bent finger correspond to a ‘1’ and the pinky is the LSB.

Chords encoding of flex glove:

Chord # Finger Position Thumb Index Middle Ring Pinky Chord Name
1 00001 B minor
2 00010 F sharp major
3 00011 A major
4 00100 E major
5 00101 G major
6 00110 D major
7 00111 E minor
8 01000 C major
9 01001 C minor
10 01010 C sharp major
11 01011 C sharp minor
12 01100 D major
13 01101 D minor
14 01110 D# major
15 01111 D# minor
16 10000 E major
17 10001 E minor
18 10010 F Major
19 10011 F minor
20 10100 F# major
21 10101 F# minor
22 10110 G major
23 10111 G minor
24 11000 G# major
25 11001 G# minor
26 11010 A major
27 11011 A minor
28 11100 A# major
29 11101 A# minor
30 11110 B major
31 11111 B minor
0 00000 Open Strings

It was actually a non-trivial problem to sample the input devices in such a way as to have Air Jam be sufficiently responsive. The gloves are sampled at 100 Hz, being roughly twenty times as fast as a human can strum a guitar. On the other hand the accelerometer needed to be sampled with frequency 8 kHz (the music sampling rate) in order to catch the very brief output spikes caused by the rapid acceleration and deceleration of the pick. After an above threshold voltage was found, the MCU saved that value and stopped polling the ADC for 100 ms to give the control code a chance to catch this trigger and process it. We tried to, based on how hard the accelerometer was strummed change the delay between successive string plucks, but it turned out that this method of polling the accelerometer output caused us to miss triggers either because we stopped polling as soon as we got any sort of trigger, or because the 8 kHz sampling frequency was such that smaller triggers were sometimes missed.

Results

Air Jam delivers. The delay between input and result is imperceptible, allowing you to jam in real time. There can sometimes be a flicker if you move the pick to fast corresponding to multiple triggers, but as long as you play at a reasonable speed, it's not noticeable. Hold the pick upside down, on the other hand, and gravitational interference causes the accelerometer to simply constantly trigger resulting in a high frequency sound.

It's impressive how much Air Jam sounds like a real guitar, but exactly how close does Air Jam get to the perfect frequency when used properly? Well, he's the gruesome details:

Strings & frets Theoretical Frequency Exact synthesized frequency
Open Low E 82.41 Hz 83.111 Hz
Low E 1 87.31 Hz 87.781 Hz
Low E 2 92.50 Hz 93.001 Hz
Low E 3 98.00 Hz 98.892 Hz
Low E 4 103.83 Hz 104.167 Hz
Low E 5 & Open A 110.00 Hz 110.035 Hz
Low E 6 & A 1 116.54 Hz 116.604 Hz
Low E 7 & A 2 123.47 Hz 124.008 Hz
Low E 8 & A 3 130.82 Hz 132.415 Hz
Low E 9 & A 4 138.59 Hz 139.509 Hz
Low E 10 & A 5 & Open D 146.83 Hz 147.405 Hz
Low E 11 & A 6 & D 1 155.56 Hz 156.250 Hz
Low E 12 & A 7 & D 2 164.81 Hz 166.223 Hz
A 8 & D 3 174.61 Hz 177.557 Hz
A 9 & D 4 185.00 Hz 186.012 Hz
A 10 & D 5 & Open G 196.00 Hz 200.321 Hz
A 11 & D 6 & G 1 207.65 Hz 211.149 Hz
A 12 & D 7 & G 2 220.00 Hz 223.214 Hz
D 8 & G 3 233.88 Hz 236.742 Hz
D 9 & G 4 & Open B 246.94 Hz 252.016 Hz
D 10 & G 5 & B 1 261.63 Hz 269.340 Hz
D 11 & G 6 & B 2 277.18 Hz 279.018 Hz
D 12 & G 7 & B 3 293.66 Hz 300.481 Hz
G 8 & B 4 311.13 Hz 312.500 Hz
G 9 & B 5 & Open High E 329.63 Hz 339.674 Hz
G 10 & B 6 & High E 1 349.23 Hz 355.114 Hz
G 11 & B 7 & High E 2 369.99 Hz 372.024 Hz
G 12 & B 8 & High E 3 392.00 Hz 411.194 Hz
B 9 & High E 4 415.30 Hz 434.028 Hz
B 10 & High E 5 440.00 Hz 459.559 Hz
B 11 & High E 6 466.16 Hz 488.281 Hz
B 12 & High E 7 493.88 Hz 520.833 Hz
High E 8 523.25 Hz 558.036 Hz
High E 9 554.36 Hz 558.036 Hz
High E 10 587.32 Hz 600.962 Hz
High E 11 622.26 Hz 651.042 Hz
High E 12 659.26 Hz 710.227 Hz

As you can see from the gruesome detail chart, our accuracy suffers more the higher the frequency gets, however, since this was known we chose the use of the chords that had the lowest possible frequencies, and thus never use any freting higher than the third or fourth fret on High E, where error is still no worse than 5% of the theoretical “perfect” frequency. However these slight inaccuracies make the guitar sound in some ways more real, as no real guitar is everperfectly in tune.

One nice fact about Air Jam is that it is totally safe. It has no moving parts, small parts, high voltage lines, or anything that would pose more of a threat than your average unplugged Playstation controller (people can choke on anything if they try hard enough). The only noise it puts out comes from the speakers, so as long as it's working it only provides more harmonic background noise than people discussing their project.

Who can use Air Jam? Well it requires an adult-sized left hand and the ability to move the pick. In short if you can play air guitar, you can play Air Jam. Like any other musical instrument it would take some practice for the fingering to become instinctive enough to play songs (especially fast passed ones), but regardless of encoding that would be the case. You can cheat by seting up the first group of chords to be the corresponding numbered chords in a particular song (so that all you need to do is count in binary using your pinky as the lsb to play successive chords). In fact it is currently set up to play Hotel California this way. The problem with this method is it involves reprogramming the fingering for each song, which would require additional interfacing (everything you can do with the glove & accelerometer corresponds to some chord).

Note on Intellectual Property:

We used an efficient implementation of the Karplus Strong Algorithm to generate guitar sounds. The algorithm was developed in the early 80’s, and is patented by Stanford University. The technique was granted US patent number 4,649,783 and also
Australian patent 570,669 and Canadian patent 1,215,869. We researched many implementations of the algorithm and decided to use Alex Strong’s original implementation which was developed for slower processors and was extremely computationally efficient. We mainly used the following online sources to understand the way the algorithm works and how Alex Strong implemented the algorithm originally.

Conclusions

Altough it has been said before, it merits being said again; Air Jam rocks! We, and others hearing us testing our project, were amazed at how life-like Air Jam sounds. Despite this there were a number of things that could have been done to make it better:

• Use a custom PC board . If we had used such a board & a battery we could have sewn together a belt to hold Air Jam's circuit and mcu, allowing for more mobility.
• Longer Wires. The lengths of the wires connecting Air Jam's glove and pick to its circuits are so short, it requires the user to sit or stand right next to the table with the circuitry, thus limiting mobility.
• Use a more powerful accelerometer. Although Freescales generous donation of accelerometers enabled Air Jam to work, if we could have had an accelerometer not rail at 2 gs, we could have used this data to spread out our thresholds and make variable the delay between successive string plucks. We may have also been able to detect an up strum vs. a down strum and changed the order & amplitude of the strings plucked accordingly.
• Make the chords played by Air Jam user programmable, allowing each user to put the chords they use most often into the position most comfortable for them.
• Include a second glove to allow for fast single string plucking required in some songs.

Budget

Total cost of this project: \$40.75

Part Cost
STK 500 development board \$15.00
Large solder board \$2.50
Power Supply \$5.00
Dip Sockets x2 \$1.00
LT op-amp 8-dip x3 \$13.25
Flex Sensors x 5 Surpluss
Accelerometer & Voltage Regulator (desoldered from battery board) Donated by Freescale
Glove \$1
Diodes 1N914 x2 Surpluss
TOTAL \$40.75

Thank you Freescale for your generous donation to this course.

Ethics

We acted in accord with the IEEE code of ethics. Since we were asked for a specific example, three of our colleagues are working on a music synthesizer capable of creating music from several different instruments. Impressed by Air Jam's sound, they asked how we made it, and we told them about Karplus-Strong, and even told them about our super-fast filter. Believe it or not, the person in charge of all soldering for Air Jam had never soldered before in his life, however we decided that since he was designing the hardware, and since he is hoping to graduate Cornell University with an Electrical and Computer Engineering degree, it would be to his benefit to actually get some practice. He is much more confident now in his aptitude at this essential Electrical Engineering skill. If that doesn't count as “Improving our technical competence,” it is hard to imagine what would. Following IEEE's code of ethics was not hard, but in the interest of giving credit where credit is due, Air Jam would not exist if not for the continuous and self-sacrificing help of Dr. Bruce Land and his wonderful group of TAs.

Thank you all very much.