by Yuan Ning (yn47) and Adam Beece (apb36)
Why a working, wearable air guitar? Bottom line, it's just that cool. Who doesn't want a machine that will take their jamming and turn it into rock they can hear? With just a glove, a pick and a little practice you can rock the synthetic acoustic guitar like no one else in the world.
High Level Design
While perhaps not the first people to have the idea of a air guitar to actually play the instrument, we were inspired more by the hardware available than any other incarnation of a working air guitar. It looks something like this:
There are three basic functional components of Air Jam: the glove, the accelerometer and the software, which implements the Karplus-Strong algorythim. Karplus-Strong uses a burst of white noise (generated in this case by the repeated use of the stdlib rand() function) that is repeatedly filtered and played back. The frequency of the delay controls the frequency of the plucked string with a simple formula:Delay Length L = Fsample/Fout
Nothing fancy about it.
Let us take a moment to consider whose toes Air Jam steps on. The Australian research company CSIRO have created a Wearable Instrument Shirt (WIS) that tracks arm position and uses it to calculated notes actually fretted. These i-Textiles are significantly more sophisticated technology than the simple flex sensors used in Air Jam's glove, not to mention the fact that CSIRO synthesizes an electric guitar, not an acoustic one.
Hardware Design
Speaking of flex sensors, how does Air Jam work? Each finger of the glove has sewn directly to it a flex sensor that is in fact a variable resistor. Wires run from the glove to our circuit board, where in sits the rest of the circuit
a voltage divider and an analog comparator implemented by an op-amp with the inverting input tied to a 3.33 V line. The voltage divider between the flex-variable resistance and R would bring the non-inverting line above 3.33 V when a digit is extended, and below that value when it is flexed. During such a flex, the sensors resistance grows from ~71 kΩ to ~310 kΩ, and via experimentation we determined the value of R that gave the best triggering from 0 V to 5 V was 400 kΩ for the pinky, ring, middle and index fingers, and 300 kΩ for the thumb, because it flexed it's sensor less than the other digits. The op-amp is a Linear Technology 1490, chose primarily because it was rail to rail.
The accelerometer is one of the ones generously donated by Freescale, whose output is an analoug value from 0 V to it's 3 V supply (achieved using a voltage regulator). This output was hooked up to the Mega32's A to D converter through a simple protective circuit (diodes to prevent any voltage less than 0 or getter than Vcc from being put on the ADC input pin). We tried to use a Kionex digital accelerometer, only to find the chips unable to function after they were exposed to multiple mis-wirings, possibly damaging the chips. Thank you, Freescale, for saving Air Jam.
The output of our circuit is a 3.5 mm stereo audio socket, an industry standard, an so you can use Air Jam with almost any set of speakers. We wanted to be able to use headphones, but the total output voltage was unfortunately too low so that without some sort of amplifier (meaning a volume adjust nob), the beautifully synthesized sound was inaudible.
Program design
How does Air Jam make that beautiful acoustic sound? Karplus-Strong, of course.
Math
In the Karplus-Strong algorithm, the pluck, which in real string can contain energy at any frequency, is simulate by filling the delay-line with noise at each note beginning. In a real string, vibration eliminates those frequencies introduce by the pluck that don't match the normal modes of the string. Friction and losses at end point will create the sound decay, with higher frequencies decaying faster than lower ones. At the end, the wave shape will almost be sinusoidal with a period corresponding to the fundamental mode of the string. The sound will finally decay to silence. In the Karplus-Strong model this can be achieve by the introduction of an averager in the loop
Math
Alex Strong device filters by simply adding the two last outputs and right shifting the result (divide by 2). The so create sample is then re-introduce in the loop. This and the length of the loop will cause self-cancellation of the "non-harmonic" partials. This will also cause higher frequencies to decay more and more at each delay-line trip, until only the fundamental remain. Like in the case of a real string, the delay-line content will finally decay to a constant value, which sounds like silence.
The period of the resulting signal is the period of the delay line plus the average group delay of the filter; the fundamental frequency, as usual, is the reciprocal of the period. The required delay D for a given fundamental frequency F1 is therefore calculated according to D = Fs / F1 where Fs is the sampling frequency.
One of the major problems of this model is that the delay-line length needs to be of an integer size. This limitation has for effect the quantization of the frequencies that can be precisely reproduced. As figure 6 illustrates, the effect is small for low frequencies (long delay-line) but can be major for higher frequencies where the difference between sr / (d + 1/2) and sr / (d + 1 + 1/2) is huge.
Since for our application the desired frequency was realatively low, this wasn’t a significant problem.
Implementation of the algorithm:
To make the algorithm extremely efficient we simply used 6 vectors of different length each representing the maximum length of the delay line for each string. To avoid shifting, we update two pointers for each vector to keep track of the elements of interest.
To avoid multiplies, we used an extremely simple low pass filter which is simply an averager that averages adjacent elements within the delay line.
We calculated the delay length which corresponds to desired frequencies of each note playable on an acoustic guitar and stored this information in 6 length 12 vectors which correspond to 6 strings and 12 frets (we ignored the higher frets for conciseness). To play a note or make a chord we simply looked up delay lengths within these vectors. This makes encoding the chords extremely simple and intuitive as most guitar tabs available online are in terms of strings and frets.
Our guitar is essentially a chord player. With 5 fingers of our flex glove we can have up to 32 different encodings of different chords we can play. For our demo, we encoded 24 major and minor chords built from each of the 12 notes of a chromatic scale within an octave. To make things simple we encoded the 7 chords of ‘Hotel California’ by the eagles for the first 7 chords. Following is a table of chord encodings for the flex glove. The chord numbers are binary. This means that the binary number you make with you hand correspond to the number of the chord you are playing. A bent finger correspond to a ‘1’ and the pinky is the LSB.
Chords encoding of flex glove:
Chord # Finger Position Thumb Index Middle Ring Pinky Chord Name 1 00001 B minor 2 00010 F sharp major 3 00011 A major 4 00100 E major 5 00101 G major 6 00110 D major 7 00111 E minor 8 01000 C major 9 01001 C minor 10 01010 C sharp major 11 01011 C sharp minor 12 01100 D major 13 01101 D minor 14 01110 D# major 15 01111 D# minor 16 10000 E major 17 10001 E minor 18 10010 F Major 19 10011 F minor 20 10100 F# major 21 10101 F# minor 22 10110 G major 23 10111 G minor 24 11000 G# major 25 11001 G# minor 26 11010 A major 27 11011 A minor 28 11100 A# major 29 11101 A# minor 30 11110 B major 31 11111 B minor 0 00000 Open Strings It was actually a non-trivial problem to sample the input devices in such a way as to have Air Jam be sufficiently responsive. The gloves are sampled at 100 Hz, being roughly twenty times as fast as a human can strum a guitar. On the other hand the accelerometer needed to be sampled with frequency 8 kHz (the music sampling rate) in order to catch the very brief output spikes caused by the rapid acceleration and deceleration of the pick. After an above threshold voltage was found, the MCU saved that value and stopped polling the ADC for 100 ms to give the control code a chance to catch this trigger and process it. We tried to, based on how hard the accelerometer was strummed change the delay between successive string plucks, but it turned out that this method of polling the accelerometer output caused us to miss triggers either because we stopped polling as soon as we got any sort of trigger, or because the 8 kHz sampling frequency was such that smaller triggers were sometimes missed.
Results
Air Jam delivers. The delay between input and result is imperceptible, allowing you to jam in real time. There can sometimes be a flicker if you move the pick to fast corresponding to multiple triggers, but as long as you play at a reasonable speed, it's not noticeable. Hold the pick upside down, on the other hand, and gravitational interference causes the accelerometer to simply constantly trigger resulting in a high frequency sound.
It's impressive how much Air Jam sounds like a real guitar, but exactly how close does Air Jam get to the perfect frequency when used properly? Well, he's the gruesome details:
Strings & frets Theoretical Frequency Exact synthesized frequency Open Low E 82.41 Hz 83.111 Hz Low E 1 87.31 Hz 87.781 Hz Low E 2 92.50 Hz 93.001 Hz Low E 3 98.00 Hz 98.892 Hz Low E 4 103.83 Hz 104.167 Hz Low E 5 & Open A 110.00 Hz 110.035 Hz Low E 6 & A 1 116.54 Hz 116.604 Hz Low E 7 & A 2 123.47 Hz 124.008 Hz Low E 8 & A 3 130.82 Hz 132.415 Hz Low E 9 & A 4 138.59 Hz 139.509 Hz Low E 10 & A 5 & Open D 146.83 Hz 147.405 Hz Low E 11 & A 6 & D 1 155.56 Hz 156.250 Hz Low E 12 & A 7 & D 2 164.81 Hz 166.223 Hz A 8 & D 3 174.61 Hz 177.557 Hz A 9 & D 4 185.00 Hz 186.012 Hz A 10 & D 5 & Open G 196.00 Hz 200.321 Hz A 11 & D 6 & G 1 207.65 Hz 211.149 Hz A 12 & D 7 & G 2 220.00 Hz 223.214 Hz D 8 & G 3 233.88 Hz 236.742 Hz D 9 & G 4 & Open B 246.94 Hz 252.016 Hz D 10 & G 5 & B 1 261.63 Hz 269.340 Hz D 11 & G 6 & B 2 277.18 Hz 279.018 Hz D 12 & G 7 & B 3 293.66 Hz 300.481 Hz G 8 & B 4 311.13 Hz 312.500 Hz G 9 & B 5 & Open High E 329.63 Hz 339.674 Hz G 10 & B 6 & High E 1 349.23 Hz 355.114 Hz G 11 & B 7 & High E 2 369.99 Hz 372.024 Hz G 12 & B 8 & High E 3 392.00 Hz 411.194 Hz B 9 & High E 4 415.30 Hz 434.028 Hz B 10 & High E 5 440.00 Hz 459.559 Hz B 11 & High E 6 466.16 Hz 488.281 Hz B 12 & High E 7 493.88 Hz 520.833 Hz High E 8 523.25 Hz 558.036 Hz High E 9 554.36 Hz 558.036 Hz High E 10 587.32 Hz 600.962 Hz High E 11 622.26 Hz 651.042 Hz High E 12 659.26 Hz 710.227 Hz As you can see from the gruesome detail chart, our accuracy suffers more the higher the frequency gets, however, since this was known we chose the use of the chords that had the lowest possible frequencies, and thus never use any freting higher than the third or fourth fret on High E, where error is still no worse than 5% of the theoretical “perfect” frequency. However these slight inaccuracies make the guitar sound in some ways more real, as no real guitar is everperfectly in tune.
One nice fact about Air Jam is that it is totally safe. It has no moving parts, small parts, high voltage lines, or anything that would pose more of a threat than your average unplugged Playstation controller (people can choke on anything if they try hard enough). The only noise it puts out comes from the speakers, so as long as it's working it only provides more harmonic background noise than people discussing their project.
Who can use Air Jam? Well it requires an adult-sized left hand and the ability to move the pick. In short if you can play air guitar, you can play Air Jam. Like any other musical instrument it would take some practice for the fingering to become instinctive enough to play songs (especially fast passed ones), but regardless of encoding that would be the case. You can cheat by seting up the first group of chords to be the corresponding numbered chords in a particular song (so that all you need to do is count in binary using your pinky as the lsb to play successive chords). In fact it is currently set up to play Hotel California this way. The problem with this method is it involves reprogramming the fingering for each song, which would require additional interfacing (everything you can do with the glove & accelerometer corresponds to some chord).
Note on Intellectual Property:
We used an efficient implementation of the Karplus Strong Algorithm to generate guitar sounds. The algorithm was developed in the early 80’s, and is patented by Stanford University. The technique was granted US patent number 4,649,783 and also
Australian patent 570,669 and Canadian patent 1,215,869. We researched many implementations of the algorithm and decided to use Alex Strong’s original implementation which was developed for slower processors and was extremely computationally efficient. We mainly used the following online sources to understand the way the algorithm works and how Alex Strong implemented the algorithm originally.http://www.music.mcgill.ca/~amburns/physique/introduction.html
http://ccrma.stanford.edu/~jos/pasp/Karplus_Strong_Algorithm.htmlConclusions
Altough it has been said before, it merits being said again; Air Jam rocks! We, and others hearing us testing our project, were amazed at how life-like Air Jam sounds. Despite this there were a number of things that could have been done to make it better:
- Use a custom PC board . If we had used such a board & a battery we could have sewn together a belt to hold Air Jam's circuit and mcu, allowing for more mobility.
- Longer Wires. The lengths of the wires connecting Air Jam's glove and pick to its circuits are so short, it requires the user to sit or stand right next to the table with the circuitry, thus limiting mobility.
- Use a more powerful accelerometer. Although Freescales generous donation of accelerometers enabled Air Jam to work, if we could have had an accelerometer not rail at 2 gs, we could have used this data to spread out our thresholds and make variable the delay between successive string plucks. We may have also been able to detect an up strum vs. a down strum and changed the order & amplitude of the strings plucked accordingly.
- Make the chords played by Air Jam user programmable, allowing each user to put the chords they use most often into the position most comfortable for them.
- Include a second glove to allow for fast single string plucking required in some songs.
Budget
Total cost of this project: $40.75
Part Cost STK 500 development board $15.00 Large solder board $2.50 Power Supply $5.00 Dip Sockets x2 $1.00 LT op-amp 8-dip x3 $13.25 Flex Sensors x 5 Surpluss Accelerometer & Voltage Regulator (desoldered from battery board) Donated by Freescale Glove $1 Radio Shack Audio Jack $3.00 Diodes 1N914 x2 Surpluss TOTAL $40.75 Thank you Freescale for your generous donation to this course.
Ethics
We acted in accord with the IEEE code of ethics. Since we were asked for a specific example, three of our colleagues are working on a music synthesizer capable of creating music from several different instruments. Impressed by Air Jam's sound, they asked how we made it, and we told them about Karplus-Strong, and even told them about our super-fast filter. Believe it or not, the person in charge of all soldering for Air Jam had never soldered before in his life, however we decided that since he was designing the hardware, and since he is hoping to graduate Cornell University with an Electrical and Computer Engineering degree, it would be to his benefit to actually get some practice. He is much more confident now in his aptitude at this essential Electrical Engineering skill. If that doesn't count as “Improving our technical competence,” it is hard to imagine what would. Following IEEE's code of ethics was not hard, but in the interest of giving credit where credit is due, Air Jam would not exist if not for the continuous and self-sacrificing help of Dr. Bruce Land and his wonderful group of TAs.
Thank you all very much.
LINKS:
CSIRO, ECE 476, US Air Guitar Championships, IEEE Ethics
APPENDIX:
