Basic Info

For our senior-year design project, we built a basic synthesizer workstation using an Mega644 microcontroller on an STK-500 board, using frequency modulation and additive synthesis to play/record 4 different instruments on to 4 separate tracks.

Matt Ferrari mjf259

Alec Hollingsworth arh68

We decided to build this project after tossing around some similar ideas with voice synthesis, beat detection, pitch correction functionalities. We hesitated with these because we had almost no familiarity with the subject matter. The synthesizer workstation seemed to be a good combination of technologies we had learned in the class (keypad input, LCD display, direct digital synthesis by Pulse Width Modulation). We outlined the project in multiple levels of functionality: the minimum viable solution, extras that would be nice, and challenging ideas. Hardware limitations (limited memory, for one) and time constraints would leave us with these functionalities (from a user perspective):

1. 1.Play notes in real-time from 4 different instruments
   

2. 2.Record those notes as they are played
   

3. 3.Play back a single track of recorded notes as any instrument
   

4. 4.Play back all 4 tracks of recorded notes as any mix of instruments
   

From an implementation perspective, we needed to accomplish:

1. 1.Instrument synthesis through a PWM output running at 62.5kHz
   

2. 2.Reading commands from a keypad
   

3. 3.Reading keyboard input from a keyboard/keypad
   

4. 4.Storing notes for each track
   

5. 5.Printing to an LCD
   

6. 6.Managing the state machine that would control the keyboard
   

7. 7.Reading input/calculating output on time
   

Now let’s get into the technological backgrounds for each:

1. 1.Instrument synthesis through a PWM output running at 62.5kHz
   

A Pulse Width Modulated signal uses a rectangular wave and a low-pass filter to generate sine waves. The idea is that while a regular square wave would be filtered by the low-pass filter, a rectangular wave with varying duty cycle (seen in pink on the left) would cause slow transitions in the average of the output (seen in green on the left). Since a slow transition would be passed by the low-pass filter, we can produce a sine wave with a digital signal. For our keyboard, we use a low-pass filter with a corner frequency on the order of 100Hz to filter the PWM frequency and to create a more pleasing sound.

The frequency synthesis is done through a table of PWM samples for one cycle of a sine wave (or frequency modulated sine wave, or an addition of multiple sine waves at different harmonics). The frequency of the note heard is changed by how fast we run through the PWM samples.

2. 2.Reading commands from a keypad
   

We used a 16-key 8-pin keypad (seen on left) to read commands. Each button on the keypad would issue one command. The keypad is wired as a matrix, so we can reliably read one key at a time by determining which column and row are connected.

3. 3.Reading keyboard input from a keyboard
   

We wired the keyboard as 23 independent switches. The STK-500 would not have that many free pins, however, so we used priority encoders to encode the key signals. We split the keys into 3 sections of 8 and used 8-3 encoders to get 4 outputs for each section (12 outputs total). We combined these signals through basic digital logic to get 6 outputs that we could hook up to the STK-500.

4. 4.Storing notes for each track
   

We used a simple C array to store the key numbers we captured during recording. Note that this is an array and not a matrix; due to the limited number of connections on the STK-500, we cannot capture every piano key at the same time, so we only play/record one note at a time. We capture ~10 notes/sec and record up to ~13 seconds, resulting in 128-length arrays for each track.

5. 5.Printing to an LCD
   

We used a 16x2 LCD to display messages to the user. Conveniently, we were able to use a library (lcd_lib.h) that could print strings to the LCD. So our output consisted of updating our global output strings when necessary, and regularly writing those strings out to the LCD.

6. 6.Managing the state machine that would control the keyboard
   

Our keyboard is controlled by a state machine that keeps track of what is going on at any given time. Most transitions between states are triggered by commands. We wrote a function handle_keypress(..) to examine what key was pressed and what state the keyboard is in and then transition to a new state and run any necessary code (for example, when a user presses a key to Delete all tracks, we have to run code that will erase the notes data).

7. 7.Reading input/calculating output on time
   

Everything we needed this keyboard to do was time-sensitive. To simplify our lives, we used the TinyRealTime kernel. Everything except the PWM calculation and output is done by tasks that run periodically. For example, during recording, we capture notes at ~10 notes/sec. This is done by running a task that executes ~100ms. The code to schedule this task can be seen on the left.

Since TinyRealTime uses the Mega644’s timer1, we use timer0 for the PWM calculation and timer2 for the PWM output. The PWM output runs at 62.5kHz, but the calculations take longer, so we run those with timer0 in an interrupt that runs every 8kHz. This combines the benefits of a high PWM frequency (don’t have to worry about filtering an audible frequency) and slower calculation frequency (takes less CPU time by running less frequently).

Overall, our keyboard is a decent attempt at a basic synthesizer. Although the notes that can be played with the keyboard synthesizer are relatively good representations of the actual instrument, the differences between the two are audibly evident; it helps to be imaginative when using our keyboard. The keyboard creates sounds that are reminiscent of the instrument but aren’t quite perfect. We knew that we wouldn’t be able to get the sounds to exactly match the sounds that the actual instruments could produce, but the sounds we were able to produce exceeded our expectations. The two sounds that are closest to their instrument equivalent are the xylophone and the brass instrument, while the guitar sound is less believable. The robotic sound has nothing to compare to, so that could be considered authentic.

Now let’s get into specific user interaction performance. These analyses are subjective. The keyboard seems quite responsive, with the LCD updating nearly immediately. Due to our method of updating the display, sometimes the display will temporarily settle between two messages, displaying half of each. This could have been avoided with a semaphore protecting the output buffers, but it’s rather minor. According to the PWM math, the frequencies we output are accurate to ~.2Hz, since the lowest frequency we could output is .12Hz. When measured using the oscilloscope, the frequencies come to within 10 Hz of the desired frequencies (we can’t measure much better with the scope in lab). Additionally, the frequency we want to play has always resulted in the correct frequency being played. Chords sound subjectively good, even octaves.

Our project didn’t involve much anything dangerous. Nonetheless, our design emphasized safety in two specific areas: soldering and sanding. While using the soldering iron, we made sure we were wearing our safety goggles and worked together whenever soldering to ensure safety. We also made sure that the soldering iron was turned off after every use. While sanding, we made sure we turned on the fan to prevent dust particles from filling the air, and we also wore safety goggles and worked as a pair. In an effort to be as safe as possible with the STK-500 board, we made sure that the board wasn’t near flammable material in the case that the board overheated.

Our keyboard needed to be tested aloud using speakers, which could interfere with other groups who are designing projects that attempt to use sounds in some processing manner (or who would be annoyed by random noises). We greatly reduced the risk of interference by keeping our speakers on as low as we could. This was also done to protect the speakers from nefarious code.

The user interface we designed for the project consists of a LCD that tells the user exactly what every press on the keypad. Any English-literate user should be able to operate the keyboard and keypad with relative ease, even without supervision. The key to this is the help mode, which is activated by pressing button 9 and gives a basic description of what every button on the keypad. A message is placed on the LCD whenever the STK-500 is turned on that prompts the user to press the help key. Due to this feature, it is possible for any English-literate user to operate the keyboard system with no external guidance.

Software Details

In this section we will give detailed explanations for how certain software objectives were accomplished:

1. Reading the keypad

2. Transitioning between various states

1. Reading the keypad

The reading of the keypad button press was similar to the code used in lab 2 of ECE4760 in which the row are driven to zero, the columns are read, the rows and columns are reset, the columns are then driven to zero, and the rows read. From here, the unique eight-bit code yielded from the previous step is compared to a key code table to determine which button push it responds to, ranging from button zero to fifteen.

2. Transitioning between various states

The keypad presses were governed by a task that called a state machine. The state machine would have its only input be the key pressed. On a key press, the state of the state machine would be updated, the message of what was happening would print to the LCD, and a function would be called. The states that the state machine could be in include Normal_State, Help_State, Recording_State, and Playback_State. Normal_State was assumed when the state machine is initialized and will take in any key and act accordingly. Help_State is activated when key 10 is pressed (corresponds to button 9 on the keypad) while the state machine is in Normal_State and stays in Help_State until key 10 is pressed for a second time. While in Help_State, pressing any key will give the user information via a message on the LCD about that key. Recording_State is entered whenever the record button is pressed (the asterisk on the keypad) and remains in the Recording_State until exited via button 8 (key 9). While in Recording_State, the notes played using the keyboard are saved in an array for later playback.

Hardware Details

Here we will explain how we solved the hardware problems we found:

1. 1.Getting keyboard signals onto the STK-500
   

2. 2.Getting signals out of the keyboard itself
   

3. 3.Encoding the signals between the keyboard and the STK-500
   

4. 4.Reading the command keypad
   

5. 5.Printing messages on the LCD
   

6. 6.Using separate sound envelopes for the four instruments
   

7. 7.Low-passing the output
   

1. Getting keyboard signals onto the STK-500

One of the main problems we ran into involved utilizing the limited number of input ports of the STK-500. The keyboard we wanted to design came with 23 keys, which meant we would need a coded input to those ports with a unique signature for every port input. The way we sought about accomplishing this involved using encoders for our key presses. However, 23 unique keys required three encoders and nine inputs to the STK-500 ports, which was too many due to our use of an LCD and keypad that helps the user interact with the keyboard. In an effort to limit the number of inputs we used, we created a Boolean circuit using the output of our encoders to create a unique code for every key press on the keyboard.

2. Getting signals out of the keyboard itself

We annihilated the interconnects of the circuit of the board we bought and instead used each key as a one-hot switch. We did this by connecting the bottoms of all of the switches to a ground and soldering wires to the top of every switch. By connecting the power source to a 1k resistor and connecting that to the switch, we are able to get an active low switch. When we combined this with the encoders to create a twelve-bit code that could be fed into the STK-500. This code acted as a key code which key on the keyboard was pressed and allows for the keys to be decoded.

3. Encoding the signals between the keyboard and the STK-500

The keyboard we used gave us only a simple switch in the form of a short, so we needed to take this value and have it fed from VCC and through a resistor (which we chose to be 1k). This allowed us to make each press of the keyboard act as a switch, and we took this switch and fed the wire to the encoder input. 

Each encoder took 8 inputs out produced 4 outputs, so a total of 12 wires were fed into the STK-500. On the STK-500, we use a simple decoding process to determine which key was pressed.

Something we tried that did not work was further encoding the outputs through a NAND to reduce the number of outputs we would have to wire into the STK-500. The three encoders each produced their own A0,A1,A2 outputs. However, if only one key is pressed at a time, these can be combined since they would stay high while the more important octave that gets a keypress pulls the output low. So we put these into 3 3-input NANDS. Unfortunately, the NAND chips we had in lab were of poor quality: the pins were corroded and the readings seemed unreliable. Fortunately, we realized we had enough inputs on the STK-500 for 12 signals, so we simply put all encoder outputs onto the board.

4. Reading the command keypad

The hardware design for the keypad was based off of the hardware design from lab 2 of ECE4760. The keypad is hooked up to the STK-500 through an input/output port so the software can switch the rows and columns of the matrix between inputs and outputs.

5. Printing messages on the LCD

The hardware design for the LCD was based off of the hardware design from lab 1 of ECE4760. The pins are simply connected to an output port on the STK-500. The libraries we used were sufficient for our uses.

6. 6.Using separate sound envelopes for the four instruments
   

We unfortunately did not get to use different sound envelopes for the different instruments. Our envelopes were precomputed and stored in tables for fast access during the PWM calculation. The limited memory on the Mega644 forced us to back down to a single envelope that would control all four instruments.

7. 7.Low-passing the output
   

The output of the STK-500 is going to have harsh harmonics due to ramping. In order to fix this, we make use of a low pass filter with a corner frequency around 100 Hz that will account for these harsh harmonics and help smooth the sound out.

The resemblance of the digitally created sound waves to the actual sound waves far exceeded our expectations. The sound waves were created using a variety of methods, including direct digital synthesis, frequency modulation, and additive synthesis. If we had another chance to do this project, one thing we might change would be to choose all of the instruments to be ones that can be synthesized by a single method rather than by multiple types of synthesis.

Another thing that was time consuming yet yielded no results was our attempt to decode the Schylling keyboard’s circuit board. The circuit didn’t seem to adhere to any specific rules regarding the decoding of a pressed key. On any pressed key, one bus of the wire would go high, indicating that a key in a specific octet was pressed. The only other wires on the board, however, were three wires that traveled to the microcontroller and would all drop to the same voltage at the same time. After spending hours trying to determine what each of the three wires did, we were unable to confirm anything except that these wires were identical to the best of our knowledge. We decided that it wouldn’t be fruitful to continue in this endeavor and instead acted to obliterate the wires connecting all of the switches to one another, making every button act as a simple switch. Had we decided to do this at the beginning of the project, we may have finished a full week earlier.

Any code that we used on this project that we did not create ourselves came from Bruce Land. This includes code from lab 2 in which we used the section that scans the key pad and gets a key code for the button pressed, as well as the code that makes the LCD work and print properly. Additionally, we heavily relied upon and used parts from the tiny real time page and the DSP page that are linked to the ECE4760 homepage.

We publicly credit Bruce Land for the code and help we received from him, as well as the companies who created the data sheet that we were able to download. Those include Texas Instruments for the 8 to 3 SN74HC148 encoders. There exist no conflicts of interest between our group and any company of any product used in this project. Neither of us is employed nor will be employed by said companies or competitors of said companies. We acknowledge that our synthesizer is not nearly up to the same standard as professionally made keyboard synthesizers, in both audio quality and musical selections and scales. Throughout the weeks of designing this project, we were always overly cautious when it came to safety and we always acted to keep the welfare and safety of the rest of our lab participants. Whenever we worked with the soldering iron, we checked to make sure the soldering station was cleared of any potential fire hazards and that water was available on hand in case of any unforeseen accidents. We also made sure that our soldering was completed as close to the fan as possible to ensure that the potentially hazardous smoke emitted from the burning solder was quickly dispersed. Additionally, we purchased a Schylling keyboard for the sole purpose of academic enrichment and have no intent to resell this product or create any type of similar product for monetary gain.

The 8-3 encoder used to reduce the inputs

Encoder datasheet and vendor:

http://focus.ti.com/lit/ds/symlink/sn74hc148.pdf

http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=296-8233-5-ND

TinyRealTime kernel information:

http://people.ece.cornell.edu/land/courses/ece4760/TinyRealTime/index.html

Frequency modulation and additive synthesis information:

http://people.ece.cornell.edu/land/courses/ece4760/Math/avrDSP.htm

16x2 LCD information:

http://people.ece.cornell.edu/land/courses/ece4760/labs/s2011/lab1.html

16-key 8-pin keypad information:

http://people.ece.cornell.edu/land/courses/ece4760/labs/s2011/lab2.html

PWM diagram by Cyril Buttay:

http://en.wikipedia.org/wiki/File:Pwm.png

All other pictures were taken by us of hardware we used in this project, except for those bottom two, which were taken by Bruce Land.

The encoder (74LS148) we used in our circuit. So in the encoder for the bottom 8 keys, we attach the switches to inputs 0-7, then read A0,A1,A2,EO.

We accomplished most tasks together, but we played to our strengths and did the tasks alone that would have taken longer together. Matt did much of the soldering and sanding, since he was much better at it than Alec (although Alec assisted in preparing and holding wires). On the other hand, Alec wrote the core state machine code (including those crucial macros) and the core synthesis/envelope code. Matt spent the time figuring out what instruments would sound best with our synthesis techniques and wrote the code that initialized the wavetables. Matt designed the encoding circuit that connected the keyboard to the STK-500 through encoders. Alec wrote the code that kept the tasks running on time, and ensured we did not use too much memory (which causes the Mega644 to hang, or restart, depending on the weather). For the most part, the hardware was tackled by Matt while the software was done by Alec.

A digital signal (pink) can create a sine wave (blue)

A 16-key 8-pin keypad

A 16x2 LCD

Our encoder circuit, showing the 8-3 encoders.

Our low-pass filter, showing the PWM input (red), resistor, capacitor, and speaker output (yellow).

The rectangular PWM waveform running at 62.5kHz.

An example of a state transition in our code. When key 10 is pressed while in the Normal Mode, we print “Help Mode on” to the LCD, transition to the Help Mode, and do nothing further.

Code example showing scheduling of a task that will run between .1 and .125 seconds in the future.

The sine wave (613.5Hz) from the filtered PWM.

The printed circuit board we used from the piano. We destroyed the existing key matrix since we couldn’t reliably decode it, and soldered our own wires to connect to the switches.

It turns out it was the red wire.

Saving the world one circuit at a time.