Guitar Synthesis on FPGA
The goal of our project has been to create a guitar simulator which could be programmed to play any song quickly and easily. In order to do this, we built a system which allowed us to encode notes on a guitar into our C code so that we could then use a timer and actually recreate an entire song. Our motivation for this project was to be able to accurately synthesize an entire song’s guitar sound on the fly. As a part of building this, we were also able to implement a distortion effect which we were optionally able to play on the guitar.
The design we implemented was the Karplus-Strong algorithm detailed in the ECE 476 webpage. The basic premise of the algorithm is that a 1D array of registers actually can be set up to behave in a very similar manner to a plucked string as can be seen in the diagram below.
Figure 1: Karplus-Strong String Simulation
A pluck of the string is initially simulated by loading random values into the delay line which then moves the values up one position in the array at a given speed. By sending the values from this array through the Digital to Analog Converter, we were able to generate a reliable sound for a guitar.
While the algorithm was useful, we still had many hurdles to solve. The first hurdle was to determine which factors to adjust in order to tune the string to the right frequencies. Modifying the low pass filter seems to merely adjust how quickly the sound decays in volume, but does not impact the actual frequency of the tone. Changing the speed of pushing the values through this actually does adjust the frequency of the tone, but it turns out that the easiest way to adjust the frequency is just to use a constant speed and change the length of the delay line.
We simulated a delay line similar to this one in matlab and were able to determine the required length of a delay line to generate the tone for a High E string. We tested by having matlab output the sound and then plugging that sound back into the microphone of the computer which was running a guitar tuner and could tell us if the delay line was the right length.
Once we found the correct value for the length of the delay line, we were able to calculate the required delay line lengths for not only all the open strings, but also those required to simulate playing notes with frets pressed down on strings.
To understand these calculations, it is important to first understand a little bit about the geometry of a guitar neck. As can be seen in Figure 2 below:
Figure 2: Guitar Neck
There are six strings on the guitar neck, and they can either be played as open strings, or they can be played by holding down a fret which functionally shortens the length of the string. The frets themselves are each 0.94387 times the width of the proceeding fret. The first fret then shortens the length of the string by a factor of .94387 and each subsequent fret does the same compared to the length at the previous fret. Thus when we wanted to generate the sound of a High E with the first fret pressed down, we shortened the delay line a factor of .94387 compared to the length for playing the string with no frets pressed.
This allowed us to calculate the delay line length for each fret once we knew the length for playing the string open. Additionally, it is relevant to know that on a guitar, playing the fifth fret of a string produces the same sound as playing the string above it. Looking at Figure 2, this means that playing the 5th fret of the A string produces the same sound as playing the D string with no frets pressed. The only exception to this rule is on the G String where playing the 4th fret corresponds to playing the open B String.
Thus we were able to calculate the lengths of the delay lines at our sample rate entirely based upon knowing the delay line length for the high E string played without any frets pressed down. Using the matlab scheme discussed above, we measured the appropriate value for the length of the delay line for an open High E as 117 registers. The rest of the numbers in table 1 below were easily calculated based on the above mathematics.
By using delay lines of this length, we were able to simulate a guitar with any string being played and with any fret being pressed.
To actually create this simulator, we used six different delay lines each of which were the maximum number of registers that a string could possibly be in length. We used a pointing variable to tell our update code exactly which register in the line represented the “last” register for the particular note being played on that line. The remaining registers had values in them which were irrelevant to the particular note being played, and thus were ignored.
Each time a new note was struck on a string, initial values were loaded into all the registers in the particular delay line for that string and a pointer was assigned to show the “end” of the line in accordance to which string and which fret was being played. Finally the ptrin and ptrout pointers were set to 0 and 1 respectively.
The practical implementation of the Karplus-Strong algorithm actually is implemented by using ptrin and ptrout to take each value in the array, average it with the next value, and store this is in the ptrin position of the array. This averaging actually acts as a very simple low pass filter from the diagram of the algorithm in Figure 1. The value of the delay line at ptrin is the value to be passed as output. Finally ptrin and ptrout are both incremented unless they have reached the end of the line in which case they are set back to 0. It is important to note, that ptrout is always one position ahead of ptrin.
To implement our design, we used both hardware and software. We used the softcore NIOS II processor to compile and execute a program file (written in C language) that generates the data for guitar strings based on Karplus Strong Algorithm. Then those data were sent to the Digital to Analog Converter of the DE2 board’s Audio Codec (enCOder/DECoder) hardware based on the audio sampling clock (fed from audio codec to nios II)to generate audio signal. The line out of the audio codec was connected to a speaker. The clock frequency of the audio sampling clock was 48 KHz (actually 46875 Hz) and the frequency of the NIOS II processor was 100 MHz. The following block diagram shows the basic hardware-software interface of our design.
The Nios II processor was built using the SOPC builder in Quartus II. The processor is a 32 bit RISC processor with 1400-1800 logic elements and an internal cache memory. The processor was also attached to a 4 KB on chip memory, an 8 MB SDRAM, a JTAG UART module, a parallel input and a parallel output interface. The basic interconnection of our NIOS II processor and its peripheral is shown in the following figure.
The 1 bit parallel input interface (AudioControlClock in figure 4) was used to send the audio sampling clock (46.875 KHz) from the hardware (Audio Codec) to the software so that the software knows when to calculate and send calculated data to the hardware to generate the audio signal. The 16 bit parallel output interface (pio in figure4) was used to send the audio data (16 bit) from the software to the hardware (ADC of the audio codec).
We had to use SDRAM along with the on-chip memory since the on-chip memory was not enough for our design. However, due to the physical characteristics of the DE2 board a problem occurs which is known as the ‘clock skew’ problem. Because of this problem, if we feed the same clock to both the NIOS and the SDRAM the system does not work. To solve this problem we had to feed the SDRAM a clock that is 3ns faster than the clock that we fed to NIOS II processor. We used the ‘Mega Wizard plug In Manager’ tool in Quartus II to create a PLL that takes the 50 MHz clock (CLOCK_50) as input and outputs two other clocks of the same frequency (100 MHz) but 3 ns delay in between them.
We used NIOS II IDE to develop, build and execute our C code in the NIOS II processor. Our C code can generate guitar sounds up to nine frets for each string. We used the similar method to develop our c code that Bruce Land used in his Matlab code, provided in the ECE 4760 webpage, for generating the sound of a single guitar string. The program takes, as its input, a number of notes with string number, fret number and the delay (in miliseconds) between two immediate notes. As its output it generates the digital values of the audio signals corresponding to the entered notes with delays in between them. These digital values are then sent to the ADC of the audio codec at every sampling clock to create the audio signal.
We used ‘structure’ in C to define note which has four variables such as string number (stringNum in the code), fret number (fret in the code), delay between notes (delay_length in the code) and the length of the delay line (p_length). Every time we want to create a note we just need to declare a note and assign the values to stringNum, fret and delay_length. We can create up to 32,767 notes in our program. We used two2-dimensional arrays in our program. One is the six by ten fretArray which has the delay lengths for every frets (upto nine) of the six strings; and the other one is the 6 by 569 delay array. This is the array where the two consecutive elements are averaged and the new value is saved in the element with lower index. The p_length element of each note gets its value (the length of delay line) from the fretArray based on the stringNum and fret value of that particular note. The numNotes variable represents the number of notes that will be played by the program. The millisecondcount variable is a counter that increments every millisecond. The note that is being played will be played until the millisecondcount equals to the delay_length of that note.
When the program starts running it first clears all the values of the two dimensional delay variable. Then the random values are assigned to all the elements of the first row of the delay variable which represents the first string or low E. After that, the infinite while loop starts. Inside the loop the audio sampling clock is checked and the millisecondcount counter is increased at every millisecond. Then, distortion is added to the current note if the dist variable set to 1. After that, with the positive edge of every audio sampling clock, two consecutive elements of the delay variable for all six strings are averaged and stored in the element with lower index. At the same audio sampling clock the averaged value is also sent to the audio codec. Then for the same audio sampling clock the next elements in the delay lines get updated until the delay time for a note is reached. When the delay time is reached then the control goes to the next notes and keeps playing until all the notes are played. The delay variable is cleared every time before the next note is played. Executing a bunch of operations for a single cycle of the audio sampling clock is possible because the audio sampling clock is much slower than the processor clock, the audio clock is 46.875 KHz and the operations are being executed at 100 MHz.
At, the beginning we were using timer interrupt to calculate the delay between each note. That procedure did not work out because interrupts were taking a huge number of cycles causing our code to run very slow. As a result we were not getting the expected sound. To resolve this problem we used a counter to manually track the audio sampling clock and to calculate the delay between each note. Beside this, there were several other tricky parts that we encountered in our design. One of them was using the simplest NIOS II processor which does not have internal cache memory. While we were using that processor our program was also running very slowly. When we changed to a better NIOS II processor that had internal cache memory we got expected output.
We were also having trouble with the audio sampling clock frequency. The provided modules in the DSP webpage for generating audio sampling clocks of 8 KHz and 48 KHz was not actually generating the clocks at those speeds. Instead the 48 KHz clock was actually generating 46.875 KHz and the 8 KHz clock was actually generating 4 KHz clock. After solving the interrupt and the processor issue, when we were trying to use the 8 KHz audio sampling clock, we were still not getting the expected output. When we measured the clock frequency in the oscilloscope, then we found out that the clock was running at half of the speed.
In the process of figuring out how to get the timing on the sample clock working correctly, we were able determine how to set up a distortion effect on the guitar. It turned out that by taking the output value on each string and setting it to a maximum value if it was above 0 and to a minimum value if it was below created an extremely accurate distortion like sound. This is similar to the way an old distortion peddle would work, except that ours created a perfect square wave, while an older distortion peddle would have curved corners on each square. This actually changes the sound a bit as ours includes many more of the higher harmonics where the peddle would not have these. The difference in sound is that the peddle sounds less a bit less aggressive. However, the sound we produced is similar to the one you could easily generate using a modern distortion peddle.
Overall, our code was a success. We were able to get the sound values calculated on the fly, in time at 46.875 KHz. We were interested in getting an audio improvement by performing multiple sample calculations per sample cycle. Our motivation resulted from a loss of accuracy at low sample rates. Specifically, when we tried using the 8KHz sample clock, it reduced the length of our delay lines so significantly that it resulted in a loss of accuracy as changing from a delay line of 15 registers to 14 registers only yields very low accuracy in tuning the strings. We attempted to resolve this by still using the 8KHz sample clock, but calculating 6 samples per cycle, and only outputting one of them. We guessed that this would let us use the delay lines lengths applicable for 48KHz sample clocks, but not actually force us to increase the sample rate. This would have been very beneficial while we were struggling to keep up with the speed of the sample clock on the NIOS.
Unfortunately, it turned out that this sampling method resulted in extensive aliasing problems which are in fact what led us down the path of testing the sample clock frequencies on the oscilloscope. We still think that this method is possible, but it would require faster sample clock rates as a baseline, and it would also require a low pass filter to remove the upper harmonics which were yielding the aliasing. In the end, we abandoned this approach because we were able to get our NIOS to run properly at the faster sample rate anyway, and decided that the necessary filters etc, would consume just as many cycles as we might benefit by performing multiple sample calculations at once.
As mentioned earlier, our design generates guitar sound for six strings and for each string up to nine frets. This design also plays song coded in the c program. Coding the notes of a song is very simple and anybody who understand or can read guitar tabs can easily use our design to create a guitar based song. We are very satisfied with the sound that our design generates. We verified the generated string frequencies of our design using guitar tuner and they were very accurate. In terms of speed, our audio signal is generated at 46.875 KHz and the c program is executed at approximately 100 MHz clock speed.
In terms of accuracy, we feel the best test of the accuracy is a comparison of the sound file we generated for the song Enter Sandman by Metallica with the guitar section of the song itself. For most people the sounds and timings will be indistinguishable.
1. "Enter Sandman" by Metallica.
2. "Enter Sandman" by Metallica with distortion.
4. "Snow" by Red Hot Chili Peppers.
At the beginning of this project we wanted to build a basic synthesizable guitar on FPGA that generates guitar sound when some buttons are pressed. Now, at the end of the project, we have a synthesizable guitar that not only generates guitar sounds but also can play songs. Therefore, we can say that the result of our design definitely met our expectation.
As a next step of this design, we would like to add some new features such as implementing ‘hammer on’, ‘pull off’ and ‘slides’. We think one approach to implement ‘hammer on’ in our design would be not clearing the values of the elements of the two dimensional delay variable before a new note is played. We think that if we just changed the StringP value (length of the delay line) for a string without actually clearing the whole line like we do for a new note, we would get something to the effect of a hammer on or pull off sound. In the case of moving the StringP value further down the delay line (a pull off sound) we think the newly available registers in the delay line would be initialized to 0 much like on an actual guitar. Unfortunately, because of the time limitation we could not try this approach.
We are very thankful to Bruce Land and Altera for their helpful tutorials and codes. We reused some of the code provided by them such as Audio_DAC_ADC.v, I2C_AV_Config.v, I2C_Controller.v, Reset_Delay.v and VGA_Audio_PLL.v.
The verilog code, the hardware-software interface such as designing Nios based SOPC and timer interrupt part (for initial software design) was performed by Kamruzzaman Rony. Avtar Khalsa developed the final C code. Both Avtar and Kamruzzaman worked on design specification, software - hardware debugging and testing.