ECE 4760 Spring 2012 Final Project
Auto-Composing Piano
Chaorong Chen (cc899) and Siyu Zhan (sz259)
“A smart piano that composes music of your own taste!” – sound bite
Overview
We designed an electric piano that automatically composes a piece of music in the ECE 4760 final project. All the user needs to do is to select a mood of the music and play two notes upon which the music is based, and then he can just lie down and enjoy the music created for him. He can also choose to train the piano by first playing several pieces of music he is fond of. The piano would then compose music based on the pattern of the music user just played.
Figure 1: Picture of the Auto-Composing Piano
High Level Design
Rationale and Inspiration
Composing has always been an almost forbidden area for people who have no systematic music trainings. Because of the complicated composing rules developed by different composers over the past hundreds of years, it could be a challenge for an amateur to create a piece of music that obeys most of the rules and thus sounds pleasant without multiple times of experimenting. Our project, “Auto-composing System”, helps solve this problem by making composing easier for non-experts.
Our project features a very friendly user interface, which include an electric piano and a keypad. It needs almost no set up and can switch between different modes easily.
Inspired by the third lab in which we used FM synthesis to generate a sequence of sound, we developed algorithms to make the microcontroller (MCU) to not only generate random sequence of piano-like sound, but a piece of “real” music that sounds pleasant and obeys most of the composing rules.
Composing Algorithm Overview
Generally, our composing algorithm takes in three inputs and consists of three stages of composing. The first input that user has to enter is the mood of music that he/she wishes to create. Currently, we support two moods: happy and tender. The other two inputs are two notes chosen by the user.
In the first stage of composing, the tone of the music is determined as a result of the three inputs. Happy mood corresponds to a major tone determined by the two input notes, while tender mood corresponds to a minor tone determined by the two input notes. The speed of the music is also determined in the first stage by the mood input.
The next stage is to create the melody for the music. The first step is to set rhythm corresponding to different moods. After the rhythm is set, the next note of the melody is chosen roughly according to the corresponding Markov probability matrix until all notes of the melody is filled. We incorporate composing rules that cannot be represented by the probability matrix to the melody composing process by either setting certain notes deterministically or changing the probability matrix when choosing notes for some special rhythm.
After the melody is set, our algorithm chooses proper chord for it. The chord is chosen so that it sounds harmonic together with the melody and also varies with some randomness to avoid monotony.
There is a special train mode in our composing system that lets the user to build his or her own Markov probability matrix instead of using our pre-trained ones. If the user chooses to enter the train mode, the system allows the user to play a piece of music on the electric piano. Then the system will compute the Markov probability matrix based on the piece of music that the user played and thus could generate music that assemblies what the user just played with some variations.
Figure 2: Block Diagram of Composing algorithm
Logical Structure
There are three primary components involved with the system: the input devices, the ATmega 644 MCU and sound output device.
The input devices consist of two parts: the piano input and the keypad. The piano consists of 23 keys ranging from G2 to F4 that user can play. The keys are divided into 3 sections and are treated as independent switches. Each section of the keys is connected to an 8-3 encoder to get four outputs that feed into the STK 500. The purpose of the keypad is for user to operate the composer. When the system starts, the user is supposed to first select a mood and whether he would like to train the piano by entering the corresponding keys from the keypad. If the user chooses to train the composer, he is also supposed to indicate he has finished training by pressing “0” button on the keypad. The user can also press the “*” button on the keypad to indicate he would like to stop the current music and restarts the process. Both the piano keys input and the keypad is debounced in software to avoid undesired inputs.
The ATmega 644 MCU is the “brain” of the auto-composing machine. It takes input from the keypad and piano and composes music catering to the user’s preferences. Described in the composing algorithm above, the MCU identifies the key user presses and the notes user plays, and updates the melody, chord and rhythm by adjusting entries of the probability matrices. The MCU then synthesizes the corresponding piano sound using Direct Digital Synthesis (DDS) and Frequency Modulation (FM) synthesis with a 16 kHz sampling rate. The sound signal is fed into the output device through Timer 0 Pulse Width Modulation.
The output devices are fairly simple. An RC low-pass filter reduces the noises carried in the PWM output signal from the MCU and is connected to a speaker that generates the physical sound. A PUTTY terminal in the computer serves as the user interface to give proper instructions to user about how operate the composer.
High Level Block Diagram
Figure 3: High Level Block Diagram
Hardware / Software Tradeoff
For the system to compose better music pieces, it requires that there to be more memory spaces in the MCU to maintain a large database for variations of rhythm and melodies. At the same time, the piano synthesis also requires more memory to generate more voices similar to piano sound and needs higher sampling rate to produce better sound quality. However, due to the limited memory space on the ATmega 644 MCU, the two sides have to be balanced. As a result, we synthesize three piano voices at 16 kHz sampling rate with FM synthesis with a single envelope. We also limit the range of notes the system can play to three octaves, and have the rhythm of the music to be a combination of eight different variations. The settings have been tested to produce the best result given the resources.
To give more space to music composition and piano synthesis, we also used PUTTY as user interface instead of LCD since updating LCD takes too much time and is not necessary for the project.
Related Projects and Research Papers
There are certain projects and research papers that are relevant to our auto-composing system, though the focus is different.
Microsoft’s SongSmith is a project that the computer generates chords to accompany the melody as user sings. It uses a Hidden Markov Model to predict the next note the user is likely to sing and generate chords accordingly. Allan and William’s research paper “Harmonising Chorales by Probabilistic Inference” focuses on generating harmonization of Johann Sebastian Bach’s style to a melody by using Hidden Markov Model.[1] Chuan and Chew’s research paper “A Hybrid System for Automatic Generation of Style-Specific Accompaniment” describes their system which is able to generate style-specific chord for a melody by incorporating the New-Riemanian transforms between checkpoints into a Markov chain. [2]
Our
project, the “Auto-composing System”, is very different from all the above
projects in that our system is able to generate the melody itself, rather than
generating a chord real time for an unknown melody. The generation of chord of
our system is different from the above projects too, since the melody is
determined and known at the time when we generate the chord, instead of using a
Markov chain to generate chord, we use a different probabilistic approach.
Hardware Design
Hardware Design Overview
The purpose of the hardware in the project is to provide the MCU with user’s selection and the notes user played. And also clearly generate the physical sound from the pulse width modulated signal from the MCU. The system is designed to minimize the operating complexity and maximize the sound quality.
Piano Keyboard Input
The 23 piano keyboard keys are each connected to a wire on the circuit board. To make them to active-low switches that are electrically detectable, we connected the wires to VCC through a 5 kW resistor. Thus, when the keys are idle, the output voltage is VCC and when keys are pressed, the output voltage approaches zero.
Due to the limited number of input ports on the STK 500, the 23 piano keys from the keyboard cannot be directly connected to the MCU. Instead, they are connected to three 8-3 hardware priority encoders to reduce the number of inputs. The piano keys are divided into three sections, each correspond to seven or eight inputs. Each section is directly connected to its own encoder and produces four encoded outputs: the Enable Output (EO) bit and A0 to A2 encoded bits. The total twelve outputs from the three sections are then connected to the input ports of the STK 500. The software will later decode and debounce the signals to determine which key has been pressed.
The idea of reading piano input via priority encoder is influenced by a previous year’s project (A Keyboard Synthesizer Workstation by Matt Ferrari and Alec Hollingswirth).
Figure 4: SN74HC148N Priority Encoder [3]
Keypad Input
The keypad we used in this project is a 9-pin one with connectors on the top. The first four pins of the connector are connected to the four columns of the keypad and the following four pins connected to the four rows of the keypad. To figure out which button the user has pushed, a lookup table is set up in the software to determine the correct button. The keypad inputs are also debounced in software using a state machine we’ve used for push buttons.
Figure 5: The keypad circuit
Low Pass Filter Output
The pulse width modulated signal output from the MCU carries high frequency components resulting from the edges of the square waves. Those high frequency noises sound very unpleasant in the speaker and would ruin the overall sound quality if not processed. Thus, we add a low-pass filter with a 10 kW resistor and a 0.1uF capacitor to get the time constant to be 1ms. This filter effectively eliminates the undesired harsh harmonics of the PWM output.
Software Design
I. Composing Algorithm
Sequencer Design
The sequencer we designed generates a piece of music of 15 bars long and the time signature of the music generated is a fixed 4/4. This time signature means that each bar is consisted of four quarter-note (crotchet) beats. The shortest beat that the algorithm calculates is eighth-note (quaver) beat, which means that for each bar, the sequencer needs to calculate eight different notes. For the total of 15 bars, the sequencer calculates 120 notes for a whole piece of each voice.
The goal of the sequencer section of our software is to generate two 1 by 120 matrices for melody and chord respectively which represents the notes to be played in sequence. A minus one entry of the matrix corresponds to an empty note, and all other non-minus-one entries correspond to a note in the range of C2 to C5, in which the entry value represents the distance between the note and C2. A semitone or half-step is counted as one in distance. The two matrices are then passed to FM synthesis process to generate corresponding notes using PWM output of the microcontroller. The notes generated by the algorithm should be relevant to the two notes that the user inputs and reflect the mood that the user chooses. Furthermore, it should obey most of the composing rules and sounds pleasant when played out on speakers. Following is a detailed description of our implementation of the sequencer that meets the above requirements.
Tone and Speed Determination
The first step in sequencer is tone and speed determination. The speed of the music is determined by the length of the quarter-note, which is determined by the time interval between two notes. Since we use TIMER1 interrupt to count time and play the next note when time1 counts to t1, we change the value of t1 to control the speed of music. After experimenting, we set t1 to 200ms for happy mood, and 350ms for tender mood. Therefore, in happy mood, the speed is 150 beats per minute, and in tender mood, the speed is roughly around 85 beats per minute.
For tone determination, a 12-note “scale” matrix (1 by 12) is created corresponding to different moods and notes input. Normal scale consists of 8 notes, but in this case, to include more possible notes for melody generation, we created a scale of 12 notes, in which the first 8 notes are exactly same as in normal scales, and the following 4 notes are the first 2 to 5 notes in a higher octave. The entry of the matrix represents the distance from current note to C2. For example, a 12-note major scale of C3 would be represented as:
[12 14 16 17 19 21 23 24]
To express different moods, we create major 12-note scale matrix for happy mood, and harmonic minor 12-note scale for tender mood.
The choice of the starting note for different 2-note input combinations can be tricky, because this choice, together with the tonality (major/harmonic minor), determines the 12-note scale from which melody is generated. The scale not only needs to include the 2 input notes, but also should span a reasonable range among the notes that could be played (C2 to C5). A scale too low or too high makes the melody sounds very unpleasant according to our experimentation.
Our final solution for tone determination is as follows. First we change both input notes into the range of C3 to C4 by changing it one octave up/down if needed. Next we calculate the distance difference between them, which should always be positive. Then we use the following lookup table to determine the starting note of the 12-note scale and generate the scale matrix based on the mood input.
|
Distance Difference |
Starting Note for Major Scale |
Starting Note for MinorScale |
|
0 |
Lower Note |
Lower Note |
|
1 |
Lower Note + 1 |
Lower Note + 1 |
|
2 |
Lower Note |
Lower Note |
|
3 |
Lower Note +8 |
Lower Note |
|
4 |
Lower Note |
Lower Note + 9 |
|
5 |
Lower Note + 5 |
Lower Note +5 |
|
6 |
Lower Note + 7 |
Lower Note +7 |
|
7 |
Lower Note |
Lower Note |
|
8 |
Lower Note + 8 |
Lower Note |
|
9 |
Lower Note +5 |
Lower Note + 7 |
|
10 |
Lower Note + 10 |
Lower Note + 10 |
|
11 |
Lower Note |
Lower Note |
Table 1: Tone Lookup Table
Melody Determination
The melody determination process generates a 1 by 120 melody matrix. The entries of this matrix represent melody notes by their distance to C2, and
-1 represents an empty note. There are two steps in melody generation, setting rhythm and setting tune.
Set Rhythm
As mentioned before, the whole piece of music generated consists of 15 bars, and each bar contains 8 eighth-notes with each corresponding to an entry in the melody matrix. To set the rhythm of the whole music, we chose 8 typical rhythm patterns for each mood (16 in total) from some classic pieces, and each rhythm pattern is exactly one bar long, which is 4 quarter-notes. Also the shortest beat in these rhythm patterns have to be eighth-note beat to be consistent with our system. A one in the rhythm pattern means that it is a beat and a zero means it is empty. For each bar of the 15 bars in the melody, a random number is generated to choose one of the eight rhythm patterns of the mood chosen. Also, to make the whole melody sounds more integrated like the real music, the rhythm of the 5th, 10th, and 15th bar is set to serve as an end to a musical phrase. In this way, the rhythm of the whole melody is determined.
One thing to note is that because it is very hard to change the delay time of a certain note without changing the timber of the sound using FM synthesis, for beats that are longer than eighth-note beat, such as quarter-note beat and half-note beat, they are represented as one eighth-note beat at first followed by a number of empty beats. We don’t repeat the note several times for a longer note because that sounds very obviously as repetition rather than a note lasting longer.
Set Tune
The tune of the melody is set based on both the 12-note scale matrix generated in tone determination process and the Markov probability matrix. There are two Markov probability matrices, each corresponding to a mood. Both probability matrices are 12 by 12, and both the row and column indices corresponds to the 12 notes in scale matrix. The row index represents current note, and each entry in the row represents the probability of the next note being the note corresponding to column index. Since we use int8_t for most of the operations, the probabilities of each row is normalized to be sum up to 127.
The general process of choosing the next note for the melody is as follows. A random number between 0 and 127 is first generated. Then the system finds the row in probability matrix corresponding to current note and compares the random number and the cumulative probabilities of each column in the row until it exceeds the random number. Then the note corresponding to that column is chosen to be the next note. An example of this process is as follows.
if current note is C3 and the row corresponding to C3 in the probability matrix is:
C D E F G A B C D E F G
[0 0 0 50 50 0 27 0 0 0 0 0]
and if the generated random number is 56, then the next note should be G3.
There are certain circumstances where the note generation does not follow the above process. The first case is the first two notes of the melody. The system is programmed to set the first two notes of the melody to the two input notes to make the music sounds more like a piece inspired by the two notes that the user played. The second case is that for notes longer than half-note, the probability of playing the first, fifth, and eighth note of the scale matrix is much higher than playing other notes, because these notes gives an ending feeling to a musical phrase.
Based on the above rules, the notes of the 1 by 120 melody matrix is filled when the corresponding beat in rhythm matrix is non-zero. For zero beats in rhythm matrix, the corresponding note in the melody is set to -1 to indicate an empty note.
Chord Determination
To ensure that the chord and melody creates a harmony in any case, there are three choices for a chord note to a melody note. It could either be 5, 7, or 12 semitones below the melody note. Five semitones below create a perfect fourth; seven semitones below create a perfect fifth; twelve semitones below creates the same note an octave down. These three choices are in perfect harmony with the original note and thus would sound pleasant in any case.
The system generates chord for the melody in the following way. Firstly, it only generates chords if the corresponding melody note is not empty. This is to ensure that the melody rhythm is clear and won’t be disturbed by the chord. Next, there are 87.5% that there is a chord for a certain non-empty melody note. This is to make sure that there are some variations in the chord. Thirdly, a random number is generated to choose a chord for the melody note from the above three choices. In this way, the chord generated is always in harmony with the melody, and is not monotonous. The chord is also a 1 by 120 matrix where each entry is an eighth-note in a bar and represents the note by its distance to C2. A -1 in the chord matrix also means an empty note.
Train
If a user chooses to train the system himself, a Markov probability matrix is calculated based on the user’s input on the keyboard. For the 24 keys on the keyboard, all keys lower than C3 is counted as one octave above because of the size of the probability matrix. All black keys are counted as the white key one semitone below it to ensure that all users’ inputs can be recorded by the probability matrix.
The first step of training algorithm is to collect the times of every note played right after every note and construct a training matrix. The training matrix is also 12 by 12 and both the row and column indices represent the 12-note major scale starting from C3. Following is an example of building a training matrix. If C3 is played right after G3, then the entry of the fifth row (G3) and first column (C3) should increment by 1. The second step is when the user finishes all his input and presses Button 0 on the keypad, each row of the training matrix should be normalized to 127. If there is an empty row, 127 is assigned to the following note in the scale except G4. If the row of G4 is empty, 127 is assigned to C4. In this way, a raw probability matrix is built according to user input.
However, there is a large chance that the user might input some sequence that is not likely to generate very pleasant music. Therefore, after multiple times of experimenting, we decided to put the training matrix through a filter before it goes into the normalizing stage. For most of the time, an unpleasant sequence results from a large jump between notes, so we build a simple band pass filter that eliminates the large jump between notes. Following is a table showing how the band pass filter works
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For each row in the training matrix, we keep the value of the shaded area and set all other entries to 0. In this way, our training algorithm works very well and is able to generate pleasant music relevant to the user’s input.
In this way, our training algorithm works very well and is able to generate pleasant music relevant to the user’s input.
II. Piano Synthesis
To synthesize the sound of a piano, we tried different synthesis methods including Karplus Strong algorithm and Frequency Modulation synthesis. We end up using FM synthesis since a combination of Direct Digital Synthesis (DDS) to generate sine waves and FM synthesis to modulate the signal produces a voice most similar to piano. As mentioned above, three piano sounds are generated at the same time in 16 kHz sampling frequency by the MCU.
Direct Digital Synthesis
Direct digital synthesis is a method of generating a pure sine wave by generating a time-varying signal. In this project, we use the 16-bit register as a phase accumulator. The higher 8 bits are used as the phase register for index array and the lower 8 bits are used as sine table lookup. The sine table is initialized in initialize function and updated in timer1 ISR to produce the base frequency.
The output frequency can be tuned by selecting the increment value in DDS. Since the output frequency is the inverse of the time it takes to overflow the 16-bit register, we get
and thus increment = 4.096 × fout for 16kHz sampling rate.
Figure 5: phase -increment in DDS
Piano Synthesis Using Frequency Modulation
FM synthesis is a form of audio synthesis where the timbre of a simple waveform is changed by frequency modulating it with a modulating frequency that is also in the audio range, resulting in a more complex waveform and a different-sounding tone. [4] The formula of FM synthesis used in this project is
In the formula, amp(t) is the modulating envelope and is a waveform of exponential rise and exponential decay which means
)
and it is the numerical solution of an ordinary differential equation calculated in Timer1 ISR using 8:8 fixed point calculation. f0 in the formula is the base frequency and is represented as inc_main in the code. The parameter A0, tf, tr, amp_fm, fm_depth and f_fm are all calculated in Timer1 ISR and can be controlled using parameters decay_main, rise_main, inc_fm, depth_fm1 and decay_fm1.
Figure 6: FM synthesis waveform
After careful tuning and experimentation, the optimal parameter combination to generate a piano sound by FM synthesis is found. The most important fact is that the modulating frequency should be the third harmonic of the base frequency to produce a piano sound.
Timer0 PWM output
In this project, three FM modulated voices are generated in Timer 1 ISR to serve as main melody and chords, each with its own note sequence selected by the sequencer. The three voices are added together to OCR0A and output to the speaker through Pin B3 which is the PWM output pin for Timer 0.
Since we have three different voices, we used three different plucks in main to start playing a note by setting pluck to 1. And the time interval is controlled using time1 variable.
Alternative Piano Synthesis Method
Another popular sound synthesis method is the Karplus Strong algorithm. The Karplus Strong algorithm is particularly well suited for implementation on a microcontroller due to its simple arithmetic. It is also a very good method to synthesize plucked string sound including the piano. For these two reasons, we attempted to implement the piano synthesis using Karplus Strong algorithm for more than a week to get it working. However, we eventually gave up the method since the high frequency notes generated by Karplus are not similar to a piano and the system produces unwanted noise when Karplus was implemented.
Results
Speed of Execution
The sound produced at 16 kHz sampling rate is clean and similar to the one generated by a piano. It takes is 662 cycles plus 64 cycles which equals 726 cycles total for the Timer 1 ISR to synthesize the sound. So with OCR1A = 999, it takes 72.6 percent of CPU time to do piano synthesis. During the operation of the system, there is no observable delay in sound generation.
Accuracy
To measure the accuracy of the base frequencies, we used a deterministic, ascending sequencer with a long time interval to output sound wave in PWM signal and used an oscilloscope to measure the wave frequencies. The accuracy is within four percent of the actual frequency.
For the hardware input, the D4 key on the piano keyboard is sometimes not detectable by the MCU due to the wiring of the keyboard circuit. We tried to re-solder the wire on the circuit board but it still wasn’t working very well. So we used D4 sharp key for D4 key when user needs that input.
Figure 7: The encoder output on the oscilloscope
The Composing Algorithm
The result of the composing algorithm is very successful. It meets all the expectations and is able to generate a piece of pleasant music according to user’s input. Tone, melody and chord determination works as expected in creating the melody and chord matrix. Training algorithm also works as expected to alter the probability matrix based on user’s input.
As for the quality of the music generated by the algorithm, currently there is no universal standard to test the quality of a piece of music, so it is hard to measure the quality of music accurately. The music generated by our system is very rarely to be unpleasant, and for around 40% of the time, it generates music that actually makes us feel very enjoyable. Therefore, we would say that the composing algorithm in all is very successful.
Conclusion
Overall, the auto-composing piano project was functioning as we expected. The music produced is similar to a piano sound and the composing algorithm is working well. For future improvement, we could use more machine learning skills to let the composer better learn the pattern of the music user inputs and composes music that are more sounding and catering to the user’s preference. We could also use a solder board instead of STK 500 to increase portability and further reduce the budget of the project.
Applicable Standards
There are no relevant standards applicable to this project.
Intellectual Property
The idea of reading piano input via
priority encoder is influenced by a previous year’s project (A Keyboard
Synthesizer Workstation by Matt Ferrari and Alec Hollingswirth). The code to
perform the DDS and FM synthesis is a reuse of the code from Bruce Land’s DSP
page. The link to the page can be found here: http://people.ece.cornell.edu/land/courses/ece4760/Math/avrDSP.htm
Ethical and Legal
Considerations
With reference to the IEEE code of ethics, we completed this project in an ethical manner, and made sure that the finished project does not violate any ethical considerations. We accepted the responsibility that our auto-composing piano would not harm other people and not violating regulations and laws. We reported our results accurately and based on facts. We have not concealed the shortcomings of our device nor have we made untruthful claims about our project. We have sought and accepted criticism in the design of this project, and referenced the projects and research papers that are similar to our project. We have explained why we believe we are not infringing on any of their intellectual property in the design of our auto-composing piano.
In the collegial spirit, we have tried our best to help our peers whenever they had any questions related and unrelated to our project without any discrimination. When designing the auto-composing system, we have also kept in mind the purpose of improving the understanding of technology, its appropriate application. We hope that this system would help people better understand technology’s impact on music and how technology and art are well connected.
Legal Considerations
We do not violate anyone's intellectual property rights nor does our auto-composing piano cause harm to others. Since we do not broadcast any RF signals, we need not to be concerned with FCC regulations. Therefore, our project is in compliance with current legal standards.
References
[1] “Harmonising Chorales by Probabilistic Inference.” Moray Allan
and Christopher K.I. Williams. NIPS 2005.
[2] “A Hybrid System for Aautomatic Generation of Style-Specific
Accompaniment.” 4th Intl Joint Workshop on Computational Creativity, June 2007.
[3] SN74HC148 Priority Encoder Data Sheet, Texas Instruments.
www.ti.com/lit/ds/symlink/sn74hc148.pdf
[4] “FM Synthesis.” Wikipedia, The Free Encyclopedia. Wikimedia
Foundation, Inc.22 July 2004. Web. 14 March. 2012
Appendix
A. Schematics of the Project
Figure 8: Hardware schematic of the project
B. Parts List and Cost Details
|
Part |
Source |
Unit Price |
Quantity |
Total Price |
|
STK 500 |
ECE 4760 Lab |
$15.00 |
1 |
$15.00 |
|
Mega 644 |
ECE 4760 Lab |
$ 6.00 |
1 |
$ 6.00 |
|
Speakers |
ECE 4760 Lab |
$ 0.00 |
1 |
$ 0.00 |
|
White Board |
ECE 4760 Lab |
$ 6.00 |
2 |
$ 12.00 |
|
SN74HC148N Priority Encoder |
Digikey |
$ 0.43 |
3 |
$ 1.29 |
|
Keypad |
ECE 4760 Lab |
$ 6.00 |
1 |
$ 6.00 |
|
Toy Piano |
ECE 4760 Lab |
$ 0.00 |
1 |
$ 0.00 |
|
5kW resistor |
ECE 4760 Lab |
$ 0.00 |
24 |
$ 0.00 |
|
1 kW resistor |
ECE 4760 Lab |
$ 0.00 |
1 |
$ 0.00 |
|
.1 uF capacitor |
ECE 4760 Lab |
$ 0.00 |
1 |
$ 0.00 |
|
|
|
|
Total |
$ 40.29 |
C. Work Division
|
Chaorong Chen |
Both |
Siyu Zhan |
|
Piano Synthesis |
Project Idea Research |
Composing Algorithm Design |
|
Hardware Input and Output Design |
Hardware/Software Debugging |
Composing Algorithm Implementation and Testing |
|
Schematic |
Project Report |
Music Tuning |
D. Commented Code
//ECE 4760 Final Project: Auto-composing Piano
//Siyu Zhan, sz259@cornell.edu
//Chaorong Chen, cc899@cornell.edu
#include <stdlib.h>
#include <string.h>
#include <avr/pgmspace.h>
#include <avr/eeprom.h>
#include <util/delay.h>
#include <inttypes.h>
#include <avr/io.h>
#include <avr/interrupt.h>
#include <math.h> // for sine
#include <stdio.h>
#include "uart.h"
// set up serial for debugging
FILE uart_str = FDEV_SETUP_STREAM(uart_putchar, uart_getchar, _FDEV_SETUP_RW);
//**********************************************************
//all the definitions
#define begin {
#define end }
#define max_amp 32767
// masks for shift register
// 32 bits
#define bit30 0b01000000000000000000000000000000
#define bit27 0b00001000000000000000000000000000
//For debouncing timing
#define t3 20*16
//for keypad scan
#define maxkeys 16
#define PORTDIR DDRA
#define PORTDATA PORTA
#define PORTIN PINA
//the for Pushstates for debouncing
#define NoPush 0
#define MaybePush 1
#define Pushed 2
#define MaybeNoPush 3
//For keypad table
#define button1 1
#define button4 5
#define button7 9
#define button_star 13 //Indicate Stop
#define button2 2
#define button5 6
#define button8 10
#define button0 14 //Indicate User don't need to train the composer
#define button3 3
#define button6 7
#define button9 11
#define button_pd 15 //Indicate User like to train the composer
#define buttonA 4 //User selected happy mood
#define buttonB 8 //User selected Tender mood
#define buttonC 12
#define buttonD 16
//for Sequencer
#define happy 0
#define tender 1
#define want_train 1
#define not_train 0
#define invalid -1
//For note base frequency
#define C2i 65
#define C2_si 69
#define D2i 73
#define D2_si 78
#define E2i 82
#define F2i 87
#define F2_si 92
#define G2i 98
#define G2_si 104
#define A2i 110
#define A2_si 117
#define B2i 123
#define C3i 131
#define C3_si 139
#define D3i 147
#define D3_si 156
#define E3i 165
#define F3i 175
#define F3_si 185
#define G3i 196
#define G3_si 208
#define A3i 220
#define A3_si 233
#define B3i 247
#define C4i 262
#define C4_si 277
#define D4i 294
#define D4_si 311
#define E4i 330
#define F4i 349
#define F4_si 370
#define G4i 392
#define G4_si 415
#define A4i 440
#define A4_si 466
#define B4i 494
#define C5i 523
//Total number of notes
#define note_range 37
//Ratio of inc_fm to base frequency to immitate certain sound
#define piano_ratio 3
//**********************************************************
//All function Declarations
void initialize(void);
//random number generator using shift register, return a random integer
int rand_shift(void);
//scan the keypad and return the butnum for which key pressed
char keypad(void);
//read the piano keys, returns the index of note played
char piano_read(void);
//debouncing the keypad every 30ms
void keypad_debounce(void);
//debouncing the piano board every 30ms
void piano_debounce(void);
//calculate the tone of the music
void set_tone(void);
//calculate the main melody of the music
void set_melody(void);
//calculate the chords accompanying the melody
void set_chord(void);
//load the sequence of music to play
void sequencer(int8_t note_1, int8_t note_2, int8_t mood);
//adjust music scale according to tone and key user played
void create_scale(int8_t c_note, int8_t tonality);
//returns the postion on the scale, helper method for create_scale
int8_t scale_pos(int8_t note);
//Let user train the composer by playing a piece of music
void train(int8_t cur_note);
//adjust the position for training
int8_t train_pos(int8_t note);
//helper methods for finding sum and max of a matrix row
int max_raw_prob(int row_num);
int sum_train_prob(int row_num);
int sum_raw_prob(int row_num);
//update the probability matrix after user's training
void train_prob_calc(void);
//filter the train prob matrix to keep it centralized
void train_prob_filter(void);
//**********************************************************
//All variables and flags
//Different Push Button Variables
volatile unsigned int PushState_piano;
//for keypad debouncing
volatile unsigned int PushState;
//key pad scan table
unsigned char keytbl[16]= {0xee, 0xed, 0xeb, 0xe7,
0xde, 0xdd, 0xdb, 0xd7,
0xbe, 0xbd, 0xbb, 0xb7,
0x7e, 0x7d, 0x7b, 0x77};
//piano table PINC value shift left by 8 and or with portA
//highest octave connected to PORTC4-7, middle PORTC0-3, lowest PORTA0-3
uint16_t pianotbl[23] =
{((0x77<<8)|0xA0),((0x77<<8)|0x90),((0x77<<8)|0xB0),
((0x77<<8)|0xC0),((0x77<<8)|0xD0),((0x77<<8)|0xE0),
((0x78<<8)|0x70),((0x77<<8)|0xF0),((0x79<<8)|0x70),
((0x7A<<8)|0x70),((0x7B<<8)|0x70),((0x7C<<8)|0x70),
((0x7D<<8)|0x70),((0x7E<<8)|0x70),((0x7F<<8)|0x70),
((0x87<<8)|0x70),((0x97<<8)|0x70),((0xA7<<8)|0x70),
((0xB7<<8)|0x70),((0xC7<<8)|0x70),((0xD7<<8)|0x70),
((0xE7<<8)|0x70),((0xF7<<8)|0x70)};
//For decoding and debouncing piano readings
uint16_t piano_reading;
char piano_index = 0;
char act_piano_index = 0;
//**********************************************************
//Keypad Variables
// The raw keyscan
unsigned char key ;
// The decoded button number
unsigned char butnum,act_button ;
// The DDS variables for voice 1: main melody
volatile unsigned int acc_main, acc_fm1 ;
volatile unsigned char high_main, high_fm1, decay_fm1, decay_main, depth_fm1, rise_main ;
volatile unsigned int inc_main, inc_fm1, amp_main, amp_fm1 ;
volatile unsigned int rise_phase_main, amp_rise_main, amp_fall_main ;
// The DDS variables for voice 2: chord number 1
volatile unsigned int acc_main_v2, acc_fm1_v2 ;
volatile unsigned char high_main_v2, high_fm1_v2, decay_fm1_v2, decay_main_v2, depth_fm1_v2, rise_main_v2 ;
volatile unsigned int inc_main_v2, inc_fm1_v2, amp_main_v2, amp_fm1_v2 ;
volatile unsigned int rise_phase_main_v2, amp_rise_main_v2, amp_fall_main_v2 ;
// The DDS variables for voice 3: chord number 2
volatile unsigned int acc_main_v3, acc_fm1_v3 ;
volatile unsigned char high_main_v3, high_fm1_v3, decay_fm1_v3, decay_main_v3, depth_fm1_v3, rise_main_v3 ;
volatile unsigned int inc_main_v3, inc_fm1_v3, amp_main_v3, amp_fm1_v3 ;
volatile unsigned int rise_phase_main_v3, amp_rise_main_v3, amp_fall_main_v3 ;
// tables for DDS
signed char sineTable[256], fm1,fm1_v2,fm1_v3 ;
// trigger
volatile char pluck_v1,pluck_v2,pluck_v3, pushed ;
volatile int8_t piano_input1,piano_input2;
// Time variables
// the volitile is needed because the time is only set in the ISR
// time counts mSec, sample counts DDS samples (62.5 KHz)
// time 3 for debouncing keypad, time 1 for note changing interval
volatile unsigned int time, time1, time3,cycle_count ;
volatile unsigned int t1; //the variable that controls time interval and rhythm, originally a define
volatile char count;
// index for sine table build
//unsigned int index;
//**********************************************************
//For Sequencer/////
unsigned int rand_num_v1,rand_num_v2,rand_num_v3;//random number for sequencer
volatile unsigned int current_note_v1,current_note_v2,current_note_v3;//the index of current note
volatile unsigned long accumulator;//for random number
//array containing the frequencies of all 3 octave of notes
unsigned int note_arr_int [note_range] = {C2i,C2_si,D2i,D2_si,E2i,F2i,F2_si,G2i,G2_si,A2i,A2_si,B2i,
C3i,C3_si,D3i,D3_si,E3i,F3i,F3_si,G3i,G3_si,A3i,A3_si,B3i,
C4i,C4_si,D4i,D4_si,E4i,F4i,F4_si,G4i,G4_si,A4i,A4_si,B4i,C5i};
//array containing all the inc_main values for notes used for FM synthesis
unsigned int inc_main_arr [note_range] = {268,284,301,319,338,358,379,401,425,451,477,506,537,568,601,
637,675,715,758,803,851,901,955,1011,1072,1135,1203,1274,1350,1430,1515,1606,1701,1802,1909,2023,2143};
//corresponding array for inc_fm1
unsigned int inc_fm1_arr [note_range];
//variables for composing algorithm
int8_t delay_flag;
//for training to determine the key pressed
int8_t train_pre;
//whether user need to train the piano
int8_t train_mode;
//whether user chose to stop and reset
char reset;
//determine the note decay time
int8_t decay_t;
int8_t decay_t1;
int8_t decay_t2;
//access the note array
int8_t note_index;
//0 for happy, 1 for blue
int8_t mood;
//the 2 note played for train mode
int8_t note_1;
int8_t note_2;
//notes to choose from to compose current piece
int8_t scale[12];
//chord and melody array
int8_t chord_1[120];
int8_t melody[120];
//Probability matrix based on default training
int8_t prob_h_origin[12][12] ={{27,69,31,0,0,0,0,0,0,0,0,0},
{44,16,41,19,7,0,0,0,0,0,0,0},
{28,33,17,24,23,2,0,0,0,0,0,0},
{0,0,58,38,28,3,0,0,0,0,0,0},
{0,3,38,33,36,17,0,0,0,0,0,0},
{0,0,7,0,51,56,0,13,0,0,0,0},
{0,0,0,0,0,0,0,127,0,0,0,0},
{0,0,0,0,64,63,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0,0,127,0,0},
{0,0,0,0,0,0,0,0,0,0,127,0},
{0,0,0,127,0,0,0,0,0,0,0,0},
{0,0,0,0,127,0,0,0,0,0,0,0}};
int8_t prob_t_origin[12][12] ={{55, 54, 12, 6, 0, 0, 0, 0, 0, 0, 0, 0},
{32, 11, 68, 3, 13, 0, 0, 0, 0, 0, 0, 0},
{9, 38, 17, 23, 35, 5, 0, 0, 0, 0, 0, 0},
{4, 0, 47, 47, 22, 7, 0, 0, 0, 0, 0, 0},
{0, 10, 11, 24, 42, 26, 4, 10, 0, 0, 0, 0},
{0, 0, 0, 0, 39, 20, 32, 32, 4, 0, 0, 0},
{0, 0, 0, 0, 18, 35, 35, 39, 0, 0, 0, 0},
{0, 0, 0, 0, 28, 13, 13, 45, 25, 3, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 45, 52, 30 ,0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 81, 46, 0 ,0},
{0, 0, 0, 127, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 127, 0, 0, 0, 0, 0, 0, 0}};
//Happy and tender matrices
int8_t prob_t[12][12],prob_h[12][12];
// The probability matrix that are used for training
int8_t train_prob[12][12];
int8_t raw_prob[12][12];
//The 8 varaitions of rhythm arrays
int8_t happy_rhy_1[8] = {1,1,1,0,1,1,1,0};
int8_t happy_rhy_2[8] = {1,0,1,1,1,0,1,1};
int8_t happy_rhy_3[8] = {1,0,1,0,1,1,1,1};
int8_t happy_rhy_4[8] = {1,1,1,1,1,0,0,0};
int8_t happy_rhy_5[8] = {1,0,1,1,1,0,1,1};
int8_t happy_rhy_6[8] = {1,1,1,0,1,1,1,0};
int8_t happy_rhy_7[8] = {1,1,1,1,1,0,1,0};
int8_t happy_rhy_8[8] = {1,0,0,1,1,0,0,1};
int8_t blue_rhy_1[8] = {1,0,0,1,1,0,0,1};
int8_t blue_rhy_2[8] = {1,0,0,1,1,1,1,1};
int8_t blue_rhy_3[8] = {1,1,1,1,1,0,1,0};
int8_t blue_rhy_4[8] = {1,0,0,1,1,0,0,0};
int8_t blue_rhy_5[8] = {1,0,0,1,1,0,0,0};
int8_t blue_rhy_6[8] = {1,1,1,1,1,0,1,0};
int8_t blue_rhy_7[8] = {1,0,0,1,1,0,1,0};
int8_t blue_rhy_8[8] = {1,1,1,0,1,0,1,0};
//**********************************************************
// timer 1 capture ISR
ISR (TIMER1_COMPA_vect){ // Fs = 8000
TCNT2 = 0; TCCR2B = 2;
//************************************************************************
// FM synthesis for voice 1
// compute exponential attack and decay of amplitude
// the (time & 0x0ff) slows down the decay computation by 256 times
if ((time & 0x0ff) == 0) begin
amp_fall_main = amp_fall_main - (amp_fall_main>>decay_main) ;
rise_phase_main = rise_phase_main - (rise_phase_main>>rise_main);
// compute exponential decay of FM depth of modulation
amp_fm1 = amp_fm1 - (amp_fm1>>decay_fm1) ;
end
// form (1-exp(-t/tau)) for the attack phase
amp_rise_main = max_amp - rise_phase_main;
// product of rise and fall exponentials is the amplitude envelope
amp_main = (amp_rise_main>>8) * (amp_fall_main>>8) ;
// Init the synth
if (pluck_v1==1) begin
amp_fall_main = max_amp;
rise_phase_main = max_amp ;
amp_rise_main = 0 ;
amp_fm1 = max_amp ;
// phase lock the synth
acc_fm1 = 0 ;
acc_main = 0;
pluck_v1 = 0;
end
//the FM DDR -- feeds into final DDR
acc_fm1 = acc_fm1 + inc_fm1 ;
high_fm1 = (char)(acc_fm1 >> 8) ;
fm1 = sineTable[high_fm1] ;
//the final output DDR
// phase accum = main_DDR_freq + FM_DDR * (FM amplitude)
acc_main = acc_main + (inc_main + (fm1*(amp_fm1>>depth_fm1))) ;
high_main = (char)(acc_main >> 8) ;
//********************************************************************
// FM synthesis for voice 2
// compute exponential attack and decay of amplitude
// the (time & 0x0ff) slows down the decay computation by 256 times
if ((time & 0x0ff) == 0) begin
amp_fall_main_v2 = amp_fall_main_v2 - (amp_fall_main_v2>>decay_main_v2) ;
rise_phase_main_v2 = rise_phase_main_v2 - (rise_phase_main_v2>>rise_main_v2);
// compute exponential decay of FM depth of modulation
amp_fm1_v2 = amp_fm1_v2 - (amp_fm1_v2>>decay_fm1_v2) ;
end
// form (1-exp(-t/tau)) for the attack phase
amp_rise_main_v2 = max_amp - rise_phase_main_v2;
// product of rise and fall exponentials is the amplitude envelope
amp_main_v2 = (amp_rise_main_v2>>8) * (amp_fall_main_v2>>8) ;
// Init the synth
if (pluck_v2==1) begin
amp_fall_main_v2 = max_amp;
rise_phase_main_v2 = max_amp ;
amp_rise_main_v2 = 0 ;
amp_fm1_v2 = max_amp ;
// phase lock the synth
acc_fm1_v2 = 0 ;
acc_main_v2 = 0;
pluck_v2 = 0;
end
//the FM DDR -- feeds into final DDR
acc_fm1_v2 = acc_fm1_v2 + inc_fm1_v2 ;
high_fm1_v2 = (char)(acc_fm1_v2 >> 8) ;
fm1_v2 = sineTable[high_fm1_v2] ;
//the final output DDR
// phase accum = main_DDR_freq + FM_DDR * (FM amplitude)
acc_main_v2 = acc_main_v2 + (inc_main_v2 + (fm1_v2*(amp_fm1_v2>>depth_fm1_v2))) ;
high_main_v2 = (char)(acc_main_v2 >> 8) ;
//********************************************************************
// FM synthesis for voice 3
// compute exponential attack and decay of amplitude
// the (time & 0x0ff) slows down the decay computation by 256 times
if ((time & 0x0ff) == 0) begin
amp_fall_main_v3 = amp_fall_main_v3 - (amp_fall_main_v3>>decay_main_v3) ;
rise_phase_main_v3 = rise_phase_main_v3 - (rise_phase_main_v3>>rise_main_v3);
// compute exponential decay of FM depth of modulation
amp_fm1_v3 = amp_fm1_v3 - (amp_fm1_v3>>decay_fm1_v3) ;
end
// form (1-exp(-t/tau)) for the attack phase
amp_rise_main_v3 = max_amp - rise_phase_main_v3;
// product of rise and fall exponentials is the amplitude envelope
amp_main_v3 = (amp_rise_main_v3>>8) * (amp_fall_main_v3>>8) ;
// Init the synth
if (pluck_v3==1) begin
amp_fall_main_v3 = max_amp;
rise_phase_main_v3 = max_amp ;
amp_rise_main_v3 = 0 ;
amp_fm1_v3 = max_amp ;
// phase lock the synth
acc_fm1_v3 = 0 ;
acc_main_v3 = 0;
pluck_v3 = 0;
end
//the FM DDR -- feeds into final DDR
acc_fm1_v3 = acc_fm1_v3 + inc_fm1_v3 ;
high_fm1_v3 = (char)(acc_fm1_v3 >> 8) ;
fm1_v3 = sineTable[high_fm1_v3] ;
//the final output DDR
// phase accum = main_DDR_freq + FM_DDR * (FM amplitude)
acc_main_v3 = acc_main_v3 + (inc_main_v3 + (fm1_v3*(amp_fm1_v3>>depth_fm1_v3))) ;
high_main_v3 = (char)(acc_main_v3 >> 8) ;
//********************************************************************
// output the wavefrom sample
// scale amplitude to use only high byte and shift into range
// 0 to 255
OCR0A = 128 + (((amp_main>>8) * (int)sineTable[high_main])>>7)
+ (((amp_main_v2>>8) * (int)sineTable[high_main_v2])>>7)
+ (((amp_main_v3>>8) * (int)sineTable[high_main_v3])>>7) ;
//********************************************************************
// time variable setup
time++; //ticks at 8 KHz
if(time1>0) {time1--;}
if(time3>0) {time3--;}
cycle_count = TCNT2;
TCCR2B = 0;
}
//**********************************************************
//Entry point and task scheduler loop
int main(void){
initialize();
//Instructions for users on the PUTTY
printf("************************************************\n\r");
printf("******Welcome to the Auto-Composing Piano!******\n\r");
printf("****** Please Select Mood and Train Mode ******\n\r");
printf("****** Happy: A, Tender: B ******\n\r");
printf("****** Train: #, Default: 0 ******\n\r");
printf("****** Stop and Reset: * ******\n\r");
printf("************************************************\n\r");
printf("\n\r\n\r");
printf("************************************************\n\r");
printf("****** After training, play two notes ******\n\r");
printf("****** And we will compose for you ******\n\r");
printf("************************************************\n\r");
printf("\n\r\n\r");
while(1) begin
//first have to input a mood and whether he need to train the machine
while ((mood == -1)||(train_mode == -1))
begin
if (time3 == 0) {keypad_debounce();}
if(act_button == buttonA) {mood = happy; act_button = 0;}
else if(act_button == buttonB) {mood = tender; act_button = 0;}
else if(act_button == button_pd) {train_mode = want_train; act_button = 0;}
else if(act_button == button0) {train_mode = not_train; act_button =0;}
end//waiting loop
//debounce the keypad and piano keys every 30ms
if (time3 == 0)
{
keypad_debounce();
piano_debounce();
}
//train
if (train_mode == want_train) {
train(act_piano_index+6);
}
//user chose to stop training
if (act_button == button0) {
train_mode = not_train;
act_button = 0;
train_prob_calc();
}
//stop
if (act_button == button_star) {
act_button = 0;
mood = -1;
train_mode = -1;
piano_input1 = 0;
piano_input2 = 0;
reset = 1;
note_index = 0;
delay_flag = 1;
//stop and reset
for (int i = 0; i<120; i++){
melody[i] = -1;
chord_1[i] = -1;
}//for
//reset probability matrix
for (int i = 0; i<12;i++) {
for (int j = 0; j < 12; j++) {
prob_t[i][j] = prob_t_origin[i][j];
prob_h[i][j] = prob_h_origin[i][j];
}
}
}//if (act_button == button_star)
//paly two notes
while ((train_mode == not_train) && (piano_input1 == 0 || piano_input2 == 0)){
if(time3 == 0) {piano_debounce();}
if(act_piano_index > 0)
begin
if(piano_input1 ==0)
{
//range of piano input is G2(#8 in array) to F4(#30 in array)
//with index starting from 1. Thus need to have a shift from
//act_piano_index to piano_input
piano_input1 = act_piano_index + 6;
act_piano_index = 0;
inc_main = inc_main_arr[piano_input1];
inc_fm1 = inc_fm1_arr[piano_input1];
pluck_v1 = 1;
}
else if(piano_input2 ==0)
{
piano_input2 = act_piano_index + 6;
act_piano_index = 0;
inc_main = inc_main_arr[piano_input2];
inc_fm1 = inc_fm1_arr[piano_input2];
pluck_v1 = 1;
}
//reset = 1;
end
}
//wait a little bit to get the second key played heared more clearly
if((delay_flag == 1)&&(train_mode == not_train))
_delay_ms(100);
//start sequencing
if (reset==1 &&train_mode == not_train)
{
printf("************************************************\n\r");
printf("****** Music loaded, Enjoy! *******\n\r");
printf("************************************************\n\r");
sequencer(piano_input1-1, piano_input2-1, mood);
reset = 0;
}
//play the music
if ((time1 == 0) && (reset == 0) && (train_mode == not_train)){
decay_t = 0;
decay_t1 = 0;
decay_t2 = 0;
//reset time1
time1 = t1;
// play melody and chords
if (melody[note_index] != -1){
inc_main = inc_main_arr[melody[note_index]];
inc_fm1 = inc_fm1_arr[melody[note_index]];
pluck_v1 = 1;
}
if (chord_1[note_index]!=-1){
inc_main_v2 = inc_main_arr[chord_1[note_index]];
inc_fm1_v2 = inc_fm1_arr[chord_1[note_index]];
pluck_v2 = 1;
}
if (note_index<119) note_index++;
else {
note_index = 0;
}
//to avoid unwanted delay
delay_flag = 0;
}//time1 == 0
end
} //end main
void initialize(void){
// make B.3 an output
DDRB = (1<<PINB3) ;
DDRA = 0xf0;
//for welcome message display
cycle_count = 0;
//init the UART -- uart_init() is in uart.c
uart_init();
stdout = stdin = stderr = &uart_str;
//fprintf(stdout,"Starting...\n\r");
// init the sine table
for (int i=0; i<256; i++)
begin
sineTable[i] = (char)(127.0 * sin(6.283*((float)i)/256.0)) ;
end
///////////////////////////////////////////////////
// Sound parameters
///////////////////////////////////////////////////
// Base frequency
// 2^16/8000*freq = 4.096*freq
inc_main = (int)(4.096 * C2i) ;
inc_main_v2 = (int)(4.096 * C2i) ;
inc_main_v3 = (int)(4.096 * C2i) ;
// rise and decay SHIFT factor -- bigger is slower
// 6 implies tau of 64 cycles
// 8 implies tau of 256 cycles
// max value is 8
decay_main = 4 ;//voice1 initialize as piano
decay_main_v2 = 4 ;//voice2 initialize as piano
decay_main_v3 = 4 ;//voice3 initialize as piano
rise_main = 2 ;
rise_main_v2 = 2 ;
rise_main_v3 = 2 ;
//
// FM modulation rate -- also a frequency
inc_fm1 = (int)((4.096 *piano_ratio)* C2i) ;
inc_fm1_v2 = (int)((4.096 *piano_ratio)* C2i);
inc_fm1_v3 = (int)((4.096 *piano_ratio)* C2i) ;
// FM modulation depth SHIFT factor
// bigger factor implies smaller FM!
// useful range is 4 to 15
depth_fm1 = 11 ;
depth_fm1_v2 = 11 ;
depth_fm1_v3 = 11 ;
// decay SHIFT factor -- bigger is slower
// 6 implies tau of 64 cycles
// 8 implies tau of 256 cycles
// max value is 8
decay_fm1 = 6 ;
decay_fm1_v2 = 6 ;
decay_fm1_v3 = 6 ;
////////////////////////////////////////////////////
// init the time counter
time=0;
t1 = 450*16; // controls the timing in the system
time1=t1;
time3=t3;
//for sequencer
accumulator = 0xab5a55aa;//a random number not zero for random number generator
pluck_v1 = 0;//don't play initially
pluck_v2 = 0;//don't play initially
pluck_v3 = 0;//don't play initially
decay_t = 0;
decay_t1 = 0;
decay_t2 = 0;
delay_flag = 1;//delay at first time or after reset
for (int j = 0; j<note_range; j++)
begin
inc_fm1_arr[j] = piano_ratio * inc_main_arr[j];
//depth_fm1_arr[j]
end
//initialize the matrices and notes to play
for (int i = 0; i<120; i++){
melody[i] = -1;
chord_1[i] = -1;
}//for
for (int i = 0;i<12;i++){
for (int j = 0; j<12;j++){
train_prob[i][j] = 0;
raw_prob[i][j] = 0;
prob_t[i][j] = prob_t_origin[i][j];
prob_h[i][j] = prob_h_origin[i][j];
}
}
////////////////////////////////////////////////////
// For Keypad
butnum = 0;
act_button = 0;
PushState = NoPush;
//For piano keyboard
PushState_piano = NoPush;
piano_input1 = 0;
piano_input2 = 0;
mood = -1;
//stop = 1;
note_index = 0;
reset = 1;
train_mode = -1;
train_pre = -1;
////////////////////////////////////////////////////
//timer0 and 1
// timer 0 runs at full rate
TCCR0B = 1 ;
//turn off timer 0 overflow ISR
TIMSK0 = 0 ;
// turn on PWM
// turn on fast PWM and OC0A output
// at full clock rate, toggle OC0A (pin B3)
// 16 microsec per PWM cycle sample time
TCCR0A = (1<<COM0A0) | (1<<COM0A1) | (1<<WGM00) | (1<<WGM01) ;
OCR0A = 128 ; // set PWM to half full scale
// timer 1 ticks at 16000 Hz
OCR1A = 999 ; // 1000 ticks
TIMSK1 = (1<<OCIE1A) ;
TCCR1B = 0x09; //full speed; clear-on-match
TCCR1A = 0x00; //turn off pwm and oc lines
////////////////////////////////////////////////////
//LCDinit();
//LCDclr();
TCNT1 =0;
// turn on all ISRs
sei() ;
}
//helper method to find the max of a row of raw_prob
int max_raw_prob(int row_num){
int temp = 0;
int max_num = 0;
for (int i = 0; i<12;i++){
if (raw_prob[row_num][i]>=max_num) {
max_num = raw_prob[row_num][i];
temp = i;
}
}
return temp;
}
//helper method to find sum of a row of train_prob
int sum_train_prob (int row_num){
int temp = 0;
for (int i = 0; i<12; i++)
{
temp += train_prob[row_num][i];
}
return temp;
}
//helper method to find sum of a row of raw prob
int sum_raw_prob (int row_num){
int temp = 0;
for (int i = 0; i<12; i++)
{
temp += raw_prob[row_num][i];
}
return temp;
}
// set the tone of the music
void set_tone(void){ //pass in note_1 and note_2 index
unsigned char diff;
unsigned char lower;
//change note_1, note_2 to middle C octave
if (piano_input1 < 12) piano_input1 = piano_input1 + 12;
else if (piano_input1 > 24) piano_input1 = piano_input1 - 12;
if (piano_input2 < 12) piano_input2 = piano_input2 + 12;
else if (piano_input2 > 24) piano_input2 = piano_input2 - 12;
//printf("input 1 %d\n\r", piano_input1);
//printf("input 2 %d\n\r", piano_input2);
if (mood == happy) t1 = 200*16;
else if (mood == tender) t1 = 350*16;
//determine tonic
if (piano_input1 >= piano_input2) {
diff = piano_input1 - piano_input2;
lower = piano_input2;
}
else {
diff = piano_input2 - piano_input1;
lower = piano_input1;
}
if (mood == happy){ //use major tones if happy mood
if ((diff == 0) || (diff == 2) || (diff == 4) || (diff == 7) || (diff == 11) || (diff == 12)){
//use the tone where the lower note is C
//C
create_scale(lower, 0);
}
else if ((diff == 3) || (diff == 8)) {
//use the tone where the lower note is E
//A-
create_scale(lower +8, 0);
}
else if ((diff == 5) || (diff == 9)) {
//use the tone where the lower note if G
//F
create_scale(lower +5, 0);
}
else if (diff == 10){
//use the tone where the lower note is D
//B-
create_scale(lower +10, 0);
}
else if (diff == 1) {
//use the tone where the lower note is B
//D-
create_scale(lower +1, 0);
}
else if (diff == 6) {
//use the tone where the lower note is F
//G
create_scale(lower +7, 0);
}
}
else if (mood == tender) { //use minor tones if tender/blue
if ((diff == 0) || (diff == 2) || (diff == 3) || (diff == 7) || (diff == 8) || (diff == 11) || (diff == 12)){
//c
create_scale(lower, 1);
}
else if (diff == 1){
//c#
create_scale(lower +1,1);
}
else if ((diff == 4) || (diff == 9)) {
//a
create_scale(lower +9, 1);
}
else if (diff == 5){
//f
create_scale(lower +5, 1);
}
else if (diff == 6){
//g
create_scale(lower +8, 1);
}
else if (diff == 10){
//b-
create_scale (lower +10, 1);
}
}//mood == tender
}//set_tone
//create the scale based on tone and note played
void create_scale(int8_t c_note, int8_t tonality){
if (tonality == 0) { //major
scale[0] = c_note;
scale[1] = scale[0] + 2;
scale[2] = scale[1] + 2;
scale[3] = scale[2] + 1;
scale[4] = scale[3] + 2;
scale[5] = scale[4] + 2;
scale[6] = scale[5] + 2;
scale[7] = scale[6] + 1;
scale[8] = scale[7] + 2;
scale[9] = scale[8] + 2;
scale[10] = scale[9] + 1;
scale[11] = scale[10] + 2;
}
else if (tonality == 1) { //minor
scale[0] = c_note;
scale[1] = scale[0] + 2;
scale[2] = scale[1] + 2;
scale[3] = scale[2] + 1;
scale[4] = scale[3] + 2;
scale[5] = scale[4] + 2;
scale[6] = scale[5] + 2;
scale[7] = scale[6] + 1;
scale[8] = scale[7] + 2;
scale[9] = scale[8] + 2;
scale[10] = scale[9] + 1;
scale[11] = scale[10] + 2;
}
}
//adjust note based on scale
int8_t scale_pos(int8_t note){
int8_t pos = 0;
if (note > 31) note = note - 12;
else if (note < 12) note = note + 12;
for (int i = 0; i<12; i++){
if (scale[i] == note) pos = i;
}
return pos;
}
//calculate the chord sequence
void set_chord(void){
int8_t rand;
int8_t rand_1;
int8_t note=0;
for (int i = 0; i<120; i++){
rand = rand_shift()%127;
if (rand >= 16 && melody[i]!=-1) { //play
if (melody[i]!=-1) { //play
note = melody[i];
rand_1 = rand_shift() % 127;
if (note < 64 && rand_1 < 60){
chord_1[i] = note -5;
}
else if (note < 100 && rand_1<64) {
chord_1[i] = note -7;
}
//else if ( note < 25 && rand_1 < 96){
//chord_1[i] = note + 12;
//}//else if
else if (note > 15 && mood == tender){
chord_1[i] = note-12;
}
else chord_1[i] = -1;
}//if (rand >= 64 && melody[i]!=-1)
else chord_1[i] = -1;
}
//printf("chord %d\n\r", chord_1[i]);
}//for
}//set_chord
//calclate the melody sequence
void set_melody(void){
//goal is to calculate the determined melody
int8_t rhythm[120];
int8_t rand=0;
int8_t k = 0;
int8_t row = 0;
int8_t col = 0;
int8_t cur_prob[12][12];
int8_t sum = 0;
//choose rhythm
if (mood == happy){
for(int i=0;i<15;i++)
begin
if (i==4 || i==8 || i==12){
for (int j =0; j<8;j++) rhythm[i*8+j] = happy_rhy_4[j];
}
else {
rand = rand_shift()%127;
if (rand < 16){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_1[j];
}
else if (rand < 32){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_2[j];
}
else if (rand < 48){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_3[j];
}
else if (rand < 64){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_4[j];
}
else if (rand < 80){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_5[j];
}
else if (rand < 96){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_6[j];
}
else if (rand < 112){
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_7[j];
}
else{
for (int j = 0; j <8; j++) rhythm[i*8+j] = happy_rhy_8[j];
}
}
end
}
else if (mood == tender){
for(int i=0;i<15;i++)
begin
if (i==4 || i==8 || i==12){
for (int j =0; j<8;j++) rhythm[i*8+j] = happy_rhy_4[j];
}
else {
rand = rand_shift()%127;
if (rand < 16){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_1[j];
}
else if (rand < 32){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_2[j];
}
else if (rand < 48){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_3[j];
}
else if (rand < 64){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_4[j];
}
else if (rand < 80){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_5[j];
}
else if (rand < 96){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_6[j];
}
else if (rand < 112){
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_7[j];
}
else {
for (int j = 0; j <8; j++) rhythm[i*8+j] = blue_rhy_8[j];
}
}
end
}
for (int i = 0; i<120; i++){
for(int i=0;i<12;i++)
begin
for (int j=0;j<12;j++)
begin
if (mood == happy)
cur_prob[i][j] = prob_h[i][j];
else if (mood == tender)
cur_prob[i][j] = prob_t[i][j];
end
end
if (rhythm [i] != 0) {
if (i == 0 ) {
rand = rand_shift() % 127;
if (rand < 64)
melody[i] = piano_input1;
else melody[i] = piano_input2;
}//if(i==0)
else if (i == 1) {
rand = rand_shift() % 127;
if (rand < 64) melody[i] = piano_input1;
else
melody[i] = piano_input2;
}
else if (i == 2){
if (melody[0] == melody[1] && melody[0] == piano_input1) melody[i] = piano_input2;
else if (melody[0] == melody[1] && melody[0] == piano_input2) melody[i] = piano_input1;
else if (melody[0] == piano_input1 && melody[1] == -1) melody[i] = piano_input2;
else if (melody[0] == piano_input2 && melody[1] == -1) melody[i] = piano_input1;
else{
k = i-1;
while (melody[k] == -1) k--;
row = scale_pos(melody[k]);
//find next note
rand = rand_shift() % 127;
sum = 0;
for (int i = 0; i<12; i++)
begin
if((rand <= cur_prob [row][i] + sum)
&&(rand >= sum))
begin
col = i;
break;
end
sum += cur_prob [row][i];
end
melody[i] = scale[col];
//printf("rand %d\n\r", rand);
//printf("sum %d\n\r", sum);
//printf("k %d\n\r", k);
//printf("melody %d\n\r", melody[i]);
}//else
}//else if (i==2)
else {
//for a long note, make it sounds either C,G,or C+
if (rhythm [i+1] == 0 && rhythm [i+2] == 0 && rhythm [i+3] == 0){
for (int m = 0; m<8; m++){
for (int n = 0; n < 8; n++){
if (n == 0 ) cur_prob[m][n] = 40;
else if (n == 4) cur_prob[m][n] = 23;
else if (n == 7) cur_prob[m][n] = 40;
else cur_prob[m][n] = 4;
}
}
}//if
k = i-1;
while (melody[k] == -1) k--;
row = scale_pos(melody[k]);
//find next note
rand = rand_shift() % 127;
sum = 0;
for (int i = 0; i<12; i++)
begin
if((rand < cur_prob [row][i] + sum)
&&(rand > sum))
begin
col = i;
break;
end
sum += cur_prob [row][i];
end
melody[i] = scale[col];
}//else
}//if (rhythm [i] != 0)
else melody[i] = -1;
}//for (int i = 0; i<40; i++)
}//set_melody
//load the sequence of melody and chords
void sequencer(int8_t note_1, int8_t note_2, int8_t mood){
set_tone();
set_melody();
set_chord();
}
//random number generator using shift register, return a random integer
int rand_shift(void){
int8_t bit0, bit1 ;
// implement the shift register
accumulator = accumulator << 1 ;
// & with bit 30 for 'linear feedback shoft register'
bit0 = (accumulator & bit27)>0 ;
// & with bit 27
bit1 = (accumulator & bit30)>0 ;
accumulator = accumulator + (bit0 ^ bit1) ;
// return lower 8 bits
return (accumulator & 0xff) ;
}
//read the piano input
char piano_read(void){
//char piano_index = 0;
piano_reading = ((PINC<<8)|(0xf0 & PIND));// Used PORTD here since keypad needs portA
for(int i=0;i<23;i++)
begin
if (piano_reading == pianotbl[i] )
begin
piano_index = i+1;
end
end
return piano_index;
}
//debouncing the piano keyboard every 30ms
void piano_debounce(void){
time3 = t3; //reset time3
//**********************************************************
//check keypad every 30ms, and get the actual button pressed
switch (PushState_piano)
begin
case NoPush:
if (piano_read()>0) PushState_piano=MaybePush;
else
begin
PushState_piano=NoPush;
end
break;
case MaybePush:
if (piano_read()>0)
begin
PushState_piano=Pushed;
act_piano_index = piano_index;
end
else PushState_piano=NoPush;
break;
case Pushed:
piano_index = 0;
if (piano_read()>0)
begin
PushState_piano=Pushed;
end
else PushState_piano=MaybeNoPush;
break;
case MaybeNoPush:
if (piano_read()>0) PushState_piano=Pushed;
else
begin
PushState_piano=NoPush;
act_piano_index = 0;
end
break;
end//case
}//debounce
void keypad_debounce(void){
time3 = t3; //reset time3
//**********************************************************
//check keypad every 30ms, and get the actual button pressed
switch (PushState)
begin
case NoPush:
if (keypad()>0) PushState=MaybePush;
else
begin
PushState=NoPush;
end
break;
case MaybePush:
if (keypad()>0)
begin
PushState=Pushed;
act_button = butnum;
end
else PushState=NoPush;
break;
case Pushed:
if (keypad()>0)
begin
PushState=Pushed;
end
else PushState=MaybeNoPush;
break;
case MaybeNoPush:
if (keypad()>0) PushState=Pushed;
else
begin
PushState=NoPush;
act_button = 0;
end
break;
end//case
}//debounce
//scan the keypad and return the butnum for which key pressed
char keypad(void){
//get lower nibble
PORTDIR = 0x0f;
PORTDATA = 0xf0;
_delay_us(5);
key = PORTIN;
//get upper nibble
PORTDIR = 0xf0;
PORTDATA = 0x0f;
_delay_us(5);
key = key | PORTIN;
//find matching keycode in keytbl
if (key != 0xff)//0xff means button pushed
begin
for (butnum=0; butnum<maxkeys; butnum++)
begin
if (keytbl[butnum]==key) break;
end
if (butnum==maxkeys) butnum=0;
else butnum++; //adjust by one to make range 1-16
end
else butnum=0;
return butnum;
}
//modifies the probability matrix. all white keys. c3 to c4
void train(int8_t cur_note) {
int8_t row_t = -1;
int8_t col_t = -1;
if (act_piano_index > 0)
begin
if (train_pre == -1){
train_pre = act_piano_index + 6;
act_piano_index = 0;
inc_main = inc_main_arr[train_pre];
inc_fm1 = inc_fm1_arr[train_pre];
pluck_v1 = 1;
}
else {
inc_main = inc_main_arr[cur_note];
inc_fm1 = inc_fm1_arr[cur_note];
act_piano_index = 0;
pluck_v1 = 1;
row_t = train_pos(train_pre);
col_t = train_pos(cur_note);
if (row_t!=-1 && col_t!=-1){
train_prob[row_t][col_t]+=1;
}
train_pre = cur_note;
}
act_piano_index = 0;
end
}
//adjust the note positions in train prob
int8_t train_pos(int8_t note){
if (note < 12) note = note+12;
if (note == 12 || note == 13) return 0;
else if (note == 14 || note == 15) return 1;
else if (note == 16) return 2;
else if (note == 17 || note == 18) return 3;
else if (note == 19 || note == 20) return 4;
else if (note == 21 || note == 22) return 5;
else if (note == 23) return 6;
else if (note == 24 || note == 25) return 7;
else if (note == 26 || note == 27) return 8;
else if (note == 28) return 9;
else if (note == 29 || note == 30) return 10;
else if (note == 31) return 11;
else return -1;
}
//filter the training probability matrix
void train_prob_filter(void){
//filter window size is 7
for (int i = 0; i<12; i++){
if (i == 0) {
for (int j=4; j < 12; j++) train_prob[i][j] = 0;
}
else if (i == 1){
for (int j =5; j<12;j++) train_prob[i][j] = 0;
}
else if (i == 2){
for (int j = 6; j<12;j++ ) train_prob[i][j] = 0;
}
else if (i == 9){
for (int j = 0; j < 6; j++) train_prob[i][j] = 0;
}
else if (i == 10){
for (int j = 0; j<7; j++) train_prob[i][j] = 0;
}
else if (i == 11){
for (int j =0; j<8; j++) train_prob[i][j] = 0;
}
else {
for (int j = 0; j<i-3;j++) train_prob[i][j] = 0;
for (int j = i+4; j<12;j++) train_prob[i][j] = 0;
}
}
}
//update the probability matrix after user's training
void train_prob_calc(void){
//Filter the train prob matrix to keep it centralized
train_prob_filter();
//normalize training matrix
for (int i = 0; i<12; i++){
for (int j = 0; j<12; j++){
if(train_prob[i][j] != 0)
if (((int)((127*train_prob[i][j])/(sum_train_prob(i))))<127)
raw_prob[i][j] = ((int)((127*train_prob[i][j])/(sum_train_prob(i))))+1;//normalizing and ceiling
else raw_prob[i][j] = ((int)((127*train_prob[i][j])/(sum_train_prob(i))));
//except when the entry is zero
else
raw_prob[i][j] = 0;
}
}
//adjusting the sum of each row to 127
for (int i = 0; i<12; i++){
//
if( sum_raw_prob(i) >127)
{raw_prob[i][max_raw_prob(i)] -= (sum_raw_prob(i) - 127);}
//if row is all 0, then play deterministically the next note
else if (sum_raw_prob(i) == 0){
if(i < 10)
raw_prob[i][i+1] = 127;
// if is G4, then play G3
else
raw_prob[i][i-7] = 127;
}
}
for(int i =0; i<12; i++){
for (int j = 0; j<12; j++){
if (mood == happy){
prob_h[i][j] = raw_prob[i][j];
}
else if (mood == tender){
prob_t[i][j] = raw_prob[i][j];
}
}
}
}