This diagram shows the overall logical structure of our program. The central piece to our program is a lock state machine that takes in the user input from a keypad, and then stores the data in the appropriate place based on what phase of the state machine we are in. The keypad input also initiates music generation based on what key is pressed. Based on the phase of the state machine, we also print commands to the LCD for a user interface, as well as control the solenoid lock to open and close the lock box.
We manage our safe using a 6-state lock state machine, with each state corresponding to a certain input from the user, as well as the status of the safe as either locked or unlocked.
State 1: User is setting the password, and entering password for the first time. The safe is unlocked
State 2: User is setting the password, and tries to match the password entered in state 1. If they do not match, we go back to step one
State 3: The passwords from state1 and state2 match, so the user has successfully set the password. The safe is now locked
State 4: User is now trying to unlock the safe. We will check to see if the password entered matches the saved password.
State5: After 3 unsuccessful attempts to enter the password, we now ask the user if they would like to hear a hint. If the user says yes, the hint will play. Either way, we will return two state 4 after this state is complete
State6: The password entered matches the saved password, so now the safe is unlocked. We now proceed to step 3, so that user can relock the safe if he/she desires to.
The keypad comes into play by taking in the input from the user. The user uses the keypad to play music, as well as indicate when the user has completed entering the password. Our keypad works by use of a three-state machine that uses two tasks, task1 and task2, to retrieve the input for each state respectively. Each task scans the keypad for an input, and then saves that output in a variable. We then use a final method writeOnScreen() that will compare the outputs of task1 and task2, and then will store the input appropriately if the outputs of the two tasks match.
The user interface uses an LCD which we hook up to the portC pins on the microcontroller. The user interface will guide the user with instructions on how to set and save a password, as well notify the user about when the safe is locked vs. unlocked. The user interface also gives the user certain options such as whether or not to hear a hint when trying to unlock the password. We save the statements that we will print to the LCD into memory with commands such as “const int8_t LCD_newPassword PROGMEM = "New Password\0";.”
In this project, we the main tradeoff between hardware and software was to decide how to implement the musical password for the safe. We chose to produce ourselves by using the karplus strong algorithm to simulate the plucking of an acoustic guitar. The hardware alternative was to create a microphone circuit that would allow the user to play music using an actual acoustic guitar, and then use digital signal processing to sample the input waveform to store data relating to a password. The benefits of our software method is that there is more simplicity for the user because the user only has to focus on which key to press in order to produce a certain tune; the user does not have to play the correct note on an instrument. This makes the safe more suitable for users without musical ability because they only need to focus on the rhythm component. On the other hand, the hardware alternative allows more complexity for the user to enter the password because an instrument has the ability to play much more than just the nine different notes that we had programmed in our keypad. Ultimately, we were biased towards the software implementation of an acoustic guitar because of a personal interest in developing music ourselves.
Another tradeoff was between the sensitivity of the keypad and the accuracy of the rhythm check. I will go further into the debouncing of the keypad later, but basically we scan the keypad, then wait for 50 milliseconds, then scan the keypad once again. Therefore, if the user presses a key in the waiting time between the two keypad scans, then the rhythm component associated with that key press will be inaccurate up to the maximum value of the waiting time between the two scans. Therefore, the maximum inaccuracy of our rhythm component check is 50 milliseconds. If we increase the waiting time to make our keypad less sensitive so that a user is less likely to double-press a key, the rhythm recording of a musical password becomes less accurate. Luckily, this issue is note particularly troublesome in our device because we use a tolerance of half a second when comparing a saved password to a user’s attempt to match that password, which eliminates the inaccuracy caused by this issue. This issue would become much more prominent if we implemented a feature to allow a user with advanced ability to play their own password on a musical instrument. Also, this issue becomes more prominent if we decreased the tolerance to increase the security of the safe.
The IEEE suggest various strategies and methods for using the karplus strong algorithm, but but no restrictions, so our music generation technique is valid under IEEE standards. All of our power supplies are valid within the limits of the devices that are being powered, and each power supply was produced by another company, so we assume they obey all standards.
Many sound generation techniques, similar to ours, such as FM synthesis, are patented by Stanford University, and licenced to companies such as Yamaha. We used a strategy based on Bruce Land’s webpage DSP for GCC. US patent 4649783, a patent from original algorithm developers Alexander Karplus and and Kevin Strong, is relevant to the basic foundation of the karplus strong algorithm to simulate string sounds.
We used port C for LCD display. Instructions of how to use this music lock are shown are it and users can follow the instruction to set, confirm, change, and enter their password.
Solenoid Drive Circuit
The lock style solenoid is used to lock or lock the door after commands sent by users. This solenoid is basically an electronic lock, designed for a basic cabinet or safe or door. Normally the lock is active so you can't open the door because the solenoid slug is in the way. It does not use any power in this state. When 9-12VDC is applied, the slug pulls in so it doesn't stick out anymore and the door can be opened.
To drive the solenoid, we chose a power transistor and diode and the circuit is shown below.
We used port D to connect the keypad with micro-controller. The keypad we chose is membrane 3X4 matrix keypad because it is more sensitive and comfortable when users play a music on it.
Our method for scanning the our 3x4 keypad consists of first driving the horizontal 4 pins of the keypad with a low voltage, and then reading the vertical 3 pins as an output. Next, we drive the vertical pins low and read the horizontal pins. Now, we OR the output from the vertical read with the horizontal read. If a button is pressed, then the horizontal pin and vertical pin corresponding to that button’s switch will both be read as low. We check for a button press by iterating through an array, with each element in the array corresponding to the 8 bits associated with a button on the keypad. We compare the output read from the keypad with each input, and store the output if there is a match.
We debounce the keypad using a tri-state state machine to scan the keypad, and then scan the kpeyad again after 50 milliseconds, and then check if the output after 50 milliseconds matches the original output. We then move to a third state if the two outputs match, meaning that a keypad is pressed. We use the following tasks to implement this state machine:
This task will read and store an input from the keypad every 70 milliseconds.
This task is called 50 milliseconds after task1(), and will once again read and store the input from the keypad.
This method is called at the end of task2(), and will check if the output from task2() and task1() match. If they match, we then store the input based on which key was pressed, and what phase of the lock state machine that we are in. If the key pressed is one of the numbers 0-9, we play the corresponding sound as described in the “Sound section.”
If the key pressed is enter, this means the user has completed entering an input, and we now move to a new state in the lock state machine based on the input from the user. For instance, if we are in state4 (user attempts to unlock the safe), and the user’s input matches the saved password, then we move back to state3, at which the safe is open, and the user can lock it again if he wishes. We check password matches here by calling a function arraytovalue() that will return a single integer from an array of numbers entered for a password. We additionally call a function rhythmcheck() to compare the rhythm components two arrays to check if they are within a certain tolerance.
If the key pressed id the pound key, this signifies a backspace. A backspace works by erasing the pitch and rhythm component of the previous sound in the current password entry. We then count the rhythm for the next key pressed as the time from when the backspace key was entered to the time that the next key is pressed.
This function will compare the rhythm components of two passwords, and return 1 if the passwords match within a certain tolerance, and return 0 if there is a difference larger than the tolerance. We set the tolerance to 0.800 seconds, but the tolerance can very easily be changed to alter the security level.
The function iterates through an array of password, returning the following value:
∑(array[i] * 10^(index) for 0 <= i < array size
Timer2 is initialized as a 1 millisecond timer that is used to increment 4 timing variables: time1, time2, time3, and time4. Time1 is used to increment a millisecond counter in order to call task1 every 70 milliseconds. Time2 is used in the same style in order to call task2() 50 milliseconds after time1. We call tasks by polling in main, checking until the time variables have reached the appropriate values, and then calling the appropriate tasks, and resetting the time variables.
Time3 and time4 are used to keep track of the rhythm of password entered. Time3 is incremented every millisecond, while time4 is incremented every second. Therefore, the value time4 + .001*time3 is the value of a rhythm component in seconds, to millisecond accuracy. Time3 and time4 are reset every time a key is pressed, so that time3 and time4 can be used to store the timing between key presses. An important point to note is that the rhythm component when the first key is pressed is not used in rhythm checks, so that the user can take an arbitrary amount of time before beginning to enter the password.
We generate sound using the karplus strong algorithm. When a key is pressed, we initialize a buffer with white noise, specifically random numbers between zero and two. We set two pointers equal to the indices of the first two elements of the buffer. The size of the buffer is equal to the size of the sampling frequency divided by the frequency of the pitch.
Each iteration of the timer0, we output a value from the buffer the average of the first two elements of the buffer, and multiplying that output by a decay constant. This output is them used to modify the OCR0A value to change the pwm voltage output on port B.3. We initially set the PWM to half-scale, with the OCR0A value set to 128, and we modify it by taking the output and scale it by bit-shifting by eight before adding it to the OCR0A value. The averaging is for the purpose of providing the effect of a low pass filter, while the decay constant is used to simulate the energy loss in the vibrating of a string. We then remove the element from the buffer that was enqueued in it first. We then input to the buffer the value that we sent out to modify the pwm. Finally we increment the two pointers so that they will point to the oldest two elements in the buffer.
To change pitch, we can change the size of the buffer, because the buffer size is inversely related to the pitch, so decreasing the buffer size will increase the pitch (increase the frequency of the note played). Therefore, relating to the keypad, each of the keys numbered 0-9 corresponds to a different buffer size.
Therefore, based on the fact that our sampling frequency is 8000 Hz, each of the keys 0-9 correspond to the following frequencies (in Hz):
Playing the passcode from memory
We have a feature in our safe that can be activated after the user as incorrectly attempted to enter the password 3 times. At this point the user can hear a hint; this hint is specifically is the password played at a distorted pitch, but at the correct rhythm, so that the user now knows the number of keys entered, as rhythm at which they were entered. Now the user now just has to remember the correct sequence of notes to play at the given rhythm. After the user hears this hint, the user once again can try to unlock the safe.
To activate this feature, the user presses enter when the LCD displays “Hint?”. We then play this hint by using the rhythm components stored in the array associated with the saved password. We using time3 and time4 to count up to the rhythm component associated with each note played in the saved password. Once that time is reached, we reset the white noise (random values) in the buffer in order to simulate the plucking of the string, and then reset time3 and time4 to count up to the next rhythm component. We will also set the buffer size corresponding to the note played, as well as reset the two pointers in the buffer. To play the sound, we modify the OCR0A value using the same technique described earlier.
The most challenging aspect of our program was combining the karplus-string sound generation, the lock-state machine, the debouncing state machine, the password storage and rhythm checks, the user interface, as well as the solenoid lock mechanism. Addtionally, implementing the coed to play a password from memmory at the correct rhythm at specific points in the program was tough as well. To put everything together, we tested out program piece by piece, adding one new aspect at a time.
To test our project, we started by just testing the function of the safe with just a simple password consisting of just a series of numbers. This way, we could see if the password comparison function worked by printing to the LCD to signify which phase of the state machine we were in. This task tested our keypad input by ensuring the appropriate data was begin stored based on which key we entered. Because we encrypt our password in a single number, we could print that number to the LCD to ensure that our password algorithm worked correctly. This also tested our connection to LCD by ensuring we could print the statements correctly for the user interface.
We then added the rhythm component, by using timer2 to keep track of the time between key presses. We tested the rhythm component by using a cell phone timer to keep track of when we were actually pressing keys, and then printing the rhythm component to the LCD. We would then compare the two values, to ensure that we were accurately storing the time between key presses.
Next, we opened a separate program to start creating the sound. We created a random noise waveform and stored it in a buffer, and then created a task that would print each number to LCD to ensure we were creating random noise correctly. We then used the low-pass filter averaging technique and applied a decay constant to create a waveform that would slowly decrease in amplitude as time passed, and translated the waveform to a voltage using a PWM so that we could visualize the waveform to make sure it looked correct.
When it came time to implement the direct digital synthesis for our waveform, we had trouble producing an output as an integer because the waveform involved numbers that were averaged together as well as multiplied by a decay constant that was a fraction of 1, so rounding this number would greatly reduce the accuracy. To solve this issue, we used floating point arithmetic using a multfix algorithm to multiply numbers together. We also used the direct digital synthesis method from Bruce Land’s webpage DSP for GCC.
Once we could play the string sound correctly, we tested our ability to control the timing of string plucking by creating a task that would play a string sound every 5 seconds. We then hooked up a constant string sound to each button 0-9 on our keypad, so that the key presses could correspond to the string sounds. Next, we tested the task of changing string sounds by changing the driving frequency o(the OCR1A value) of the timer1 PWM. This action greatly increased the lag of the program, especially because of the competing CPU memory dedicated to printing to the LCD. We then changed our method of changing string sounds to changing the buffer size, because varying the buffer size varies the frequency of the pitch played.
Problems we encountered
When generating sound: we initially were only able to create a sample waveform using floats instead of integers. This was because we could not average numbers, as well as factor in decay constants, without using floats. We solved this problem by using fixed point arithmetic from Bruce Land’s Webpage on DSP for GCC.
We also had the issue of generating sound by preallocating memory to store the waveforms for playing sound. We were able to generate a wave, but the arraysize was too small to play the sound for a decent period of time. So we switched to dynamically generating sound in the interrupt vector itself.
We additionally had trouble getting the rhythm checks to work well. One issue we fixed was factoring in the time before the first key was pressed, so that the user can wait an arbitrary amount of time before beginning to enter a password.
A further issue was coordinating sound with the keypad presses; We originally tried to change sounds by changing the driving frequency (OCR1A), but then we found that changing the buffer size worked much more smoothly to change the pitch. Changing the driving frequency significantly slowed down the program, and required us to greatly optimize our code, such as printing to the LCD, to keep the program even working decently fast. Once we changed pitches by altering the buffer size, our program speed was great.
We initially created the data for the string sound by preallocating memory to store the waveform. We created arrays in the size of the order of 1000 in order to store waveform data for nine different pitches. However, when playing these pre-created sound waves, the arrays were too short to play a sound longer than a fraction of a second. So next, we tried wrapping around the array data by replaying the wave from the beginning once it had finished playing through. But because the waveform was so short, the sound produced wasn’t a smooth string sound but rather a rough sound as if someone was constantly using their nail to lightly chip at the string. This strategy was taken from a computer science course from Princeton University, which works well when programming to a device with a large amount of memory such as a desktop computer.
Here is the data from a sample password, as well as the sample data from a successful attempt to unlock the passcode, as used in our demonstration video, in which the keys, 1, 2, and 3 are pressed respectively:
Encryption number: 321
Rhythm: [1.621, 0.593, 0.467, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
Successful password match:
Encryption number: 321
Rhythm: [3.564, 0.628, 0.318, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
Notice that the password arrays and therefore the encryption numbers must be exactly the same for two passwords to match, but the rhythm array values can be different as long as they are within the tolerance of 0.800 seconds. Additionally the rhythm array value in first index is not considered in the password comparison; therefore that value does not need to be within a tolerance when comparing two password attempts.
Here are the output waveforms on Port B.3 of the microcontroller for the pitches corresponding to the keypad buttons 1, 2, and 3:
Button 1 waveform:
Button 2 waveform:
Button 3 waveform:
Looking at the waveforms, we can visually observe the decay in the waveform to simulate the loss of energy in plucking a string, as well as the low-pass averaging slowing decreasing the amplitudes of the peaks.
Here are the frequency measurements for buttons 1, 2, and 3:
Button 1 frequency measurement:
Button 2 frequency measurement:
Button 3 frequency measurement:
The speed of execution for generation music is 250cycles/16000000Hz = 1.56E-05 seconds
This is because we set timer0 to run at a full rate (16MHz), and it takes approximately 250 cycles to generate sound because we must reset a white noise table of length 256 to random values to simulate the beginning of a string pluck. The rest of the values are generate dynamically through a low-pass filter averaging technique, so they do not significantly take up CPU time.
The rhythm capture is accurate to about 50-70 milliseconds because of limitations discussed earlier in debouncing the keypad; the exact time when a user presses a key is not the exact time when a keypress is recorded. We tested the exact accuracy by running a stopwatch while entering a password, and then immediately stopping the time once the password was finished being entered. Then, we summed the rhythm component values stored for the password and compared this sum to the stopwatch time. We found that the accuracy was good up to the 2nd decimal place, or to the hundredths of a second. For the password data used in the beginning of the results section, this gives a 1.33% error.
We enforced safety in the design by controlling the lock mechanism of the safe, because leaving a DC voltage applied to the solenoid lock for a long period of time will heat up the solenoid greatly, which could possibly harm the user or damage the circuit. Additionally, we hook up the solenoid in parallel with a shottky diode in order to prevent voltage spikes, since a 12V power supply is powering the solenoid. Finally, we use a power transistor (BUZ73) with a current rating of 7A.
Our device can be used very easily in place of many locks because the only input from the user is on a simple keypad, which is anyway present on must locking devices. Therefore, we increase the complexity of storing passwords without adding more complexity for the user in terms of the user interface. We could additionally replace our LCD interface with a few LEDs to inform the user of the current state of the lock. This device, although it plays music, can be used even by a deaf person to enter passwords. This is because as long as a user can remember a sequence of keys, as well as the rhythm at which the keys were entered, can use the device.
This project met all the expectations we listed in the proposal. However, since this is only a five weeks project, a few aspects could be improved in the future. First of all, the music lock can only play single music notes in the current version of the music lock. If time allowed, the program should be able to scan multiple keys at the same time and play the two waveforms together as a chord, to further increase password complexity. Secondly, the system will be controlled remotely whether using IR or WiFi. Circuits conformed to all the standards.
Additionally, there were a few shortcoming in our project as well. For instance, there are occasions where the safe does not identify a correctly entered password to unlock the safe, giving a false negative. This is due to a glitch in the user interface, going to an unknown state in the lock state machine if the user does not follow the exact instructions of the interface. To improve our product, we could add more protection to prevent the user from entering a glitch.
Code and Design Acknowledgements
We modified the sound synthesis part from Cornell University ECE476 and the LCD library files which were originally designed by Dr. Bruce Land. All the other code was written by Shryan and Shuhan or from their previous lab work. In designing the product, we did not have to sign non-disclosure to get a sample part.
We did not develop a new sound synthesis technique of an acoustic guitar, and we also did not invent the idea of a lockbox, so there is no patent opportunity. However, our unique combination of acoustic guitar sounds on a keypad, combined with the idea of a lockbox, in addition with our password hint feature with sound distortion, may be developed into a publishing opportunity with more work.
Referring to the IEEE Code of Ethics, we tested each of the circuit step by step. Before we put them in the real circuit, we simulated them beforehand. Also, when we soldering the circuit, we tried to make the connection neat and clean so that it is less possible to cause a fire or other accidents. We are honest and realistic in stating claims or estimates based the data we got. Also, we turned down our speaker to the minimal level that we can hear so that the sound did not disturb other people who were also working in the same time. Meanwhile, whenever other groups have troubles and ask us for help, we gave our honest criticism of technical work directly. Also, we accepted such advice from others, too. After successfully completing this project, both of us had a better understanding of what we learned this semester, especially the sound synthesis and using hardware timers to capture important password information accurately.
A. Program Listing
low pass filter circuit
circuit to drive the solenoid
C. Parts List and Costs
|Lock style solenoid
|12v power supply
|9v power supply
|Wires, Resistors, Capacitors
D. Division of Labor
Our group worked very cohesively to complete this project, although Shuhan concentrated on hardware while Shryan concentrated on software. We worked together for all debugging and optimization of our code. Below are some of the areas that we concentrated in:
- Implemented Karplus Strong algorithm
- Implemented rhythm check algorithm
- Researched lock mechanism
- Implemented keypad debouncing
- Created password hint feature
- Designed password encryption algorithm
- Implemented the lock state machine
- Implemented the solenoid lock circuit
- Hooked up LCD and keypad to microcontroller
- Combined music generation with user keypad input
Inspirations for Code/Designs
We would like to thank Bruce Land for all his help to us in designing and developing this project. Additionally, we would like to acknowledge Bruce Land's webpage DSP for GCC, as well as the Cornell VideoNote page for providing us with useful videos. Especially, we would like to thank our lab TA Alex Jaus for his help, guidance, and support throughout the semester. We would also like to thank Adafruit industries for their helpful schematics to help us incorporate their parts into our device.