When brainstorming for project ideas we were very excited about the variety of projects in the past that had to do with audio and music. However, while some of us knew a lot about music others in our group had little to no musical experience, making a music based project difficult to come up with. This led us to the idea of making a project based on learning about music. We had seen some keyboard assist tools in the past, but were intrigued by the capabilities of the PIC to make a play-along keyboard. While some members of the group knew a lot about music, others knew little to nothing so it made sense to design the tool not only from the perspective of a designer, but also from the perspective of someone who wants to use the play-along keyboard to start to learn about music themselves. As we continued to explore we found that adding the sheet music capabilities of our system would be greatly beneficial and we had not seen a system with the play-along LED capabilities of ours and the sheet music before. As we continued to flesh out this idea we realized that our play-along keyboard could be a great and useful tool while providing some fun microcontroller related challenges.
Our system runs on a few subsystems that were implemented as independent units. Firstly, we have our LED system. This system simply uses the PIC32 microcontroller to light up thirty-two LEDs, one for each key. We drove our LEDs with four shift registers to reduce the number of output pins on the PIC32. We wrote serial data from the PIC32 to the LEDs and drove the LEDs with the parallel output of the shift registers. After implementing the LED system hardware we were able to test the LEDs as if it were an LED strip independent of the keyboard. Implementing the system in this way allowed us to have confidence that our LED outputs were all working properly regardless of the rest of the code.
Our second "system" is our input system. We ordered the Alesis Melody 32 keyboard we were going to use for the project without knowing how keypresses were already detected. Our plan was to splice into the existing keypress system and make a key matrix using diodes to detect keypresses. This key matrix would function like the one in Lab 1 for this class, where the PIC32 scans columns and constantly rows. If a keypress is detected we can determine which key by finding the column and the row. With an eight by four key matrix this gives us 32 key inputs using only 12 GPIO pins. Although this was the plan, the plan changed slightly when we opened the keyboard and examined the inside. The keyboard was already using its own key matrix to detect presses and it was clear after using the oscilloscope that their method of keypress detection was similar. However, they used an eight by five key matrix giving us forty key inputs instead of thirty-two. We ended up splicing into their key matrix and reading that into the PIC32 to detect keypresses. Using the pre-existing matrix was a complicated process using the oscilloscope and some other fun debugging to figure out how to read their key matrix, but in the end it saved us a lot of space and we could effectively detect keypresses on the PIC32.
The last high-level system was our software and the program itself. With the other systems working we had to compile them into a single working program. The software was responsible not only for reading inputs and writing LED outputs, but also for parsing songs, guiding the playback loop, and displaying sheet music. We had a lot of discussion before starting any part of the software about the best way to represent piano music and this translated into an easier time implementing the necessary software features once we got there. The specifics of the software will be discussed in the software section of the report, but the general gameplay loop was as follows: display the notes on the LEDs and the highlight the note in the sheet music; scan for the correct keypresses until all required keys are pressed at the same time, parse the next step of the song and repeat.
The first thing we designed for this project was our system for representing songs in a way that gave us all the information we needed for the keyboard. After some discussion and learning about sheet music, music notation, and what we needed we came up with the following system. Much of this system is based on significant previous knowledge of music notation and how we best felt we could represent what we needed in code. First, each step of music we call a quanta, this is our own made up name. Because the keyboard needs to go step by step, we need a unit for each step that doesn't necessarily line up with the already existing rhythmic representation of music. We represent rhythm in sixteenths, this means there are sixteen possible notes in a measure. While some musical rhythms are divided into smaller thirty-second notes, this is uncommon and we decided we would not need to represent that in our software. Each note is represented by a string. The string consists of one letter, A-G; followed by an option # or b, indicating sharp or flat; and ending with the octave number of the note. Examples would be: A4, F#3, or Db5. In software we can deal with how this translates into a song. In our software each quanta contains the following information:
The keyboard can now proceed quanta by quanta, translating each quanta onto the LEDs and into the sheet music.
The major tradeoff we had to consider for this project was in terms of making our own key input matrix, or splicing into the existing one. There were some smaller tradeoffs we made like using shift registers instead of directly driving the LEDs, however these were relatively easy decisions to make. The problem with using the existing key input matrix was that we did not know exactly how it functioned. If we created our own matrix this would add a lot of hardware into the already cramped keyboard, but we would have a very clear idea of how to write the software. We could even use our previous software from reading a smaller keypad from lab 1. We ended up using the existing matrix at the cost of some more complicated software and a long period of reverse engineering their matrix. We had to map their rows and columns to keys, use the oscilloscope on their row and column lines, and write software that polled already existing lines instead of driving the columns like you would do on a normal keyboard matrix. We also had to connect the grounds of our microcontroller and the keyboard. Although we saved much needed space in the keyboard it came at the cost of a complicated debugging process and more complicated software. In the end, this was the major tradeoff we made and we feel that it was the correct choice.