Introduction

In our final project, we have built a multi-functional car controlled by a web application on the PC, which allows users to drive the car as well as play music from the interface on the website.

All the requests from the website are collected by JavaScript and the runtime environment Node.js serves to transmit the command via Bluetooth to the car, which enables the car to perform different functions based on the commands:

  • Driving Mode: Move toward one specific direction
  • Speaker Mode: Read .WAV files stored in the SD card and play it through speakers, as well as showcase the amplitude of the audio on the TFT display
  • Piano Mode: Play certain notes of one piano and display which key being pressed on the TFT

In general, we applied what we have learned in this course into this project and integrating many functions together was quite fun and a good learning experience.

High level design

Rationale

Node.js is an open source server environment and it runs single-threaded, non-blocking, and asynchronously, which is very memory efficient. In addition, Node.js has a built-in HTTP module that allows Node.js to transfer data over the Hyper Text Transfer Protocol(HTTP) and it can create an HTTP server that listens to server ports. The SerialPort object of Node.js is also quite useful to open a port and it allows reading or writing to the serial port at any time. So it is feasible to use the SerialPort object as the bridge between JavaScript and the microcontroller. Via Bluetooth transmission, we can send commands to the microcontroller through the website remotely.

Our group decided to pursue this subject as our final project because we can not only leverage what we have learned through this semester, such as generating PWM to run motors by output compare units, playing music through DAC, serial communication through UART, concurrent programming and etc, but also learn a lot of new concepts such as file system, operations on shared SPI channels, waveform audio file format and some modern front-end technologies.

Background

The audio format to be played is .WAV and it stores data in "chunks". There are two sections in the .WAV files: the header and the data. The header is the beginning of a WAV(RIFF) file. The header is used to provide specifications on the file type, sample rate, sample size and bit size of the file, as well as its overall length. The header of a WAV(RIFF) file is 44 bytes long. The data section is the individual samples. An individual sample is the bit size times the number of channels.[1]

In the header section, for example, bytes from 40 to 43 give the actual filesize. So the filesize is given as

(byte43 << 24) | (byte42 << 16) | (byte41 << 8) | byte40

The Timer3 interrupt is enabled to transmit audio data to the DAC based on the sample rate of the music, and it is also easy to get the sample rate of the music given by

( byte24 | (byte25 << 8)) & (0xffff)

As for the data section, for 16-bit PCM files, the data is stored as 16-bit Signed samples, whereas for 8-bit PCM files, the data is stored as 8-bit Unsigned samples[2]. The way the audio data is origanized is shown below. Considering the MCP4822 has 12-bit resolution, the audio samples are converted to 12-bit values along with channel specifications for the DAC. For mono files, both channels play the same value. As for 8-bit samples, the 12-bit data is obtained by bit-shifting left 4 times. For 16-bit samples, the 12-bit data is obtained by combining the 8 bits of the higher byte with the 4 bits of the lower byte.

Figure 1: The actual audio data stored in .WAV files with each block representing one byte

There are also two FIFO(First-In-First-Out) cyclic buffers for both channels and the size of these cyclic buffers is 2000. The cyclic buffers are used to ensure smooth audio playback and I managed two indices ReadIndex and WriteIndex to indicate the positions where to retrieve the audio data sent to the DAC and where to store the new audio data read from the SD card. Besides, considering the yield time for this reading thread is 10 milliseconds, it would be appropriate to read more than 0.01 * sample rate audio data into the cyclic buffer at the first time. And there are also two circumstances shown below to indicate how much data should be read from the SD card. When the ReadIndex is larger than the WriteIndex, the total number of data can be stored in the cyclic buffer is given by Buffer Size - (ReadIndex - WriteIndex). Otherwise, the total number is given as (WriteIndex - ReadIndex).

Figure 2: Two general circumstances to indicate how much data should be read from the SD card

Logic Structure

At a high level, our project can be divided into 6 components shown in Figure 3. The PIC32 is the heart of our project and it is used to control different sections. When the user send requests from the interface on the website, the runtime server Node.js would take over the command and send it to the serial port that is connected to one Bluetooth module. This information is communicated in real-time by the means of Bluetooth transmission, the PIC32 can receive that command through the serial communication with the other Bluetooth module and based on the command transmitted from the PC, it will perform different functions by controling different sections including generating PWM signals to run motors, reading audio data in the SD card, displaying different patterns on the TFT display or sending audio data to the DAC.

For example, when the user sets our mutil-functional car working at driving mode and the user presses 'W' consecutively, the JavaScript will detect 'W' whose keyCode is 87 has been being pressed, then it will send 'W' to the server. At that time the web server Node.js receives the command from the website, it will send the command to the serial port. Through the Bluetooth communication, the PIC32 can finally get the same 'W' command from the other Bluetooth module and it will generate PWM signals to the motor driver, as well as output digital signals as the control signal to control the direction of motors' rotations. On the other hand, if the user sets our car working at speaker mode and the PIC32 receives a command to play music, it will split the information transmitted from the PC into the format of "command" + "data" and then it will use "data" as the file name to open the specific audio file in the SD card, turn on the timer with given sample rate, read audio data into cyclic buffers from the file and write the audio data to the DAC. Also, If the PIC32 receives the command of changing to the piano mode, the PIC32 will draw the piano interfece on the TFT display, wait for certain keys to be pressed on the keyboard, and open the corresponding audio files saved in the SD card as well as output the audio data to the DAC.

Figure 3: High-level block diagram of our final project

Hardware and Software Trade-offs

Both hardware and software significantly contributed to the success of the project. However, we had to make some trade-offs between hardware and software functionalities. For example, one of the design decisions we made is to use HM-10 as the Bluetooth module to implement the wireless communication between PIC32 and PC because it is quite reliable, but the downside of the HM-10 module is that we need to manually enter AT command to bind the two Bluetooth modules every time we turn on the whole circuit, which is not what we expected. But we successfully solved this in software and each time while starting the Node.js web server, it will automatically send the AT command to bind the Bluetooth modules first. Although there are some other Bluetooth modules that can store the bounded addresses and automatically bind with each other, the HM-10 is the most reliable module among all the Bluetooth modules we have ever tried. When we used HC-05, there are even some errors when sending commands to the PIC32 and we tested that by displaying sent commands on the TFT. For example, there was one time when we tested our HC-05 Bluetooth module and we transmit a character 'w' through serial monitor, but the PIC32 receives 't' instead for unknown reasons.

Another design decision is that we decided to take the TFT off the Big Board. In our project, the TFT display and the MicroSD adapter shares the same SPI 1 channel. As shown in the course website, it is easy to discover that the MISO of TFT is left unconnected and the MOSI is connected to RB11 on the PIC. However, due to the hardware limitation of the Peripheral Pin Select(PPS): RB11 can be both SDO1 and SDI1 but RB13 can only be SDO1. So in order to ensure they can work properly sharing the same SPI channel, we took the TFT off the Big Board to make it share the SCK, MOSI, and MISO with the MicroSD adapter and also modified the tft library. At the same time, since the MicroSD adapter shares the same SPI channel with the TFT display, there is not much time left to draw complicated patterns on the TFT display and the SD card adapter will take up the SPI 1 channel for the most time, for which reason it only displayed some simple patterns on the TFT.

Standards

The Bluetooth standard (IEEE 802.15.1) is commonly used with devices that involve Bluetooth transmission of data. This standard states that Bluetooth should be used in a range of 100m with 1-3 Mbps data transmission, a bandwidth of 2.14 GHz, a power consumption from 2.5 to 100 mW and uses that include short range control and/or monitoring [3].There are no patents, copyrights, or trademark considerations relevant to this project.

Program/Hardware Design

Hardware Design

As is shown in Figure 3, there are 5 peripheral components connected with the PIC32 and to accomplish this project, we decided to tackle smaller parts individually and assemble them at the end. The 5 peripheral components are as follows:

  1. Motor diver and motors(Output Compare Unit)
  2. TFT display (SPI1)
  3. MicroSD adapter (SPI1)
  4. DAC, amplifier and speakers (SPI2)
  5. Bluetooth (UART)

Motor diver and motors

Since we have 4 motors in our project, so we need 2 L298N motor drivers and each L298N is connected with 2 motors. We use +7.4V to power the L298Ns. As shown in Figure 5, the mechanism of L298N is that there are two outputs in each side, which are connected to the positive and negative pole of the motors. The Enable A and Enable B are connected to two separate PWM outputs from PIC32, Enable A controls the speed of Motor A, and in the same way, Enable B controls the speed of Motor B. The logic pins Input 1, Input2 take charge of the working state of Motor A and Input3, Input4 control Motor B. The Input3 and Input4 work the same way as Input1 and Input2, as shown in Table 1. In our project, Enable A is connected to RB7, Enable B is connected to RB8.

Figure 5: The schematic of L298N
ENA/ENB Input1/Input3 Input2/Input4 State
0 x x Stop
1 0 0 Brake
1 0 1 Rotate Clockwise
1 1 0 Rotate Counterclockwise
1 1 1 Brake
Table 1: Control mode and motor state for different inputs

TFT display

The display we are using is a 320x240 color LCD with 16-bit color specification. Communication is by SPI 1 channel from the PIC32. The circuit diagram is shown in the Figure 6. The SCK is connected to RB14. The MISO and MOSI are connected to SDI(RB11) and SDO(RB13) of the SPI 1 channel, which shares with MicroSD card adapter. In addition, the Chip Select line is connected to RB1 and the RST is connected to RB2. The D/C port connects to RB0. Vin connect to the Vin on the Big Board. GND is the same GND as PIC32.

Figure 6: The schematic of TFT

MicroSD adapter

The circuit diagram is shown in Figure 7. Similar to the TFT, the MISO and MOSI are connected to SDI(RB11) and SDO(RB13) of the SPI 1 channel. The SCK is connected to RB14. However, the only difference between the connection with the TFT display is that the Chip Select line is connected to RB3 while the chip select line of TFT display is connected to RB1. Vcc connects to the Vin on the Big Board. GND is the same GND as PIC32.

Figure 7: The schematic of MicroSD adapter

DAC, amplifier and speakers

MCP4822 is a dual channel 12-bit Digital-to-Analog converter (DAC) with internal voltage reference. This device offers high accuracy and low power consumption, and is available in various packages. Communication with the device is accomplished via a simple serial interface using SPI protocols.[9] In our project, the communication is accomplished via SPI 2 channel.

PAM8403 is a powerful and small Class D amplifier mounted on a 28 x 20 mm breakout board. It includes a super smooth potentiometer combination volume / on-off knob.

In our project, the PIC32 writes the audio data with channel configurations to the SPI 2 channel, and the analog output signal will be amplified through a PAM8403 amplifier. In the end, it outputs to 2 speakers. The Chip Select line of DAC is connected to RB4, and the SCK is connected to RB15. Vin of DAC is +3.3V. The full connection is shown in Figure 8.

Figure 8: The schematic of DAC and amplifier

Bluetooth

Wireless Communication with PC is accomplished through two HM-10 Bluetooth modules: one for PC and the other one for the PIC32. On the PIC32 side, the communication between the PIC32 and the Bluetooth module is accomplished by serial communication through UART. The RX of the Bluetooth module is connected to the TX of the PIC32, which is RB10, and the TX of the Bluetooth module is connected to RA1, which is the RX of the PIC32. GND is connected to the common ground and VCC is +5V on the Big Board.

At first, we need to bind the two Bluetooth modules. And we used a serial monitor to bind the Bluetooth modules. The steps are listed below:

PC Bluetooth:

  1. Set the arduino serial monitor terminator to 'no line ending'
  2. Set the main module to peripherial mode with 'AT+ROLE0'
  3. Use 'AT+ADDR?' to get BLE address

PIC32 Bluetooth:

  1. Set the main module to manual connect mode using 'AT+IMME1'
  2. Set the main module to Central mode with 'AT+ROLE1'
  3. Use 'AT+CONaddress' to connect. e.g. 'AT+CONA78DB2F1405AA'. If the connection is successful you should see 'OK+CONNA'
  4. Make sure that the LEDs stop blinking
Figure 9: The schematic of Bluetooth

Software Design

The software program has two major parts: the program on the PIC32 and the code on the PC to set up the web server using Node.js and to create one website for interacting with events and sending requests to the server. The program in the PIC32 can be decomposed into several threads. To best discuss the program, it is easiest to elaborate the structure of the program following the program's thread structure.

  1. LED Thread
  2. Serial Communication Thread
  3. Retrieving audio data from MicroSD card adapter and drawing objects on the TFT display Thread
  4. Animation Thread

LED Thread

The yield time for this timer thread is 1 second to toggle one LED. This thread is for debugging purpose.

Serial Communication Thread

This thread keeps scanning the serial buffer, PT_term_buffer that is declared in pt_cornell_1_3_2.h. We used the PT_GetMachineBuffer() method to get the input string from the PT_term_buffer. And according to different working modes of the car, the PT_terminate_time is different. Because in driving and speaker modes, we want the car to stop immediately whenever the user releases the key. So the PT_terminate_time should be short enough that if there is no input characters in the PT_term_buffer, then the car should stop, which means the outputs of the output compare units should be smaller than the threshold. However, in piano mode, the intention is that once you press a key for a while, you probably don't want to play the same note twice. In this case, PT_terminate_time should be large enough to eliminate this kind of situation.

We used switch cases to construct one state machine to decide which case the PIC32 should take. The state machine can be divided into two parts. At first, the car is in default mode, which is driving mode and speaker mode. And also, the car can change into the piano mode or change into driving mode from piano mode anytime when the user wants. If the current state is in default mode, we can control the car move forward by pressing 'w', and move backward by pressing 's', turning left by pressing 'a' or turning right by pressing 'd'. If we click a music to play on our website, the client application will send command to the web server, which will send the same command to the serial port. Through Bluetooth communication, this thread will receive this command and the desired file name to get the related parameters, open timer3 interruption, and set interruption period corresponding to the sample rate of the music. If we pause the music, Playenabled and startread is false. We can also skip songs and go back to the former song then Playenabled and startread will also be false, which means the process of reading audio data from the SD card and sending audio data to the DAC terminates. If the car is in piano mode, the thread will call DrawKeyBoard() to draw the piano keypad on the TFT. Then get the desired audio file name based on which key the user presses and read audio data from the file, as well as send the audio data to the DAC. The flow chart of this thread is shown in Figure 10.

Figure 10: The flow chart of serial thread

Retrieving audio data from MicroSD card adapter and drawing objects on the TFT display Thread

This is an important thread. We use this thread to read in the desired WAV file stored in the SD card. We have a global file pointer fp to point at the desired file. We have a boolean start read to record if we switch to either music mode or piano mode, and a boolean variable Playenabled to help us judge if we are can play our music ( it is the first time to read, the audio buffers are empty, in this case we cannot write to the DAC directly).

If Playenabled is false, we read the given fp file pointer, read in 1400*blockAlign(relate to WAV file itself) bytes, process the bytes we read in, generate the data output to DAC. Different WAV file have different blockAlign. For 8-bit MONO file, blockAlign = 1, for 8-bit stereo file, blockAlign = 2, for 16-bit mono file blockAlign = 2, for 16-bit stereo file blockAlign=4. The 8 or 16 bit here is the sampledepth mentioned in the GetMusicInfo() function. Then we use a four loop to get each byte and do two steps(the step of the for loop should equals to blockAlign since the generation of DAC signal use blockAlign bytes) 1. generate the value to write to DAC, 2. draw rectangulars on TFT screen. If the WAV file is 16-bit, we use the current byte and the next byte, get the lowest 4 bit of the next byte as the highest 4 digit of the value we want to output to DAC and draw on TFT, get the highest 4 bit of the current byte as the lowest 4 digit of the value we want to output to DAC and draw on TFT(If the channels==2, then process 4 bytes at a time, generate data for separate channels, 2 bytes each channel, do the same procedure to generate value for each channel). If the WAV file is 8-bit, just use the lowest 4 bit as the highest 4 bit of the DAC value. If the WAV file is 8-bit and 2 channels, take the next lowest 4 bit as the highest 4 bit of the data of the other channel. After having the value for DAC, we need to write to DAC. Remember to plus the value with 2048. Each time we generate a byte, save it to LeftAudioData and RightAudioData(a part of memory we allocated, unsigned int arrays). Then scale the DAC value to TFT scale, draw a rectangular for each byte. Same procedure happens when the Playenable is true, but this time we will read the rest data, instead of the first 1400 bytes.

If we are in piano mode, the DAC signal generation is the same. The only thing is that we will draw a red straight line pointing to the key we pressed (the drawing of keyboard is done the time we switch to piano mode). Here we show the working flow in Figure 11.

For this part, we got a lot of insight from Syed Tahmid Mahbub’s project “PIC32 Based Audio Player with MicroSD Storage”. We learned how the whole reading process should be executed, and what should we do to import MDD library to our own project. Also, we learned how to process read-in WAV file data. We will list the link at the reference section.

Figure 11: The flow chart of reader thread

Animation Thread

This yield time of this thread is 0.1 second to see if there reaches the end of the playing function. If the boolean value of end is true, which means it reaches the end of the music playing, then it will call TFT_ECE() to display the initial "ECE 4760" pattern on the TFT display.

The website for demo

Results of the Design

Here is a demo video of our project!

We were able to meet our project objective and got the result that’s very responsive and functional. After finishing all parts stated in the final proposal, we also added piano mode into our project. In the end, we have built a responsive and reliable web controled Multi-functional car. As shown in the demo video, we are able to control the car through the website with no error and unobservable delay. Here are some aspects of our program.

  • Speed of execution: there are four protothreads running and three interruptions running in our program. The yield time of the reader thread is 10 milliseconds, in this thread, we make sure that the total number of audio data read from the SD card at the first time is larger than 0.01 times sample rate to eliminate some overhead that might be caused by the concurrency or the timers, which also prevents the overflow of the buffer. The yield time of the animation thread is 100 milliseconds to wait for the end of the playing function and redraw the TFT display. The yield time of the timer thread is 1 seconds for the LED toggling every 1 seconds. The timer2 interruption runs at 1KHz to generate the wave period for the output compare unit. The timer3 interruption follows the sample rate of the music. As for timer1, timer1 is used in the protothreads system
  • Accuracy. As shown in the demo video, the quality of the music is quite good and it can also be proved by generating pure sine wave, which demonstrates that retrieving audio data from the SD card does not interfere with the timer3 interruption. The result is shown in Figure 12. The car is also very responsivecan and can move or stop quickly when we press or release the keys.
  • Safety. We didn’t use any dangerous components. Also, the frequency of the Bluetooth modules are safe for human beings.
  • There might be poential interference from other nearby Bluetooth devices. But we did not find any interference during the whole project.
Figure 12: Pure sine wave indicates that retrieving audio data from the SD card does not interfere with the timer3 interruption

Conclusions

Overall our project met the expectations we had when we began. We perfectly completed what we proposed on the proposal and also completed this piano mode.

For future work, we considered about adding a camera module on this system. But we are not familiar with that and it seems requires high data processing rate, we didn’t make it. Maybe we can work on that and find a solution. Also there are many available modules that can be added to this car: obstacle avoiding, tracking, etc.

As for ethics and safety, we tried our best to follow the IEEE Code of Ethics. The only thing that interacts with people is the PC client terminal. Our car is powered by battery, in the range of safe voltage and current for human. There should be no direct harm to a person through the use of our project. It is worth to consider whether our use of Bluetooth module is legal and safe. For our wireless communications we used a 2.4GHz wireless transceiver module. According to the Federal Communications Commission (47 CFR 15) our module classifies as a Class B transceiver meaning that it is intended for residential use only. Our modules are also limited by FCC regulations to have a peak transmit power that shall not exceed 100 microwatts multiplied by the square root of the emission bandwidth in hertz. According to the datasheet, HM-10 will not work over a power of 4 microwatts. So, HM-10 conforms to its relevant standard. No children (persons who have not attained the legal age for consent) will be asked to use this device thus no provisions are made for soliciting the assent of the children. For more information, users may access the FDA Code of Federal Regulations.

Appendix

Appendix A

The group approves this report for inclusion on the course website.

The group approves the video for inclusion on the course youtube channel.

Appendix B

Commented program listing

The commented code is published in Zihao Xue's public github repository.

Schematic

Here is the full schematic of our system in Figure 13

Figure 13: The full schematic of our system

Cost details

Item Vendor Total Cost
PIC32MX250F128B Lab rental $5
Big Board Lab rental $10
TFT LCD Lab rental $10
L298 *2 Amazon $4.94
HM-10*2 Amazon $9.99
AMS1117 Voltage Regulator Amazon $0.99
Micro SD Card Adapter for Arduino Amazon $1.64
Car Kit (base board, wheels, bolts, motors, batteries, battery container) Amazon $40
PAM8403 Amplifier + 2 Pcs 4ohm, 3watt speakers Amazon $11.99
Wires, boards for solder Lab rental $2.5
Total Cost $97.05

Appendix C

Work Distribution

Zihao Xue

  1. Set up the runtime environment web sever using Node.js, created the website with user-friendly interfaces to control the car with Bootstrap framework, and wrote JavaScript to achieve the communication between the website and the web server
  2. Adaption of MDDFS and TFT libraries
  3. Wrote the protothread for playing music from the SD card
  4. Wrote the protothread for piano mode
  5. Tested different modules and integrated together
  6. Car Assembling

Wangzekun Gao

  1. Bluetooth connection
  2. Set up the UART and tested the communication between microcontroller and the Bluetooth module
  3. Set up the output compare unit and tested motor driver to control movements of the car
  4. Car Assembling

Appendix D

References

[1] Digital Audio - Creating a WAV (RIFF) file

[2] Syed Tahmid Mahbub’s “PIC32 Based Audio Player with MicroSD Storage”

[3] Microchip Libraries for Applications (MLA)

[4] Instructions on Big Board and Protothread 1_3_2

[5] HM-10 Bluetooth Configuration

[6] Website Template, Released for free under the Creative Commons Attribution 3.0 license Template Co.

[7] PIC32MX1xx/2xx datasheet

[8] PIC32 Peripheral Libraries for MPLAB C32 Compiler Peripheral Libraries

[9] MCP4822 Datasheet

Acknowledgments

We would like to thank our course instructor Bruce Land and the TAs for their support during this course.