ECE 476 Final Project - ESD Foam Touch Video Game

Introduction

As technological devices become more advanced and a bigger part of our daily lives, the user interface of devices is becoming more important; intuitive and modern interface provides a real means of transferring the pure computational power of a device to the user experience. This project focused on exploring the technology behind the next development in user interface, the human touch. Today, many devices feature this interface from its most basic form, resistive touch, to its most advanced multitouch. An implementation of a extensible and robust touch interface on an 8-bit microcontroller serves as a fundamental introduction behind the techniques and complexities behind the technology. We built a touch interface from Electrostatic Discharge (ESD) foam and tested the possibilities of using this device as either a one-dimensional input with our rendition of a classic game, Brick Blaster, or as a two-dimensional input to a simple on screen drawing program.

High Level

Inspiration
Hardware Software Tradeoffs
Logical Structure
Background Math

From cell phones, music players, to computers, many modern devices are moving towards human touch as an intuitive interface between humans and machines. Imagine the arm-rest of your favorite sofa or couch featuring a touch screen that you could use to change the channel on your television, change the temperature in the room, or control the lighting in the room. This is just one possibility of a touch screen interface. The idea for the project was sparked by the possibilities of the technology, the recent research and developments with the interface, and the obvious transition of this interface being the industry standard. Our project evolved from the idea of developing a multi-touch interface using the open-source library TouchLib, a web cam, and other common household items. However, TouchLib is written in C# and requires very high level computing, so we moved to interfacing a simple 5-wire resistive touch panel. We borrowed a touch panel from a group from last semester, however the device was very erratic and underwhelming.

In order to detect the position of a touch on a resistive touch panel, a voltage gradient must be created on the current orientation—let us say the x-position— of the touch device. This gradient must vary enough to have good resolution of positions. The initial 5-wire touch panel did not provide a good enough gradient, probably due to the age of the device. After several long days of trying to improve the performance of the device, we decided to look to a more productive and hopefully innovative solution. Thanks to Prof. Land, who had ESD foam laying around in the lab and a beautiful mind to think about the possibility of using the foam instead, we decided to move to using the ESD foam.

The ESD foam only provided the hardware solution to the project. We were committed to demonstrating the uses of a touch device as an input method for future lab consideration. We decided to develop a video game that utilized the touch interface as an input, Brick Blaster. The video game, however, only required the use of one-dimensional input device, so we opted to make another program to demonstrate the two-dimensional capabilities of the device. We decided on a drawing program that drew points on a black and white television of a detected touch. The video aspect of this project provided several challenges. The most pertinent challenge was performing calculations for the touch input under the strict requirements of video code. The touch interface also using several interrupts to make the software design easy to implement and understand. With video code, there can only be one interrupt for the timing of sending the signals to the television.

This provided an opportunity to interface two microcontrollers by sending and receiving data between the processors. We use one microcontroller for strictly video generation and video game/drawing logic, while the other handles the calculations for the touch interface. These calculations include Analog to Digital (ADC) conversion for reading the position of the detected touch, calibration of the raw ADC values to television pixels, and determining drags and holds on the touch screen.

In order to read the position of a touch, the x and y position must be determined separately by creating a voltage gradient in the orientation that is being read. To create this gradient, we wove bare wire into the foam on each of the four sides, i.e. top, bottom, left, right. We then connect each of the four sides to output ports on the Mega644 MCU and set the sides to either ground or Vcc to create the gradient. When determining the x position of the touch, the top and bottom wires are set at a high impedance state or disconnected from the MCU, while the left side is set to ground and the right side is set to Vcc. This creates a voltage gradient through the ESD foam from left to right, ground to Vcc. For determining the y position, the left and right side become disconnected or set to a high impedance state, while the top is set at ground and the bottom at Vcc to create the gradient from top to bottom, ground to Vcc. The voltage readings are done by our handy pencil that has a wire as its tip that is connected to an ADC port of the MCU. The pencil MUST be used to touch the foam in order to get a reading. This extra requirement is a result of the extreme simplicity of the touch device.

Configuration	Top	Bottom	Left	Right	Pencil
X-Pos	Hi-Z	Hi-Z	Gnd	Vcc	Hi-Z
Y-Pos	Gnd	Vcc	Hi-Z	Hi-Z	Hi-Z
Standby	Hi-Z	Hi-Z	Gnd	Gnd	Input

The readings for the x and y positions are done one after the other, and the raw data is then calibrated to video pixels using the following linear equations provided by Atmel.

The determination of holds and drags also uses methods described by Atmel in their handy application note for implementing a touch interface on a microcontroller. If a touch ends within 100ms then it is determined to be a simple tap. If it however takes more than 100ms, then it is either a drag if the coordinates stored within that time are different or a hold if the coordinates are the same, within reason.

Design Detail

SOFTWARE

Description of Brick Blaster Video Game

The video game we implemented is called Brick Blaster, designed after the famous game, Brick Breaker. Brick Breaker is a single-player multi-level game, similar to pong. In the game, the player controls a paddle at the bottom of the screen (moves left and right only) to keep a ball from falling off the screen. The ball bounces off the paddle and against the walls of the game in hopes of breaking the bricks that are distributed around the screen.

The video game code was originally written using the video code from Lab 4 for the Lunar Lander video game. Although the video game displayed correctly for a given level, the TV quality suffered when implementing additional levels. Since a TV is controlled by periodic synchronization pulses, a late sync pulse results with poor TV picture quality.

The game was designed to have four levels with each level having a different brick setup. As mentioned earlier, the TV picture quality suffered when implementing additional levels. As a result, a different version of the video code, which uses USART as a pixel shift register, was used instead. With this alternate code, the sync signal is on pin D.0 and the video on pin D.1. Instead of having the main put the MCU to sleep, it uses a separate timer interrupt. It sets the USART into MSIPM mode (SPI master mode) which turns off start/stop bits. This modified version allows us to stream pixels at uniform rate without assembler through the USART transmit-double-buffer. Hence, the code can run faster because frame calculations do not have to be precisely timed to fit between frames.

Wall and Paddle Collision Detection

The main components of the game consist of detecting collisions between the ball and other elements, such as the paddle, bricks, or the walls. The easiest to implement was the wall since the video code for Lab 4 was used as a basis. However, when the ball hits the bottom wall, the player loses a life and the settings are again initialized so the player can release the ball from the paddle. The ball was 4x3 pixels, which displayed as a square ball on the TV. The paddle was 2x21 pixels. All objects on the screen were drawn starting from the top-left corner to maintain easier comparisons.

Next, paddle collision was implemented. Detection occurs when the ball is located a line above the paddle (bottom of ball + 1 = top of paddle) and within a given range in the x-direction. For the left bound, we check the right column of the ball in comparison to the left column of the paddle. For the right bound, we check the left column of the ball with the right column of the paddle. Using this detection, we can check when the ball hits the paddle. Due to some undetected collisions that could not be resolved, an alternate method for paddle collision was used. It checks if the ball is between the line above the paddle and the bottom wall, and uses video_set function to check if pixels underneath the ball are white.

Brick Design and Brick Collision

The video game was designed to have up to 16 bricks (4 rows and columns) in the game. Some computations were made to space the bricks evenly across the screen in the x- and y-directions. Each brick was designed to be 11x25 pixels. Each brick is separated from the next brick by 9 pixels in the y-direction and/or 7 pixels in the x-direction.

Detecting brick collision and erasing the correct brick was probably the most troublesome in the video game. When brick collision was first implemented, it was still using the variable speed ball. Brick collision required case by case checks, which included corner hits, top/bottom hits, and left/right hits. The details of brick collision will only be discussed for the single speed ball and the top-left corner of the ball for simplicity. To check if the top-left corner of the ball hits the bottom-right of the brick, we first check the velocity directions. The only way the ball will bounce back in the reverse x- and y- directions occurs when the ball is traveling to the left (vx < 0) and up (vy < 0). Next, we check if the top-left corner pixel of the ball is white and if its neighbor to the right and neighbor below are black. This check is used for a single corner. The same idea is used for the other three corners. To check left/right side collisions, we check if the top-left pixel of the ball is white and the pixel underneath it is white. To check top/bottom collisions, we check if the top-left pixel of the ball is white and the pixel to its right is white. The same idea applies to the other three sides. For varying ball speed, there was double the amount of comparisons.

Ball and Paddle Movement

Let us now consider the update animation of the ball and paddle. Ball movement is quite simple as the ball has no acceleration. It is simply updated by adding the old position with the velocity to obtain the new position. Since the video game was written separately from the hardware, the paddle movement was first controlled by switch buttons on the STK500. The buttons controlled when the ball was released from the paddle and paddle movement in the left and right directions. Each paddle movement button incremented or decremented the paddle position by one pixel in the x-direction. The paddle was limited by the boundaries of the game.

Bonus features

Bonus features were implemented in the game to make it more interesting. The bonuses added in the game are Longer Paddle (displayed as symbol “B”), Extra Life (displayed as symbol “L”), and Point Doubler, (displayed as symbol “S”). The bricks assigned with these bonuses were pre-determined before each level. The bonus features created a new problem where the ball detected a brick collision when it hit parts of the bonus symbol. To resolve this problem, all the symbols were designed to be 5x3 pixels. Since the symbols are located in the horizontal center of specified bricks, it is safe to perform bonus collisions assuming the symbols as a 5x3 box. Hence, we can easily check the sides of the ball in comparison to the sides of the bonus symbol “box.”

Other Problems and Solutions

At first, the code was implemented to allow a clean bounce in the center that reverses the velocity in the y-direction. To make things more interesting, we allowed the ball to change speeds depending on where it hit the paddle. The closer to the paddle’s edge the ball hit, the faster the ball would bounce back. However, this created a variety of problems as it allowed the ball to travel more than 1 pixel per frame, sometimes 1.75 pixels per frame. With the variable speed, the ball was going so fast it could not detect the paddle hit. This resulted with checking the ball with the top and bottom line of the paddle. With the additional features, it became harder and harder to implement the various speeds. Hence, it was not used in the final version of the game.

One of the most recurring mistakes that had a drastic effect was continually forgetting that positions were fixed numbers so we cannot simply add an integer to it.

Drawing Program Extra
The software for the drawing software used the basic example code provided by Video Generation on the ECE 476 website. This program simply got the calibrated values from the MCU with the touch screen detection, and drew a point at the location that the touch screen detected. The point is transferred using PORT B and C for x and y, respectively.

HARDWARE

The software to determine the position of the touch was adapted heavily from Atmel’s application note on 4-wire and 5-wire touch screens. The code from Atmel was written to support the ATmega88, so the code was updated to work with the ATmega644. These updates included the ports used for ADC and set up of the ADC frequency. The ATmega644 offers the ADC on PINA, while the ATmega88 used PINC. The following description only serves as a paraphrase of the document written by Atmel (link to doc)

Pin Change Interrupt:

The Pin Change Interrupt is used to detect a touch and wake up the AVR from sleep mode. This interrupt will start Timer0.

Timer0, Compare Match A Interrupt:

This interrupt is first used to debounce the touch screen. If this process succeeded, the interrupt will start a new ADC measurement each time it occurs. If no touch activity is detected the timer will increment the SLEEP_COUNTDOWN and finally set the Sleep Flag.

Timer0, Compare Match B Interrupt:

Chronologically before OCMA, this interrupt is only active while the ADC is enabled and measurements are done. It configures the I/O pins and sets the analog input channel.

Timer1, Compare Match A Interrupt:

Independent of Timer0, this is used as a timer for the drag and hold events. This is setup to run every ms, and thus determines whether a drag or hold event occurred if the time between the start of a touch and the end time of the time is greater than or equal to the predetermined time span.

ADC Conversion Complete Interrupt:

In this interrupt the ADC values are read and the filtering is done.

Touch Screen Initialization:

This function handled all the initial setup of the interrupts, ADC converter, and the Timers. This also reset all the values of the flags and puts the touch screen in a standby mode.

Start Measurement:

Enables the ADC pin and starts a conversion.

Stop Measurement:

Disables the ADC and resets the interrupts for Pin Change. It resets the code to prepare for another touch.

Insertion Sort

A private method that helps sort the sample points collected in order to use the median as the true value.

Store Valid Data

After the touch pad is debounced and the sample values determined, this function picks the median value to store as the actual reading for the position.

ADC Measurement

This function handles the main logic for determining the position of a touch. This function checks to make sure that every reading is within a predefined range, then switches between the different configurations to measure and store the values for the X and Y coordinates.

Side	Port
Top	A0
Bottom	A1
Left	A2
Right	A3

Flow diagram

After the touch library determined a value from the ADC, we used a linear calibration method to transform the values to TV pixels. The calibration is also described in Atmel’s document.

Design Results

The speed of measuring the coordinates of a touch is variable depending on the accuracy required. For demonstration purposes, our code calculated about 6 positions every 200ms, using 20 sample points to determine the true location of 1 point. Also, the touch is debounced with a state clock of 16ms. The speed of the touch can be greatly increased by reducing the debounce time and the number of sample points required to determine 1 true location.

After calibration, we found our touch interface to be about 97% percent accurate in the middle of the screen. The inaccuracy increases as we go near the bare wires as the voltage readings are more erratic. Also the human error in calculating the points to be used for calibration is significant because it is hard to see through the foam to get the exact location of the point. The same can be said about the measurement of the points used to calculate the accuracy.

Calibration Analysis
Display (x,y)		Measured (x,y)		Distance	% Diff
14	20	13	20	1	2%
71	179	69	184	5	2%
128	100	127	100	1	0%
50	50	49	50	1	1%
100	100	97	95	6	4%
76	31	70	35	7	5%
				Avg % Diff	2%

The safety of the device is taking into serious consideration as we hide all our bare wires, except the ones woven in the foam, and all ends of the wires are folded over to avoid any injuries from scratches or pokes. The circuitry is hidden under a wooden box to avoid users from playing with the connections and wires.

The usability is very straightforward and intuitive. The main consideration for the project was the usability of the final project. The code used to run the touch interface is easily adaptable and reusable. While the actual hardware is easy to make, maintain, and operate.

Conclusion

Meeting our Expectations

Our final project seemed to meet our expectations in an unexpected way. At first, we strived to develop a video game utilizing a resistive five-wire touch pad for the user interface. Although we did not use the touch screen in the end, we resulted with an ESD foam that performed much better than the touch screen. In addition, we interfaced two MCUs, one for the TV and one for the touch screen.

An improvement for the future would be to implement hardware before starting software designs. A week before we were to present our final projects, our touch pad died. Luckily, Bruce Land had ESD foam that worked in a similar manner as the touch pad. However, we spent a long two weeks trying to work with the faulty touch screen before it died. In less than a week, we had an ESD foam interfaced with a multi-level video game.

One of our hardest obstacles was the fact that both of us were more software-oriented than hardware-oriented. Hence, there was a lot for both of us to learn, such as how to build a prototype board, communicate it with the other Mega644, and interface the ESD foam with the video game.

Conforming to the Applicable Standards

The RS-232 serial protocol was used by the microcontroller to communicate with the PC. It was implemented in the hardware of the Mega644 microcontroller and in the MAX233 level transceiver.

Intellectual Property Considerations

Our touch interface code was taken from Atmel’s code interfacing a Mega88 with a resistive five-wire touch pad.

The basis of our final project’s video game code was taken from the video code written by Shane Pryor and modified by Bruce Land. The design of the video game came from a popular video game, “Brick Breaker.”

We made a custom prototype board for one of our ATmega644s using the schematics provided by Bruce Land.

Samples for our project did not require a signed non-disclosure. We made sure to give them credit in the Costs section.

Ethical Considerations

While planning, designing, and implementing our final project, we made sure to abide by the IEEE Code of Ethics. With so many students working in the lab, we wanted to ensure our safety and others’ safety as well. Ensuring safety included being careful when using the soldering iron. When working with electronics, we notified other students if we may have burned an electrical component. In addition, we assisted other students in the lab if we could. Throughout the project and semester, we received the

consultation of Bruce Land and the teaching assistants. Their advice was very helpful in debugging problems and providing improvements. Conflicts of interest were avoided as much as possible. However, when conflicts did occur, we tried to resolve these issues quickly and respectfully. Also, note bribery would be rejected in all its forms, if offered. The descriptions provided in this report are accurate to the extent of our knowledge. As we progressed through the project, day after day for five weeks, we were able to enhance our understanding of technology, its appropriate application, and potential consequences. Furthermore, we respected Bruce Land, the teaching assistants, and other students from the class, regardless of their gender, race, religion, disability, age, or national origin.

Legal Considerations

Our project did not use potentially hazardous chemicals or substances. There should not be any copyright issues as we do not plan to make a profit from this project design.

Appendix

Cost List

Part	Quantity	Vendor	Cost
STK500	1	476 Lab	$15
Mega644	2	476 Lab	$16
Power supply	2	476 Lab	$10
White board	1	476 Lab	$5
TV	1	Ruke Ufomata	Free
Max233CPP	1	Maxim IC
Custom PC board (old version)	1	476 Lab	$2
Resistors, Wires		476 Lab	Free
Wooden Base	1	Tim Bond (Manager of Civil Infrastructure Complex)	Free
ESD Foam	1	476 Lab	Free
Foam Board	1	Yinan Tang and Nana Wu	Free
RS232 connector for custom PC board	1	476 Lab	$1
Total			$49