Guoping Huang (gh75)

Guoping Huang (gh75)

Mark Austin Berger (mab227)

Toni Ivanov (tii2)

ECE 476: Final Project

Handwriting Recognition System

I. Introduction. 1

II. High Level Design. 1

III. Hardware Design. 2

IV. Software Design. 4

V. Results. 8

VI. Conclusions. 11

VII. Appendixes: 12

Final Code. 12

Schematic. 35

Budget 36

Task Division. 36

References. 36

I. Introduction

Simply write; your computer will undersand!

We have designed and implemented a Handwriting Recognition System using a touch screen from a Palm Pilot m125, a black and white TV and a Mega32 microcontroller.

Unfortunately, due to the lack of specifications regarding the built-in LCD on the Palm Pilot, we were unable to reverse engineer the LCD protocol. We were, however, able to understand how the touch screen works after some careful investigations.

This project is highly adaptive. With sophisticated algorithm, it should be able detect any patterns. In our project, however, we choose to use a simple algorithm, Nearest Neighborhood Algorithm, as we have very limited amount of time. Thus far it can only recognize simple characters but it is easily extensible. There is no fundamental difference between recognizing a character and any other kind of patterns using our algorithm.

II. High Level Design

Figure 1. Hardware Block Diagram

Source and Rationale of the Project

Initially we started Automatic Speech Recognition. After a few days of careful considerations, however, we decided that lab is a horrible environment for voice recording and that Mega32 is too slow for full-fledged Fast Fourier Transform algorithm. We came across Palm-PPP, a project from last year when we were looking for new ideas. It is a Simon game based on a Palm m125 touch screen. We thought it would be fantastic idea to do handwriting recognition like Graffti. It requires a touch screen, a novel hardware input device and some smart programming, a perfect combination for a final project.

Theory of Operations and Background Math

There are essential two three parts to this project, data acquisition via touch screen, Recognition Algorithm and Video Generation.

1. Data Acquisition

After reading through the Palm-PPP project, we realized that touch screen was not that hard to use. The device driver, therefore, should be an easy thing to write.However, it is not the case as they stated. As mentioned in Palm-PPP project, the touch screen has four pins, each connected to top, right, bottom, and left side of the screen. It is also correctly stated as a purely analog device that detects position by varying resistance between two pairs of pins (top and bottom, left and right). They claim that the touch screen has very low resolution (and it is not even linear).

‘We determined early on that the touch screen’s analog output did not have a high enough resolution or linear relationship to obtain precise and intricate motions.’

Their game therefore divides the screen to only four blocks, four giant pixels essentially. They only need to detect which of the four blocks a stylus touches on. This is simply not true that you can’t better motions beyond four giant pixels as I have used Palm m125 to stretch arbitrary curves and it works pretty works tracing my stylus movements. Either engineers at Palm use some really crazy non-linear interpolating algorithm to magically compensate the shortcomings of their touch screens or the touch screen must be linear and very easy to deal with. We prefer the later scenario. We also believe that linearity is a safe bet. There must be something that they have done wrong.

Initially, the four pins’ behaviors are described as bizarre at best. For one setting, moving in direction, the voltages at different pins will change simultaneously. There seem not to be any independence between pins corresponding to any direction. In some random scenario, two pins will behave in exactly the same way, able to detect movement in, say, X direction, while any Y direction movement is completely ignored. We ‘swap’ the pin settings for detecting X position and were only overjoyed to find that we were able to isolate Y position as well. We realize that it is possible to isolate the movement in one direction, or another, but not both.

Figure 2. Palm m125 Touch Screen

Figure 3. Palm m125 Touch Screen Connected

Careful investigation of various settings that give us isolated readings of X, Y positions reveals that the touch screen is actually a very simple device that behaves exactly like a potentiometer with a little twist, that it has four pins instead of the usual three. As it turns out, it is essential that it has four pins. You can imagine that with Top connected to Vcc and Bottom connected to GND, left and right can be read as the inputs for Y positions. The voltages at both pins will vary at the same time and be of equal magnitude. When reading X, we need to connect Left to Vcc and Right to GND, TOP and BOT then can serve as inputs.

We also use a pair of amplifier and RC filter with a cutoff of 11Hz to filter out digital noises for each input. This will be discussed in the hardware section.

2. Background Math

There are a lot of choices concerning the algorithm we can use to recognize patterns. Recognition algorithms fall under two categories. We can track the motions of the stylus for feature extraction for each pattern; or we can record positions for feature extraction. We choose the latter since the former will involve a lot more complicated implementations at the software level and will surely require more computational power as offered by our microcontroller.

The mathematical fundamentals for our Nearest Neighbor algorithm are very simple. Imagine our bit map of each pattern lives in N-dimensional space. Each pattern is a vector in that space. We will take 3d space as an example.

Figure 4. 3D Presentation

As you can see, character A is the red vector in our 3d space; B is the yellow vector and W the green. It is reasonable to expect that A is closer than B than it is to W because A appears more similar to B than it is to W. Let A’, the brown vector, be the pattern rewritten by someone else using a stylus on a touch screen. It is closer to A than any other vector, supposedly.

To see how close one vector is to another, we need to find the dot product between two vectors. This would give us information on the angle between two vectors. It is also very easy to do dot product between two vectors. It naturally brings us to the question on how we vectorize each character. This will be explained in the software section.

Logical Structure

Figure 5. Logic Structure in Flowchart Presentation

Figure 5 shows the logical structure of the project. This will explained in more detail in the software section.

Hardware/software tradeoffs

There are no hardware and software tradeoffs in our project because we do not have sections of the project where hardware can be substituted by software or vice versa. For example, to have exact timing, as required by video generation, we have to use hardware interrupt instead of any other kind of software timing scheme.

Except for NTSC standard used in video generation, whose code was provided by Professor Bruce Land, we do not use any standards known to us. The touch screen torn from Palm m125 is only a simple analog device that we believe does not implement any sort of standard. We also do not believe that we violate any patent laws since all the technologies and products we used are strict on public domain.

III. Hardware Design

There are five hardware components. They are Touch screen, Op-amplifier and RC filter, Mega32 Microcontroller, Digital and Analog converter and TV. They are connected exactly in this order Touch screen->Op-amplifier and RC filter->Mega32 Microcontroller->Digital and Analog converter -> TV.

There are only four pins on the touch screen. They are connected to Op-amps and Mega32 as shown by the table below.

PINS	Connections
Left	A.1(after going through filter-amplifer pair) and C.1
Top	A.5(after going through filter-amplifer pair) and C.3
Right	C.5
Bottom	C.7

Table 1. Connections between touch screen and Mega32 and Filter-Amplifier pair

Table 1 shows the connections between the touch screen and Mega32. Notice that PORT A is our Analog to Digital conversion ports and that Left and Top are inputs to Mega32 and they have to be filtered and amplified through our filter and amplifier pair before connecting to A.1 and A.5. Figure 6 shows two pairs of filter-amplifiers.

Figure 6. Filter-Amplifier

Figure 7. Filter-Amplifier Scope View

Our filter is simple RC filter with R = 30kΩ, C = .47μF and a cutoff of 11Hz. Our Op-amp is a standard Op-amp gain of 3.

PORTD is our standard output for video signals. Video generation is extensively documented. PORTD.5 and 6 are connected to a resistive DAC as follows

Figure 8. Resistive DAC used in lab 4

We then use the standard black and white TV provided in the lab to display our results. We could have used a graphical LCD or the LCD built into m125 but we don’t have a graphical LCD around the lab nor do we have specifications regarding the LCD therefore we choose to use our bread and butter black and white TV, which is also the cheapest option.

Figure 9. When things are all hooked up

IV. Software Design

Figure 5. Logic Structure in Flowchart Presentation

Figure 5 is reproduced here for software section because it exactly demonstrates how software portion of this project works. Rectangles represent predefined routines; diamond represents control routine; cylinders represent data structures. As usual, we have an infinite loop as our starting point. As the program executes each loop, it generates one frame with TV Signal Generation Interrupt, provided by Professor Land. During each loop, a control routine runs to see if we still need sampling or we should start recognize patterns. Sampling is always running as long as a flag, ACCEPT, is false. ACCEPT turns true when user tap on a predefined portion of the touch screen to signal the beginning of recognition. There are therefore, several important components of the code, TV signal generation, Sampling, Data Storage, Control, and of course recognition.

Sampling

PINS	READ Y		READ X
Left	Amp-filter -> PINA.1 set to A2D conversion	C.1 set to INPUT MODE = high impedance	Amp-filter -> PINA.1 set to Don’t care	C.1 set to OUTPUT mode = Vcc
Top	Amp-filter -> PINA.5 set to Don’t care	C.3 set to OUTPUT MODE = GND	Amp-filter -> PINA.5 set to A2D conversion	C.3 set to INPUT MODE = high impedance
Right	C.5 = INPUT set to high impedance		C.5 = OUTPUT set to GND
Bottom	C.7 = OUTPUT set to Vcc		C.7 = INPUT set to high impedance

Table 2. Mode Switching for Independent Sampling

Sampling is not exactly hard once we understand how the touch screen works. The function Sample() implements sampling and is called by while loop each frame. Notice that as we explained above, we can only independently either read X or Y but not both. Therefore, we need to switch inputs and outputs in order to get proper reading out of the touch screen. Table 2 specifies each modes for each port in each situation. Sampling divides the touch screen into a 40x40 bit map so digitally we can only represent any writing the screen with 40x40 = 1600 pixels, which is more than enough for our purpose. Sampling calls draw() to actually draw points on TV screen using vidieo_pt(), courtesy of Professor Land, which essentially put one pixel to the massive screen[] array one at a time. However, we can not possible store 40x40 = 1600 pts in our tiny memory, we chose to down sample each direction by a factor of 5, which reduces our resolution to that of 8x8 = 64. Sampling calls writeMap() function to do the down-sample and pixels are therefore stored in map[8], a one-byte array of size 8, with each bit representing each pixel.

Figure 9. Left is 40x40 resolution, Right is our 8x8 representation that is actually stored.

Control() will clear the screen if sample() detects that a user taps on a clear command portion of the touch screen. Control() will turn on recognition routine if sample() detects that a user taps on a command portion of the touch screen. testChars() is the routine that performs recognition algorithm.

Recogniton Algorithm

The basic mathematical theory is explained in the High Level Design section of this report. writeMap() essentially vectorizes 40x40 bitmap into map[8], a one-dimensional array, which can be seen as a long string of zeros and ones if you serialize each byte of the array. testChars() will then go through each character in the library and uses testLine() to perform line by line dot product on each character. The results will be stored in rank[3], which specifies the results of dot product and letter[3], which stores the corresponding character ranked by their results. The following is a example of a vectorized letter E in a 21x21 array.

Figure 10. Letter E vectorized.

Things That Did Not Work:

Everything that we tried worked. There are extra features (described below in the Conclusion section) that we did not have time to implement.

V. Results

Figure 11: Finished Circuit and Recognized Characters

Speed of Execution & Accuracy:

The pictures above show the results of our project. The entire hardware (not including the programming board) of the handwriting recognition system is shown middle. Although the circuitry looks rather simple, getting the touch screen to work constituted the most difficult task of the project. The algorithm was relatively intuitive and straightforward to implement. We were successful in recognizing simple alphabet patterns like the letter C (left). For more complicated patterns (right), the handwriting recognition system was very successful if proper handwriting rules are obeyed (80~90% accuracy with proper training). We have limited success with random writings but are satisfied with the result. It was able to recognize the overall shape of the pattern fairly well. The speed of execution is fast and efficient because there were no flickering and delay. Overall, the results are good and worthy of our time investment.

Safety:

This project is safe.

Usability:

This project is very user friendly. The concept of writing on a platform, whether writing pad or touch screen, is well known and used by people of any age.

VI. Conclusions

Design Analysis:

We expected to be able to interface the palm touch screen with the microcontroller and to process and analyze the user input pattern. This project has met our expectations. We were able to detect the handwriting on the screen in a fairly accurate and efficient manner given the project time constraint. There are features and enhancements that we would have liked to implement if we had more time. For example, it would be interesting to explore other handwriting recognition algorithms and compare the quality and efficiency tradeoffs of the results. In addition, we would have liked to implement a training mode for the system, but we did not have enough time. In conclusion, we were satisfied with our design because it was practical and produced good results.

Standards:

The palm touch screen and stylus were used in this project. NTSC is used in video generation. However, there is no known IEEE standard for the touch screen and stylus.

Intellectual Property:

The video generation code was developed by Professor Bruce Land. We used his code to display the touch screen user input and output the recognition pattern. The code is available in the public domain of the class ece476 website. There was no tampering, reverse-engineering done with the code. The code is not patented and is available for students of ECE 476 without a non-disclosure agreement. There are no patent opportunities for this project.

Ethical Considerations:

This project complies with the IEEE Code of Conduct.

1. To accept responsibility in making decisions consistent with the safety, health and welfare of the public, and to disclose promptly factors that might endanger the public or the environment;

This project is safe because the user only interfaces with the palm touch screen using the stylus.

2. To avoid real or perceived conflicts of interest whenever possible, and to disclose them to affected parties when they do exist;

We understand that the handwriting recognition system has been implemented and embedded in numerous applications before. As a result, we are not interested in pursuing any patent nor do we need to disclose our information to anyone.

3. To be honest and realistic in stating claims or estimates based on available data;

Given the allotted time for this project, we are realistic about the tasks we could accomplish. Our estimates are made using available data and are accurate to the best of our knowledge.

4. To reject bribery in all its forms;

There were no briberies offered.

5. To improve the understanding of technology, its appropriate application, and potential consequences;

This project improves the understanding of touch screen applications used in palm pilots and notebooks by exploring some level of the hardware and software design involved in recognizing handwriting.

6. To maintain and improve our technical competence and to undertake technological tasks for others only if qualified by training or experience, or after full disclosure of pertinent limitations;

After finishing this project, we have improved our understanding and appreciation of the touch screen technology. In addition, we did not attempt to undertake any technological tasks for anyone else during the project.

7. To seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors, and to credit properly the contributions of others;

We could not have completed this project without the indispensable help of Professor Bruce Land. Professor Land soldered the tiny wires that connect to the touch screen, which made it possible for us to interface with the touch screen and complete our project.

8. To treat fairly all persons regardless of such factors as race, religion, gender, disability, age, or national origin;

The handwriting recognition system does not discriminate among any kind of people.

9. To avoid injuring others, their property, reputation, or employment by false or malicious action;

No one was injured in any way during the making of this project.

10. To assist colleagues and co-workers in their professional development and to support them in following this code of ethics.

We were not involved with the professional development of our colleagues. However, if there was a need, we would have gladly assisted them.

Please refer to the IEEE Code of Ethics for more information.

VII. Appendixes

Final Code:

//video gen and sound

//D.5 is sync:1000 ohm + diode to 75 ohm resistor

//D.6 is video:330 ohm + diode to 75 ohm resistor

//B.3 is sound and should have a 10k resistor to gnd

#pragma regalloc- //I allocate the registers myself

#pragma optsize- //optimize for speed

#include <Mega32.h>

#include <stdio.h>

#include <stdlib.h>

#include <math.h>

#include <delay.h>

//cycles = 63.625 * 16 Note NTSC is 63.55

//but this line duration makes each frame exactly 1/60 sec

//which is nice for keeping a realtime clock

#define lineTime 1018

#define begin {

#define end }

#define ScreenTop 30

#define ScreenBot 180

#define N 36

#define xBound 40

#define yBound 40

#define bitMapSize 8

//NOTE that v1 to v8 and i must be in registers!

#pragma regalloc+

char syncON, syncOFF;

int LineCount;

int time;

//animation

unsigned char x, y;

unsigned char wait, prevX, prevY;

char screen[1200], t, ts[10], tss[20];

char cu1[]="CHARACTER";

char cu2[]="RECOGNITION";

char cu3[]=":";

char first[]="1ST";

char second[]="2ND";

char third[]="3RD";

char alt = 1;

unsigned char map[bitMapSize];

unsigned char letter[]={99,99,99};

//Musical note values

//C below middle C to C above middle C

//zeros are rests

flash char notes[] = {239,213,189,179,159,142,126,

120,106,94,90,80,71,63,60,0,0,0,0};

char note, musicT;

//Point plot lookup table

//One bit masks

flash char pos[8]={0x80,0x40,0x20,0x10,0x08,0x04,0x02,0x01};

//define characters to be regonized

flash unsigned char lib[N][bitMapSize]={

// //0

// 0b00000000,

// 0b00111100,

// 0b01000110,

// 0b01001010,

// 0b01010010,

// 0b01100010,

// 0b00111100,

// 0b00000000,

// //N

// 0b00000000,

// 0b01100010,

// 0b01010010,

// 0b01001010,

// 0b01000110,

// 0b01000010,

// 0b00000000,

// //S

// 0b00000000,

// 0b00111100,

// 0b01000010,

// 0b00110000,

// 0b00001100,

// 0b01000010,

// 0b00111100,

// 0b00000000};

//0

0b00000000,

0b00111100,

0b01000110,

0b01001010,

0b01010010,

0b01100010,

0b00111100,

0b00000000,

//1

0b00000000,

0b00010000,

0b00110000,

0b01010000,

0b00010000,

0b01111100,

0b00000000,

//2

0b00000000,

0b00111000,

0b01000100,

0b00000100,

0b00111000,

0b01000000,

0b01111110,

0b00000000,

//3

0b00000000,

0b00111000,

0b01000100,

0b00011000,

0b01000100,

0b00111000,

0b00000000,

//4

0b00000000,

0b00100100,

0b00111100,

0b00000100,

0b00000000,

//5

0b00000000,

0b01111100,

0b01000000,

0b01111000,

0b00000100,

0b01111000,

0b00000000,

//6

0b00000000,

0b00011110,

0b00100000,

0b00111100,

0b00100010,

0b00011100,

0b00000000,

//7

0b00000000,

0b01111100,

0b00000100,

0b00001000,

0b00010000,

0b00100000,

0b01000000,

0b00000000,

//8

0b00000000,

0b00111100,

0b01000010,

0b00111100,

0b01000010,

0b00111100,

0b00000000,

//9

0b00000000,

0b00111000,

0b01000100,

0b00111100,

0b00000100,

0b01111000,

0b00000000,

//A

0b00000000,

0b00011000,

0b00100100,

0b00111100,

0b00100100,

0b00000000,

//B

0b00000000,

0b01111000,

0b00000100,

0b01111000,

0b00000100,

0b01111000,

0b00000000,

//C

0b00000000,

0b00111000,

0b01000100,

0b01000000,

0b01000100,

0b00111000,

0b00000000,

//D

0b00000000,

0b01110000,

0b01001000,

0b01110000,

0b00000000,

//E

0b00000000,

0b01111100,

0b01000000,

0b01100000,

0b01000000,

0b01111100,

0b00000000,

//F

0b00000000,

0b01111100,

0b01000000,

0b01110000,

0b01000000,

0b00000000,

//G

0b00000000,

0b00011000,

0b00100100,

0b00100000,

0b00101100,

0b00100100,

0b00011000,

0b00000000,

//H

0b00000000,

0b01000100,

0b01111100,

0b01000100,

0b00000000,

//I

0b00000000,

0b00111110,

0b00001000,

0b00111110,

0b00000000,

//J

0b00000000,

0b01111100,

0b00010000,

0b01010000,

0b00100000,

0b00000000,

//K

0b00000000,

0b00100010,

0b00100100,

0b00111000,

0b00100100,

0b00100010,

0b00000000,

//L

0b00000000,

0b01000000,

0b01111100,

0b00000000,

//M

0b00000000,

0b01000100,

0b01101100,

0b01010100,

0b01000100,

0b00000100,

//N

0b00000000,

0b01100010,

0b01010010,

0b01001010,

0b01000110,

0b01000010,

0b00000000,

//O

0b00000000,

0b00011100,

0b00100010,

0b00011100,

0b00000000,

//P

0b00000000,

0b00111100,

0b00100010,

0b00111100,

0b00100000,

0b00000000,

//Q

0b00000000,

0b00111000,

0b01000100,

0b01010100,

0b01001100,

0b00111000,

0b00000000,

//R

0b00000000,

0b01111000,

0b01000100,

0b01111000,

0b01010000,

0b01001000,

0b01000100,

0b00000000,

//S

0b00000000,

0b00111100,

0b01000010,

0b00110000,

0b00001100,

0b01000010,

0b00111100,

0b00000000,

//T

0b00000000,

0b00111110,

0b00001000,

0b00000000,

//U

0b00000000,

0b00100100,

0b00011000,

0b00000000,

//V

0b00000000,

0b01000100,

0b00101000,

0b00010000,

0b00000000,

//W

0b00000000,

0b00100010,

0b00101010,

0b00110110,

0b00100010,

0b00000000,

//X

0b00000000,

0b01000010,

0b00100100,

0b00011000,

0b00100100,

0b01000010,

0b00000000,

//Y

0b00000000,

0b01000100,

0b00101000,

0b00010000,

0b00000000,

//Z

0b00000000,

0b01111110,

0b00000100,

0b00001000,

0b00010000,

0b00100000,

0b01111110,

0b00000000};

//define some character bitmaps

//5x7 characters

flash char bitmap[38][7]={

//0

0b01110000,

0b10001000,

0b10011000,

0b10101000,

0b11001000,

0b10001000,

0b01110000,

//1

0b00100000,

0b01100000,

0b00100000,

0b01110000,

//2

0b01110000,

0b10001000,

0b00001000,

0b00010000,

0b00100000,

0b01000000,

0b11111000,

//3

0b11111000,

0b00010000,

0b00100000,

0b00010000,

0b00001000,

0b10001000,

0b01110000,

//4

0b00010000,

0b00110000,

0b01010000,

0b10010000,

0b11111000,

0b00010000,

//5

0b11111000,

0b10000000,

0b11110000,

0b00001000,

0b10001000,

0b01110000,

//6

0b01000000,

0b10000000,

0b11110000,

0b10001000,

0b01110000,

//7

0b11111000,

0b00001000,

0b00010000,

0b00100000,

0b01000000,

0b10000000,

//8

0b01110000,

0b10001000,

0b01110000,

0b10001000,

0b01110000,

//9

0b01110000,

0b10001000,

0b01111000,

0b00001000,

0b00010000,

//A

0b01110000,

0b10001000,

0b11111000,

0b10001000,

//B

0b11110000,

0b10001000,

0b11110000,

0b10001000,

0b11110000,

//C

0b01110000,

0b10001000,

0b10000000,

0b10001000,

0b01110000,

//D

0b11110000,

0b10001000,

0b11110000,

//E

0b11111000,

0b10000000,

0b11111000,

0b10000000,

0b11111000,

//F

0b11111000,

0b10000000,

0b11111000,

0b10000000,

//G

0b01110000,

0b10001000,

0b10000000,

0b10011000,

0b10001000,

0b01110000,

//H

0b10001000,

0b11111000,

0b10001000,

//I

0b01110000,

0b00100000,

0b01110000,

//J

0b00111000,

0b00010000,

0b10010000,

0b01100000,

//K

0b10001000,

0b10010000,

0b10100000,

0b11000000,

0b10100000,

0b10010000,

0b10001000,

//L

0b10000000,

0b11111000,

//M

0b10001000,

0b11011000,

0b10101000,

0b10001000,

//N

0b10001000,

0b11001000,

0b10101000,

0b10011000,

0b10001000,

//O

0b01110000,

0b10001000,

0b01110000,

//P

0b11110000,

0b10001000,

0b11110000,

0b10000000,

//Q

0b01110000,

0b10001000,

0b10101000,

0b10010000,

0b01101000,

//R

0b11110000,

0b10001000,

0b11110000,

0b10100000,

0b10010000,

0b10001000,

//S

0b01111000,

0b10000000,

0b01110000,

0b00001000,

0b11110000,

//T

0b11111000,

0b00100000,

//U

0b10001000,

0b01110000,

//V

0b10001000,

0b01010000,

0b00100000,

//W

0b10001000,

0b10101000,

0b01010000,

//X

0b10001000,

0b01010000,

0b00100000,

0b01010000,

0b10001000,

//Y

0b10001000,

0b01010000,

0b00100000,

//Z

0b11111000,

0b00001000,

0b00010000,

0b00100000,

0b01000000,

0b10000000,

0b11111000,

//figure1

0b01110000,

0b00100000,

0b01110000,

0b10101000,

0b00100000,

0b01010000,

0b10001000,

//figure2

0b01110000,

0b10101000,

0b01110000,

0b00100000,

0b01010000,

0b10001000};

//================================

//3x5 font numbers, then letters

//packed two per definition for fast

//copy to the screen at x-position divisible by 4

flash char smallbitmap[39][5]={

//0

0b11101110,

0b10101010,

0b11101110,

//1

0b01000100,

0b11001100,

0b01000100,

0b11101110,

//2

0b11101110,

0b00100010,

0b11101110,

0b10001000,

0b11101110,

//3

0b11101110,

0b00100010,

0b11101110,

0b00100010,

0b11101110,

//4

0b10101010,

0b11101110,

0b00100010,

//5

0b11101110,

0b10001000,

0b11101110,

0b00100010,

0b11101110,

//6

0b11001100,

0b10001000,

0b11101110,

0b10101010,

0b11101110,

//7

0b11101110,

0b00100010,

0b01000100,

0b10001000,

//8

0b11101110,

0b10101010,

0b11101110,

0b10101010,

0b11101110,

//9

0b11101110,

0b10101010,

0b11101110,

0b00100010,

0b01100110,

//:

0b00000000,

0b01000100,

0b10101010,

0b01000100,

0b00000000,

//=

0b00000000,

0b11101110,

0b00000000,

0b11101110,

0b00000000,

//blank

0b00000000,

//A

0b11101110,

0b10101010,

0b11101110,

0b10101010,

//B

0b11001100,

0b10101010,

0b11101110,

0b10101010,

0b11001100,

//C

0b11101110,

0b10001000,

0b11101110,

//D

0b11001100,

0b10101010,

0b11001100,

//E

0b11101110,

0b10001000,

0b11101110,

0b10001000,

0b11101110,

//F

0b11101110,

0b10001000,

0b11101110,

0b10001000,

//G

0b11101110,

0b10001000,

0b10101010,

0b11101110,

//H

0b10101010,

0b11101110,

0b10101010,

//I

0b11101110,

0b01000100,

0b11101110,

//J

0b00100010,

0b10101010,

0b11101110,

//K

0b10001000,

0b10101010,

0b11001100,

0b10101010,

//L

0b10001000,

0b11101110,

//M

0b10101010,

0b11101110,

0b10101010,

//N

0b10101010,

0b11101110,

0b10101010,

//O

0b01000100,

0b10101010,

0b01000100,

//P

0b11101110,

0b10101010,

0b11101110,

0b10001000,

//Q

0b01000100,

0b10101010,

0b11101110,

0b01100110,

//R

0b11101110,

0b10101010,

0b11001100,

0b11101110,

0b10101010,

//S

0b11101110,

0b10001000,

0b11101110,

0b00100010,

0b11101110,

//T

0b11101110,

0b01000100,

//U

0b10101010,

0b11101110,

//V

0b10101010,

0b01000100,

//W

0b10101010,

0b11101110,

0b10101010,

//X

0b00000000,

0b10101010,

0b01000100,

0b10101010,

//Y

0b10101010,

0b01000100,

//Z

0b11101110,

0b00100010,

0b01000100,

0b10001000,

0b11101110

};

//=================================//This is the sync generator and raster

generator. It MUST be entered from

//sleep mode to get accurate timing of the sync pulses

#pragma warn-

interrupt [TIM1_COMPA] void t1_cmpA(void)

begin

//start the Horizontal sync pulse

PORTD = syncON;

//update the curent scanline number

LineCount ++ ;

//begin inverted (Vertical) synch after line 247

if (LineCount==248)

begin

syncON = 0b00100000;

syncOFF = 0;

end

//back to regular sync after line 250

if (LineCount==251)

begin

syncON = 0;

syncOFF = 0b00100000;

end

//start new frame after line 262

if (LineCount==263)

begin

LineCount = 1;

end

delay_us(2); //adjust to make 5 us pulses

//end sync pulse

PORTD = syncOFF;

if (LineCount<ScreenBot && LineCount>=ScreenTop)

begin

//compute byte index for beginning of the next line

//left-shift 4 would be individual lines

// <<3 means line-double the pixels

//The 0xfff8 truncates the odd line bit

//i=(LineCount-ScreenTop)<<3 & 0xfff8; //

#asm

push r16

lds r12, _LineCount

lds r13, _Linecount+1

ldi r16, 30

sub r12, r16

ldi r16,0

sbc r13, r16

lsl r12

rol r13

lsl r12

rol r13

lsl r12

rol r13

mov r16,r12

andi r16,0xf0

mov r12,r16

pop r16

#endasm

//load 16 registers with screen info

#asm

push r14

push r15

push r16

push r17

push r18

push r19

push r26

push r27

ldi r26,low(_screen) ;base address of screen

ldi r27,high(_screen)

add r26,r12 ;offset into screen (add i)

adc r27,r13

ld r4,x+ ;load 16 registers and inc pointer

ld r5,x+

ld r6,x+

ld r7,x+

ld r8,x+

ld r9,x+

ld r10,x+

ld r11,x+

ld r12,x+

ld r13,x+

ld r14,x+

ld r15,x+

ld r16,x+

ld r17,x+

ld r18,x+

ld r19,x

pop r27

pop r26

#endasm

delay_us(4); //adjust to center image on screen

//blast 16 bytes to the screen

#asm

;but first a macro to make the code shorter

;the macro takes a register number as a parameter

;and dumps its bits serially to portD.6

;the nop can be eliminated to make the display narrower

.macro videobits ;regnum

BST @0,7

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,6

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,5

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,4

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,3

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,2

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,1

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

BST @0,0

IN R30,0x12

BLD R30,6

nop

OUT 0x12,R30

.endm

videobits r4 ;video line -- byte 1

videobits r5 ;byte 2

videobits r6 ;byte 3

videobits r7 ;byte 4

videobits r8 ;byte 5

videobits r9 ;byte 6

videobits r10 ;byte 7

videobits r11 ;byte 8

videobits r12 ;byte 9

videobits r13 ;byte 10

videobits r14 ;byte 11

videobits r15 ;byte 12

videobits r16 ;byte 13

videobits r17 ;byte 14

videobits r18 ;byte 15

videobits r19 ;byte 16

clt ;clear video after the last pixel on the line

IN R30,0x12

BLD R30,6

OUT 0x12,R30

pop r19

pop r18

pop r17

pop r16

pop r15

pop r14

#endasm

end

#pragma warn+

//=================================//plot one point

//at x,y with color 1=white 0=black 2=invert

#pragma warn-

void video_pt(char x, char y, char c)

begin

#asm

; i=(x>>3) + ((int)y<<4) ; the byte with the pixel in it

push r16

ldd r30,y+2 ;get x

lsr r30

lsr r30 ;divide x by 8

ldd r12,y+1 ;get y

lsl r12 ;mult y by 16

clr r13

lsl r12

rol r13

lsl r12

rol r13

lsl r12

rol r13

add r12, r30 ;add in x/8

;v2 = screen[i]; r5

;v3 = pos[x & 7]; r6

;v4 = c r7

ldi r30,low(_screen)

ldi r31,high(_screen)

add r30, r12

adc r31, r13

ld r5,Z ;get screen byte

ldd r26,y+2 ;get x

ldi r27,0

andi r26,0x07 ;form x & 7

ldi r30,low(_pos*2)

ldi r31,high(_pos*2)

add r30,r26

adc r31,r27

lpm r6,Z

ld r16,y ;get c

;if (v4==1) screen[i] = v2 | v3 ;

;if (v4==0) screen[i] = v2 & ~v3;

;if (v4==2) screen[i] = v2 ^ v3 ;

cpi r16,1

brne tst0

or r5,r6

tst0:

cpi r16,0

brne tst2

com r6

and r5,r6

tst2:

cpi r16,2

brne writescrn

eor r5,r6

writescrn:

ldi r30,low(_screen)

ldi r31,high(_screen)

add r30, r12

adc r31, r13

st Z, r5 ;write the byte back to the screen

pop r16

#endasm

end

#pragma warn+

//=================================// put a big character on the screen

// c is index into bitmap

void video_putchar(char x, char y, char c)

begin

v7 = x;

for (v6=0;v6<7;v6++)

begin

v1 = bitmap[c][v6];

v8 = y+v6;

video_pt(v7, v8, (v1 & 0x80)==0x80);

video_pt(v7+1, v8, (v1 & 0x40)==0x40);

video_pt(v7+2, v8, (v1 & 0x20)==0x20);

video_pt(v7+3, v8, (v1 & 0x10)==0x10);

video_pt(v7+4, v8, (v1 & 0x08)==0x08);

end

//=================================// put a string of big characters on the screen

void video_puts(char x, char y, char *str)

begin

char i ;

for (i=0; str[i]!=0; i++)

begin

if (str[i]>=0x30 && str[i]<=0x3a)

video_putchar(x,y,str[i]-0x30);

else video_putchar(x,y,str[i]-0x40+9);

x = x+6;

end

//=================================// put a small character on the screen

// x-cood must be on divisible by 4

// c is index into bitmap

void video_smallchar(char x, char y, char c)

begin

char mask;

i=((int)x>>3) + ((int)y<<4) ;

if (x == (x & 0xf8)) mask = 0x0f; //f8

else mask = 0xf0;

screen[i] = (screen[i] & mask) | (smallbitmap[c][0] & ~mask);

screen[i+16] = (screen[i+16] & mask) | (smallbitmap[c][1] & ~mask);

screen[i+32] = (screen[i+32] & mask) | (smallbitmap[c][2] & ~mask);

screen[i+48] = (screen[i+48] & mask) | (smallbitmap[c][3] & ~mask);

screen[i+64] = (screen[i+64] & mask) | (smallbitmap[c][4] & ~mask);

end

//=================================// put a string of small characters on the screen

// x-cood must be on divisible by 4

void video_putsmalls(char x, char y, char *str)

begin

char i ;

for (i=0; str[i]!=0; i++)

begin

if (str[i]>=0x30 && str[i]<=0x3a)

video_smallchar(x,y,str[i]-0x30);

else video_smallchar(x,y,str[i]-0x40+12);

x = x+4;

end

//=================================//plot a line

//at x1,y1 to x2,y2 with color 1=white 0=black 2=invert

//NOTE: this function requires signed chars

//Code is from David Rodgers,

//"Procedural Elements of Computer Graphics",1985

void video_line(char x1, char y1, char x2, char y2, char c)

begin

int e;

signed char dx,dy,j, temp;

signed char s1,s2, xchange;

signed char x,y;

x = x1;

y = y1;

dx = cabs(x2-x1);

dy = cabs(y2-y1);

s1 = csign(x2-x1);

s2 = csign(y2-y1);

xchange = 0;

if (dy>dx)

begin

temp = dx;

dx = dy;

dy = temp;

xchange = 1;

end

e = ((int)dy<<1) - dx;

for (j=0; j<=dx; j++)

begin

video_pt(x,y,c) ;

if (e>=0)

begin

if (xchange==1) x = x + s1;

else y = y + s2;

e = e - ((int)dx<<1);

end

if (xchange==1) y = y + s2;

else x = x + s1;

e = e + ((int)dy<<1);

end

//=================================//return the value of one point

//at x,y with color nonzero=white 0=black

char video_set(char x, char y)

begin

//The following construction

//detects exactly one bit at the x,y location

i=((int)x>>3) + ((int)y<<4) ;

return ( screen[i] & 1<<(7-(x & 0x7)));

end

// This the sampling portion of the code, our touch screen device driver. This will provide a interface between //hardware and software.

unsigned char sample(char axis)

{

unsigned char position, sample;

if(axis == 0) //scaling x

begin

position = (unsigned char)((float)ADCH/256*xBound);

DDRC = 0b10001000; //Set PORTC.7 and PORTC.3 to outputs

PORTC = 0b10000000; //pull everything to GND except PORTC.7 for reading Y

ADMUX = 0b00100001;

end

else //scaling y

begin

sample = ADCH;

if(sample <= 180)

position = (unsigned char)((float)sample/180*yBound);

else

position = (unsigned char)((float)sample/256*(3*yBound/2));

DDRC = 0b00100010; //Set PORTC.1 and PORTC.5 to outputs

PORTC = 0b00000010; //pull everything to GND except PORTC.1 for reading X

ADMUX = 0b00100101;

end

//enable ADC and set prescaler to 1/128*16MHz=125,000

//and clear interupt enable

//and start a conversion

ADCSR.6=1;

return position;

}

void writeMap(unsigned char h, unsigned char v)

// This routine maps 40x40 bitmap to 8x8 bitmap for storage in map

{

unsigned char byte, b,mask, sel_X, sel_Y;

unsigned char sts[10];

byte = v/5; //computing byte address in map[8]

b = h/5; //computing b address in map[byte]

sprintf(sts,"X%03dY%03d",b,byte);

video_putsmalls(64,70,sts); //Displaying 8x8 map on right side of screen

switch(b)

begin

case 7:

mask = 0b00000001;

break;

case 6:

mask = 0b00000010;

break;

case 5:

mask = 0b00000100;

break;

case 4:

mask = 0b00001000;

break;

case 3:

mask = 0b00010000;

break;

case 2:

mask = 0b00100000;

break;

case 1:

mask = 0b01000000;

break;

case 0:

mask = 0b10000000;

break;

end

map[byte] = map[byte]|mask; //writing to map[byte]

sel_X = 52+4*b;

sel_Y = 12+4*byte;

video_putsmalls(sel_X, sel_Y,cu3);

}

void draw() //drawing routine that handles all the graphics stuff on screen

{

unsigned char q,result_posY;

char tmp[10];

//update the positions

sprintf(ts,"X%03dY%03d",x,y);

video_putsmalls(8,70,ts);

for(q=0;q<3;q++)

//This is a mapping function for displaying correctly recognized character on right side of the screen

begin

if(letter[q] != 99)

begin

result_posY = 22+20*q;

switch(letter[q])

begin

// case 0: tmp[0] = '0'; tmp[1] = 0; break;

// case 1: tmp[0] = 'N'; tmp[1] = 0; break;

// case 2: tmp[0] = 'S'; tmp[1] = 0; break;

case 0: tmp[0] = '0'; tmp[1] = 0; break;

case 1: tmp[0] = '1'; tmp[1] = 0; break;

case 2: tmp[0] = '2'; tmp[1] = 0; break;

case 3: tmp[0] = '3'; tmp[1] = 0; break;

case 4: tmp[0] = '4'; tmp[1] = 0; break;

case 5: tmp[0] = '5'; tmp[1] = 0; break;

case 6: tmp[0] = '6'; tmp[1] = 0; break;

case 7: tmp[0] = '7'; tmp[1] = 0; break;

case 8: tmp[0] = '8'; tmp[1] = 0; break;

case 9: tmp[0] = '9'; tmp[1] = 0; break;

case 10: tmp[0] = 'A'; tmp[1] = 0; break;

case 11: tmp[0] = 'B'; tmp[1] = 0; break;

case 12: tmp[0] = 'C'; tmp[1] = 0; break;

case 13: tmp[0] = 'D'; tmp[1] = 0; break;

case 14: tmp[0] = 'E'; tmp[1] = 0; break;

case 15: tmp[0] = 'F'; tmp[1] = 0; break;

case 16: tmp[0] = 'G'; tmp[1] = 0; break;

case 17: tmp[0] = 'H'; tmp[1] = 0; break;

case 18: tmp[0] = 'I'; tmp[1] = 0; break;

case 19: tmp[0] = 'J'; tmp[1] = 0; break;

case 20: tmp[0] = 'K'; tmp[1] = 0; break;

case 21: tmp[0] = 'L'; tmp[1] = 0; break;

case 22: tmp[0] = 'M'; tmp[1] = 0; break;

case 23: tmp[0] = 'N'; tmp[1] = 0; break;

case 24: tmp[0] = 'O'; tmp[1] = 0; break;

case 25: tmp[0] = 'P'; tmp[1] = 0; break;

case 26: tmp[0] = 'Q'; tmp[1] = 0; break;

case 27: tmp[0] = 'R'; tmp[1] = 0; break;

case 28: tmp[0] = 'S'; tmp[1] = 0; break;

case 29: tmp[0] = 'T'; tmp[1] = 0; break;

case 30: tmp[0] = 'U'; tmp[1] = 0; break;

case 31: tmp[0] = 'V'; tmp[1] = 0; break;

case 32: tmp[0] = 'W'; tmp[1] = 0; break;

case 33: tmp[0] = 'X'; tmp[1] = 0; break;

case 34: tmp[0] = 'Y'; tmp[1] = 0; break;

case 35: tmp[0] = 'Z'; tmp[1] = 0; break;

end

letter[q] = 99;

video_puts(110,result_posY,tmp);

end

//Print "1ST"

video_putsmalls(108,14,first);

//Print "2ND"

video_putsmalls(108,34,second);

//Print "3RD"

video_putsmalls(108,54,third);

//This portion handles pixel drawing from the data from sample();

//This code is here to avoid glitches.

if(y <= yBound && y > 0){

if(wait == 0){

prevX = x;

prevY = y;

wait = 3;

}

wait--;

if((prevX+3 >= x) && (prevX-3 <= x) && (prevY+3 >= y) && (prevY-3 <= y))

{

video_pt(x,y+12,1); //drawing a pixel

writeMap(x,y); //calling writeMap to transform 40x40 bitmap to 8x8 bitmap

}

void clear() begin

int q;

//This erases the screen, the portion that is our bitmap

for(q=192;q<1104;q++)

begin

if(q%16 == 0) screen[q] = 0b10000000;

else if(q%16 == 15) screen[q] = 0b00000010;

else if(q%16 == 12) screen[q] = 0b00100000;

else screen[q] =0;

end

for(q=0;q<8;q++) map[q] = 0;

end

//This routine tests one line of pixels in one character from our library and returns ranking score for that one line

unsigned char testLine(unsigned char libIndex, unsigned char libLineIndex)

begin

unsigned char q, testLine, result, value=0;

testLine = lib[libIndex][libLineIndex];

result = testLine & map[libLineIndex];//perfoming bitwise product = dot product

//summing it up, returning ranking score

for(q=0; q<8; q++)

if((result>>q)&0b00000001 !=0) value++;

return value;

end

//This routine tests all characters. This will call testLine() to do the grunt work

void testChars()

begin

unsigned char libIndex, libLineIndex, value, q, changeFlag, tmp[10], j, offsetY,

offsetX;

unsigned char rank[] = {0,0,0};

for(libIndex = 0; libIndex < N; libIndex++)//looping through all characters

begin

value = 0;

changeFlag = 0;

for(libLineIndex = 0; libLineIndex < 8; libLineIndex++) //looping through one line at a time

value += testLine(libIndex, libLineIndex);

for(q=0;q<3;q++)

begin

if(value > rank[q] && changeFlag == 0)

begin

switch(q)

begin

case 0: rank[2]=rank[1];

rank[1]=rank[0];

letter[2]=letter[1];

letter[1]=letter[0];

break;

case 1: rank[2]=rank[1];

letter[2]=letter[1];

break;

end

rank[q] = value;

letter[q] = libIndex;

changeFlag = 1;

end

void control()//Control basically handles all user commands, of which there are two, clear() and testChars()

begin

if(y>40 && y<54)

begin

if(x>6 && x<12) clear();

else if(x>29 && x<32) testChars();

end

//=================================// set up the ports and timers

void main(void)

begin

unsigned char j,k,sel_X,sel_Y;

//init timer 1 to generate sync

OCR1A = lineTime; //One NTSC line

TCCR1B = 9; //full speed; clear-on-match

TCCR1A = 0x00; //turn off pwm and oc lines

TIMSK = 0x10; //enable interrupt T1 cmp

//init ports

DDRD = 0xf0; //video out and switches

DDRB = 0x00; //inputs. Reset, for example

//D.5 is sync:1000 ohm + diode to 75 ohm resistor

//D.6 is video:330 ohm + diode to 75 ohm resistor

//initialize synch constants

LineCount = 1;

syncON = 0b00000000;

syncOFF = 0b00100000;

//Print "CHARACTER"

video_puts(2,3,cu1);

//Print "ECE476"

video_puts(60,3,cu2);

//side lines

#define width 126

video_line(0,0,0,74,1);

video_line(width,0,width,74,1);

//top line & bottom lines

video_line(0,0,width,0,1);

video_line(98,11,98,74,1);

video_line(0,11,width,11,1);

video_line(0,74,width,74,1);

//init software timer

t=0;

time=0;

DDRC = 0b10001000; //Set PORTC.7 and PORTC.3 to outputs

PORTC = 0b10000000; //pull everything to GND except PORTC.7 for reading Y

ADMUX = 0b00100001;

ADCSR = 0b11000111;

x = 0;

y = 0;

wait = 0;

prevX = 0;

prevY = 0;

//init musical scale

note = 0;

musicT = 0;

//use OC0 (pin B.3) for music

DDRB.3 = 1 ;

//enable sleep mode

MCUCR = 0b10000000;

#asm ("sei");

//The following loop executes once/video line during lines

//1-230, then does all of the frame-end processing

while(1)

begin

//stall here until next line starts

//sleep enable; mode=idle

//use sleep to make entry into sync ISR uniform time

#asm ("sleep");

//The following code executes during the vertical blanking

//Code here can be as long as

//a total of 60 lines x 63.5 uSec/line x 8 cycles/uSec

if (LineCount == 180)

begin

//sample routine

if(alt == 1)

begin

alt = 0;

x = sample(alt);

end

else

begin

alt = 1;

y = sample(alt);

end

//draw routine

draw();

control();

end //line 231

end //while

end //main

Schematic:

Budget:

Item #	Part	Manufacturer	Part #	Cost
1	Palm m125 Touchscreen	Palm		$10	Pre-owned
2	Palm m125 Stylus	Palm		$0	Pre-owned
3	Mega 32	Atmel		$8
4	STK 500	AVR		$15
5	Dual Op Amp	Analog Devices	LM7111	$1
6	TV			$5
7	Resistors & Capacitors	N/A		$0	FREE
8	Boards x 2	N/A		$12
			TOTAL COST:	$50

Task Division:

Our third partner Toni, helps out with the wirings the beginning. However, he is missing from other parts of the lab. This is essentially a two-person project.

Task:	Developed By:
Touch Screen Device Driver	Guoping
Amplifier and Filter	Guoping & Toni
Recognition Algorithm	Guoping & Mark
Touch Screen Calibration	Mark & Guoping
Testing, Refinement	Mark
Report	Guoping

References:

Character Recognition Basics

Palm Inc. (Now Palm One) for the touch screen and stylus from a Palm m125

IEEE Code of Ethics