USB Magnetic Mouse/Touchpad

Abby testing Yiyin testing
Abby Drawing and Yiyin Scratching his Head
May 2007
ECE 476
Yiyin Ma(ym82) and Abby Lin(ayl26)


Introduction

High Level Design

Program and Hardware Design

Results

Conclusions

Acknowledgements

Appendix A: Budget and Costs

Appendix B: Specific Tasks

Appendix C: References

Appendix D: Code

Appendix E: Schematics

Introduction

 This project implements Hall effect sensors and a magnet to mimic the function of a typical USB mouse (similar to a tablet pen’s function). Many digital artists draw with mice on computer or use tablets. However, tablets are often very expensive. Using the mouse is not always as accurate and one needs a steady hand. Our project attempts to create a lower budget alternative to the tablet that is easier to draw with. We use a stack of magnets to act as a pen on pad. Hall sensors detect the location of the magnets, and the information is sent to the computer through USB. Pushbuttons are used to act as mouse buttons.

High Level Design

 Rationale and Logical Structure
Human interface devices(HIDs) have integrated themselves into our daily lives. First, there came PS-2 keyboard. Then came serial keyboard and mice. The current technology is USB keyboards and mice. (Most laptops nowadays only have USB ports.)

Digital art is also very popular, used in movies and other forms of entertainment. Some artists, both expert and beginners, work with digital art through mice or tablets. However, tablets are expensive and not always within budget. The mice are not always accurate to use for those with shaky hands. The tablet’s pen, on the other hand, is more accurate since it mimics the shape of a pen with a clearly defined pressure point.

Since we don’t have the budget to buy tablets, we decided to make an alternative, a low resolution USB magnetic based mouse. We chose to use USB because it is the most compatible with computers. We decided on “magnetic” since we wanted to try out Hall effect sensors for distance.

The mouse moves and operates based on the user’s actions. Left clicks and right clicks can open documents and change properties on files. When the user moves the mouse, the mouse will move as well in the same general direction. The direction and displacement of the mouse is based on relative motion not absolute motion. How much the mouse moves depends on how fast the user moves the mouse. If the user moves the mouse faster, the mouse will move a greater distance. Likewise, if the user moves the mouse slowly, the mouse will move a shorter distance.

Our project can be split into two parts; the USB component and the mouse component. The data from the mouse component is sent to the USB component, which transfers the data to display on the computer. (The gray cable was used to connect to an STK500 board to program our chips. The serial connector is for testing purposes when we used Hyperterm.)
DiagramFull Circuit
Figure 1 and 2: Basic Idea of Project and The Whole Layout in Real Life
The USB apparatus is a USB connector connected to a programmable board with Atmel Mega32 chip. The board is powered by the VCC of the USB connector. The wires going into the MCU are data inputs coming from the mouse MCU outputs; relative X/Y displacements and pushbuttons.
USB Component
Figure 3: USB Component
The mouse component is more complicated. It also uses a board with an Atmel Mega32 chip (powered by a power supply). Wires from PORTB and D go to the USB component as input data. The three pushbuttons (left click, right click, and draw option) connect to the chip. Another bigger soldered board holds six Hall effect sensors and acts as the “drawing pad.” To draw, the magnets(held together by duck tape) are moved across the pad, and the sensors detect the magnetic field data. The push buttons act as left and right mouse buttons. The battery powers the Hall sensors and grounds the MCU. We decided to use a battery since it gave us less fluctuating results for the sensors.
Mouse Component
Figure 4: Mouse Component
Background Math
The background math used in this project was used to determine location of the magnets on the drawing pad.

Trilateration uses at least three reference points to determine the location of an unknown point on a 2D plane. Since our voltage – distance relationship is not linear, we could not locate reference points and determine the unknown point and use this method.

Triangulation also uses three reference points and uses triangle lengths to determine the location of the unknown point. We also could not use this method for the same reason as trilateration.

Bilinear Interpolation consists of two linear interpolations, one in the x direction and one in the y direction and is used to find the location of a point in between four known points. The following equation from Wikipedia explains the relationship between the points. F(0, 0), F(1, 0), F(0, 1), and F(1,1) are the four known points. F(x,y) is the unknown point. In our case, we used ADC values from our Hall sensors and our calibration points to solve for x and y with multiple forms of the equation. Bilinear Interpolation

First Norm is the sum of the absolute values of difference between the numbers.
Second Norm is commonly known as the distance formula in mathematics.

Hardware/Software Tradeoffs
We really didn’t have any tradeoffs. The only alternative was to use a serial USB connector hooked up to the mouse MCU instead of modifying USB code to use on another MCU.

Standards and Copyrights
The only standard that applies to the project here is the USB 1.1 standard. The source of our AVR USB HIDKEYS code is an open source licensed as the GNU General Public License Version 2 (GPL2) and can only be used for educational purposes with "OBJECTIVE DEVELOPMENT GmbH’.

Program and Hardware Design

 Hardware Design
We soldered and built the our two protoboards following Professor Bruce Land’s guidelines. We soldered on pins to the board instead of our wires and chips for convenient replacement.

The Hall sensors are about 1.3mV per Gauss. To detect anything, we needed an amplifier to make the signal readable by the ADC. The Hall sensors were also bipolar, and we wanted them to be unipolar. So, we needed a reference difference of 2.5V.

To get the same amplification for our sensors, we needed a fixed gain differential amplifier. Resistors by themselves are not accurate and can differ 5% from each other, so we used resistor packs to decrease the error. We used the resistors in the packs in parallel to get 2.5V. We found that using a battery power instead of a power supply resulted in less noise from Hall sensor values.

The output of the differential amplifier goes through a low pass filter to with capacitance of 1µF and 51k ohm resistor. This takes care of the high frequency noise, but allows the Hall sensor data to go through.
56
Figure 5 and 6: Hall Sensor Circuit and Hall Circuit Real Life (bottom of drawing pad)

The USB Data+ and Data- operates on a 3.3V signal. The AtMega32 MCU operates at 5V. The Zener diodes and resistors are used to pull down the system to 3.3V. It is optimal to use 3v6 diodes, but we did not have any available, so we made 3v9 diodes. The resistor values work well between 30ohm and 100ohm. We chose to use 75ohm resistors. The 2kohm resistor is a pull up resistor for Data in. The circuit below is based off of one also implementing USB HID system and information from the Objective Development forum.
USB to MCU Circuit
Figure 7: USB to MCU Circuit
Program
USB
A big part of the USB code modification was in understanding the demo HIDkeys code and HID descriptor reports. The HIDkeys demo code implements a simple 17 key keyboard using an USB driver and a main that contains the HID descriptor report and code necessary to call the USB driver. The code is in AVR gcc, and we were not familiar with it. In class, we had always used Codevision and C. However, AVR gcc is not compatible with Codevision and the syntax is different from Codevision C. After trying to convert avr gcc into C to be compatible with Codevision and failing (since we weren’t familiar with gcc), we downloaded WINAVR and AVRStudio4. From there, we changed the MakeFile to be compatible with STK500 and the AtMega32 chip. We also modified main. We replaced the keyboard report descriptor with a mouse report descriptor to handle a mouse. In the buildreport function, we changed the reportBuffer to 3 bytes; buttons, X displacement, and Y displacement. The “keypress” function originally read the key pressed for the keyboard. We scrapped the original code and changed it so that the array key stored data from PORTB(buttons), PORTA (X relative data), and PORTC(Y relative data) from the mouse MCU outputs. In order to program with the STK500 and AVRStudio, we also had to upgrade the firmware on the board. We also edited the usbdrv header file. There, we changed the report descriptor length and parameters to make it compatible with the changes we made in main to convert the keyboard code to mouse code.

The code sends data to USB every time there is a change in data and if the USB is ready to receive data. Since we did not have the mouse component finished at this point, we tested by putting a counter in the keypressed function. After it was called a few times, the value of key would be changed, triggering a change in data and a response from the USB. For a while, we couldn’t get the code to work. We knew that the report descriptor was supposed to be three bytes to hold the button, X, and Y information. However, when we put the info into the report buffer, we put each bit of X and Y into separate positions in the key array, defeating the three byte standard. So, nothing worked. Another mistake we made in the code was a typo in the report descriptor. Until we caught, the computer kept saying “USB device not recognized.” When we did get the code to work, the mouse cursor would always drift.

Mouse
We first ran Professor Land’s ADC code as a starting base. We modified it so the program read six ADC values, one from each Hall sensor using the ADC interrupt. These values are placed into the Ain array. When Ain becomes full, a flag done is set, showing that one ADC value has been read from each sensor.

The most difficult part of this program was mapping coordinates to figure out where the magnets were. We tried many different methods, most involving gridding the pad.

The Hall effect sensors sense magnetic fields and output a respective voltage. The relationship between voltage and distance from the sensor is not linear, making mapping location difficult.

Unused Ideas: The first idea we thought of and heard as a suggestion from a fellow classmate, Brian Chu, was trilateration. This method required us to know the coordinates of our Hall effect sensors and extrapolate the location of the magnet from three sensor locations. Our setup has six sensors and was too complicated for this method to work. Also, even if we had the sensor location, the magnetic data is in voltages not in terms of distance. Knowing this, we did not attempt to implement trilateration. The same happened with the idea of triangulation.

Attempt # 1 to n (We lost count of how many times we tested and modified this method. This section will describe some of the attempts we made on this method): Next, we tried a 10 by 6 grid. We then calibrated by taking down the six ADC values for each square. (We placed the magnet in the middle of the square.)

Our six sensors are distributed on the border of the drawing pad in the shape of a hexagon, which can be separated into three regions; two triangular regions and one rectangular region. We split up the pad into these three regions with the left triangle as Region I, right triangle as Region II, and right triangle as Region III.
Regions
Region I: The sensors in the figure are labeled A0 to A5. In this region, we used A5 to pinpoint the row that the sensor value fell in between the calibration numbers by comparing the box to the right and the current box. We then used A4 and A3 to pinpoint the column by comparing if their values were in between the calibration values of the grid boxes. The coordinates returned belonged to the upper left box that the magnet intersected with.

Region III: This region employed the same tactic as Region I. Instead of using A5, we used A1. Instead of using A3 and A4, we used A2 and A0. To start from the minimum value, we started traversing from the left side of the region. In order to keep to the convention of returning the upper left box, we subtracted 1 from the x value that we found.

Later to be more accurate, we tried comparing diagonal values too. This worked better on certain places, but was worse in others.

Region II: This was the worst of the three. In this region, there are four sensors across from each other and no side sensor to help pinpoint location row location. We tried to mix and match sensor comparisons to find row and column. We always had issues with the locations at the borders of the regions. To put up with points not found in the other two regions, we had the data go through region II as backup. The resulting box reported back was not always accurate. After multiple if statement combinations, we tried comparing pairs of diagonals. This seemed to work, but unfortunately, was not good enough.

We also tried to partition each region even more by checking is say, A3 was greater than A5, meaning that the magnet was closer to A3 and on the top of Region I. However, we found that more conditioning did not work well. We also tried to use ADC values greater than 3 since at some point, when the magnet is far from the sensor, the sensor reading is 3 or less and does not change much. This also did not work very well since it also counted as conditioning.

To further pinpoint the magnet location, we implemented bilinear interpolation, suggested by Professor Bruce Land. We used the two maximum sensor values because bigger values mean that the magnet is closer to the sensor. We then used the equation described in Background Math to solve for a location. It seemed to work pretty well, and we started using it to try to write better “locating box” code. However, there seem to be places that bilinear interpolation fails, and some of those failures are in the middle. Therefore we would occasionally get spikes when we are moving the mouse. We found out that the problem with this is the same as finding the magnet in 4 boxes; we would detect the magnet at the wrong boxes, and using reverse bilinear interpolation gave us values that were totally off the charts. We tried many things to improve on this, but they didn’t help much.

Attempt #n+1: First Norm was also recommended by Professor Bruce Land. We employed a bigger grid (27 x lines by 15 y lines), taking data at the intersection of lines. The results of this attempt were much better than the previous attempts. It probably also helped that we had a bigger grid (though it took hours to calibrate the whole grid). We also tossed out the idea of regions and used brute force by traversing through the entire matrix to calculate the first norm of each cell. To account for fluctuating ADC values, we used averaging for four cycles of the ADC values. Each norm value is compare to the minimum norm of the matrix and replaces it if the norm at the cell is smaller. The cell that contains the smallest norm is the closest to the magnet. This method gave us consistent results that we could use for position sensing. However, the resolution of the touchpad depends on the size of our grid, which isn’t too great.

Attempt #n+2: Second Norm was also suggested by Professor Bruce Land. However, nothing changed with the addition of the second norm. No improvement in accuracy was observed, but the second norm takes more time to calculate. Therefore the second norm was dropped in favor of the first norm.

Final Design: To compensate for the lack of resolution, we used velocity as a way to determine the amount of mouse movement. The user would experience little change in position if the magnet is moving slowly. A big swing would occur if the user moved the magnet rapidly. We also implemented a button that would switch between velocity calculated movement, and position calculated movement. (We used 14-2 fixed point for calculations. For more information, this is very similar to 8-8 fixed point developed by Bruce Land.) Lastly, the left and right mouse buttons were implemented. We also fixed the drifting mouse cursor problem by removing the USART from the program. It was communicating with the MAX232 serial controller, on TxD and RxD, which for the ATmega32 is on PORTD (Y axis).

Combining Mouse and USB
When we ran the code together, we ran into many issues. We ran into problems programming the Mouse MCU because PORTB was being used for output driving one of USB MCU’s input. The programming bits are on PORTB and were fluctuating randomly because it was being used. We solved this by turning on the USB MCU. Some of this was dealt with in the Mouse section since we found problems when we combined the two components.

Results

 The result of this project wasn’t as good as expected. Because of the lack in resolution, the mouse would make jagged movements. The velocity calculated movement isn’t as stable as we hope. The speed of the mouse changes randomly at times, probably because of slight changes in velocity. The absolute position movement is responsive, but it’s very jaggy, and can only move in boxes. Also, the magnet behaves differently from what normal users would expect when the magnet is left up. There isn’t much we can do to improve this unless we have a pressure sensor on the pad.

Safety: There was nothing in our project besides the magnet that was really dangerous. We did not blow up anything (though we did burn a few of our parts when making the circuits and boards).

Usability: The USB magnetic mouse is not recommended for people with pacemakers since the magnets are rather strong and may interfere with pacemakers. Since the magnets are that strong, it is also not recommended to be used on top of a computer. Putting the magnets on top of a hard drive may completely wipe it clean. This apparatus depends on sight, and is not compatible with blind persons.

Conclusion

 Our project worked for the most part and conformed to the USB 1.1 standard. Through working on this project, we realized that it is quite difficult to locate objects and relay how fast they are moving. We really learned to appreciate the current technologies that employ these features. We also learned that it’s very difficult to do position sensing, and our project lacks the resolution needed to be a useful mouse replacement. It is, however, suitable to draw boxes and polygons.

Instead of using six Hall sensors, it may have been easier to triangulate by using three or four sensors. An expansion for the project would be to implement a bigger “pad” for the magnet movement. To make the project look more user friendly, we could package the microcontrollers and wires and just have the pad, USB connector, and magnet visible. We could also try to make both MCU’s USB powered. That way, no power supplies are needed.

Intellectual Property: We reused AVR USB’s demo HIDKeys code and modified it. Since it is open source under GNU, we have not violated any laws. We will follow the license agreements and post our project online with correct acknowledgement and linkage. Our project did not involve any reverse engineering, and we did not have to sign non-disclosure to get sample parts. Our project does not have any patent opportunities since we used the open source AVR USB code. However, if we wrote our own USB driver, we could patent the project.

Ethics: We followed the IEEE Code of Ethics during the development of the USB magnetic mouse. The project idea was a fun idea to us, and we did not intend to discriminate against anyone in making it. During beginning research, we discovered that magnets of the strength we were using could be hazardous to those with pacemakers, so we have placed a notice on the webpage at the top. We also learned from Professor Land that if the magnets were placed on top of a hard drive, the hard drive could be permanently erased. In lab, we warned others of the magnets and have also placed it in the webpage. Along the way, we hit many obstacles and pooled resources from many places to progress. We accepted advice and did not vehemently argue that we were correct in method when we were actually not. To thank and recognize those who have helped, we have listed them in this webpage. We also discussed with classmates our ideas and methods when we found we had some overlapping concepts. We didn’t just take knowledge and not share it. We did not take any bribes for our project regarding anything and we most definitely did not go and sabotage other groups’ projects.

Acknowledgements

 We could not have done this project without the following companies and people:
-FreeScale for the board and battery clip
-Maxim for the differential amplifiers and MAX232
-Allegro for the A1302 Hall sensors
-Professor Bruce Land for all his advice and help and parts
-The 476 TAs that opened the lab extra times for us and helped us! (Idan Beck, Kashif Javed, Donn Kim, Bryan Kressler, Sam Lee*, Eric Okawa, Donald Yunfan Zhang)
-Our ECE 476 classmates who we shared knowledge and ideas with!
-Objective Development for their USB driver HIDKeys demo code
THANK YOU!

Appendix A: Budget and Costs
Part Name Number Total Price Source
D42-N50 Magnets 10 $3.20 KJ Magnetics
Atmel Mega 32 2 $16.00 Bruce Land
MP 120 (12 Hz Crystal) 1 $0.94 DigiKey
9V Battery 1 $2.00 Local Grocery Store
Custom PC Board 2 $10.00 Bruce Land
Solder Board 1 $2.50 Bruce Land
Power Supply 1 $5.00 Bruce Land
RS232 Connector 1 $1.00 Bruce Land
FreeScale Board 1 Sampled FreeScale
9 volt battery clip with 3 and 5 volt regulators 1 Sampled FreeScale
MAX232 1 Sampled Maxim
A1302EU-AT (Linear Hall Sensor) 6 Sampled Allegro
Max4462 (Diff Amps) 6 Sampled Maxim
Pushbuttons 3 Already Owned Yiyin Ma
USB Cable 1 Already Owned Yiyin Ma
Plastic CD Case and Foam 1 Already Owned Yiyin Ma
Total: $40.64

Appendix B: Specific Tasks
Task Name Person
Sampling and ordering/buying parts Shared
USB code modification Abby
Mouse Component Code Shared
Code testing Shared
Soldering Yiyin
Webpage Abby
Research Shared

Appendix C: References

 - AVR USB HIDkeys Demo
- USB.org HID Information
- Bruce Land's Protoboard Design
- Bruce Land's ADC Examples
- Wikipedia
- Allegro
- Maxim
- KJ Magnetics
- Objective Development Forum
- DigiKey
Datasheets
- A1302EU-AT
- ATMEL Mega32
- MAX4462
- MAX232

Appendix D: Code

 USB
The Original Demo Code

Our Modified Code


The files we modified were "MakeFile", "usbconfig.h", "main.c" Mouse

Code

Appendix E: Schematics

 These are scattered in the sections before the appendices. Here are some additional pictures.
Top
Top View of Whole Circuit
Pad
Drawing Pad