Skip to main content


Ultramouse 3D is a spatial human interface device that lets the user interact with computer programs in three dimensions.

Introduction

Ultramouse 3D times the delay of high-frequency sound waves from the unit held by the user to each of three receivers and passes this information along a serial cable to the computer. The accompanying open-source API provides easy functions that let any Win32/C++ application start reading 3d data in less than 5 minutes.

This is our unique final project for ECE 4760 at Cornell University.

Karl Gluck
MCU software implementation, C++ API and 3d demo code
David DeTomaso
Hardware design, refinement and analysis

Design

Details of the software and hardware setup for Ultramouse3D

Conclusions

Results of our project and ideas for future improvements

Appendix, Videos & Downloads

Everything from schematics and a bill of materials to the C++ API and a DirectX demo project.

High-Level Design

Sound travels through air at approximately 345 m/s. From this, the time it takes for sound to travel between a transmitter and receiver can be described as:

From this, a reasonably fast counter could keep track of a time delay between transmission and reception and back calculate a distance.

If the time of transmission and the time of reception on each of three receivers is known, R1, R2, and R3 can be calculated. Then, with known values of w and h, the 3d coordinates of the transmitter can be calculated by solving a system of equations of the form:

A high-level diagram of the entire system is pictured here:

Pulses from the transmitter are initiated by the MCU. The MCU then checks input pins from each of three receivers every 6.25 µs while keeping track of a counter variable. When an input is detected, the current counter value is saved. After all three inputs have been detected, the three saved counter times are sent to the PC for processing and update of the mouse position.

The only patent we could find is US Patent # 4862152 which details a similar system of ultrasonic 3d positioning and was filed in 1985. However, beause the patent does not seem to be extended and because our design differs in the specific way pulses are detected and position data is filtered, we do not think that our project is in violation of this patent.

Design Considerations

Placement of the ultrasonic sound receivers will affect the accuracy of the measurements. In order for sufficient resolution, the distances between receives should be on the same scale as the distance between the transmitter and the receivers. Receivers very far apart are not practical for the user and receivers close together will have very small time offsets. For the application space we intended, it made sense to mount the three receivers on three corners of a computer monitor.

In order for the MCU to accurately resolve the three distances, a fast counter is needed. However, because the MCU’s onboard Timers cannot be interrupted with three separate external inputs we must emulate a timer by calling an interrupt at a fixed rate. Increasing the rate of this interrupt will allow the system to resolve distances more accurately but also limit the amount of code that can be executed within the interrupt. We chose to run this interrupt at a rate of 160 kHz so that accuracies as fine as 2mm could be resolved and the interrupt could use up to 100 processor cycles to handle the receiver signals.

Analog Design

Receiver Circuit

The main purpose of the receiver circuit is to amplify the small 40 kHz signal coming from the ultrasonic receiver so that a digital input port on the MCU will read a logic high when a signal is being transmitted and a logic low at all other times. Additionally, the delay through the circuit should be minimal to allow for accurate positioned based on the sounds propagation delay in air. To accomplish this, we have cascaded three operational amplifiers configured for an overall gain of around 80 dB. For each amplifier, we required the following specifications:

  • An input range that includes ground
  • Low input offset voltage
  • 30-40 dB of gain at 40 kHz
  • High slew rate (at least 1 V/µS)

We found the following specifications well satisfied by the Japan Radio Co. NJU7031D operational amplifier. The resulting circuit worked well within the required specifications and was able to amplify input signals to small to be discerned on the oscilloscope to levels detectable by the digital input pin.

Transmitter Circuit

Our transmitter circuit is modeled after the fan driver circuit found in lab 5. To drive the ultrasonic transmitter an alternating voltage at 40 kHz must supplied across the transmitters two leads. To accomplish this, we used a digital output pin the MCU to drive the gate of a power transistor. By placing the transmitter, in parallel with a resistor, between the transistors drain and the supply voltage, a 40 kHz signal supplied by the MCU can drive a 40 kHz signal across the transmitter ranging from supply to ground. To transmit as strong a signal as possible, we use the voltage coming directly out of the AC/DC transformer which tends to range from 9 – 13 volts.

MCU Software Design (AVRStudio / GCC)

The most critical section of software design is the TIMER0_COMPA ISR. When enabled this interrupt runs at 160 kHz by setting Timer 0 at the full 16 MHz speed, activating interrupt-on-overflow, and setting the overflow value at 99. Each pass, the interrupt updates timebase_160kHz, a counter variable, generates the 40 kHz transmit signal if transmission is active, and polls each of 3 receiver inputs, copying over the timebase_160kHz value if a trigger is detected. Because of its high frequency of occurrence, the interrupt must run in under 100 cycles. To achieve this specification, various optimizations have been made. Several cycles were saved by outputting timebase_160kHz on port B and using B.1 as the 40 kHz transmit signal. Redundant polling of receiver pins is avoided by checking first to see if the pin had already been triggered and recorded.

The main method of the MCU program is used to initialize required values and to take over once all three receivers are triggered. After variables are initialized, an infinite while loop continuously checks to see whether three receiver triggers have been detected or if the timeout value has been reached. If either of these is the case, TCCR0B is set to 0 to disable the 160 kHz interrupt so that transmission can be handled efficiently. If three triggers have been detected, the loop waits as ISR USART0_UDRE is called to send time delays to the PC. In either case, delay and counter variables are re-initialized and the 160 kHz interrupt is enabled and set to transmit for its first millisecond.

PC Software Design (Visual C++)

The API for UltraMouse 3D exposes three main functions:

  • UM3DAPI_Create

Initializes the serial port and Ultramouse3D structure

  • UM3DAPI_Update

Called every loop of the main program to read data from the device and calculate its coordinates

  • UM3DAPI_Destroy

Cleans up the structure when the program exits

The functionality of these methods is clearly explained in the source file, but a few design points are worth mentioning.

First, the incoming data from the receiver is very noisy--for every hundred good samples, there are a few huge spikes in the delays. Standard linear filters we attempted (low-pass, mean, moving average, exponential weighting) were unable to remove these effectively without creating such large delays in the data that the interface felt sluggish. Instead, the implementation uses a median filter on the incoming data points. This effectively eliminates all large jumps in the data; however, it leaves smaller ones. In the figure below, data was taken for the user drawing a square (more data here). The blue stars are the source data, and the red line is the median-filtered data.

The second important point is how we removed the smaller jitters in the data. By smoothing the triangulated spatial coordinate from the output of the median filter into the "current" position by an exponential weight, the smaller jumps are hardly noticeable. Unfortunately, this means that the filter's ideal exponential constant has to be tuned to the rate at which UM3DAPI_Update is called.

Finally, the Z-axis is different than one would expect. Although the math derived to calculate a true z-coordinate works fine in theory and MATLAB simulation with ideal data, in practice it is unusable. Instead, the UM3D API simply takes the sum of square distances to each receiver and calls that the "Z" coordinate. As a result, "Z" is more sensitive on the top and left side of the screen (where there are two transmitters close together); however, the slight distortion isn't nearly as bad as using the extreme distortion experienced with a true mathematical solution. A second effect is that Z is positive coming out of the screen, making this a left-handed coordinate system.

What Went Wrong?

Initially we tried using a much cheaper op-amp for signal amplification. Problems arose because of a large offset voltage on the inputs (limiting gain) and a slew rate too low to support 5V output swings at 40 kHz. Ordering the NJR NJU7031D op-amps solved both of these problems.

We were originally going to implement this receiver as a plug-and-play mouse replacement by using standard USB mouse drivers for communication with a PC. However, implementing USB proved to be difficult and require an additional MCU. Because USB output was not an interesting or vital aspect of our project we decided to simplify and use serial output instead. This proved to be a good decision because it allowed us to implement the mouse in full 3d (mouse input only handles 2d) and to handle filtering of the pulse delays in the C++ program on the PC.

We also found that the serial interface on the computer is very slow. The MCU has to delay 30 milliseconds (enough time for 480,000 instructions!) every loop to avoid filling up the PC's serial-port buffer. The symptom of transmitting too quickly is that motion becomes delayed by several seconds as the buffer is slowly emptied by the receiving program.

One final "gotcha" was that if you start up the MCU before starting the client program, there is a chance that the program will start picking up bytes in the middle of a transmission. The incoming data will look like junk (delays should be less than 400 most of the time, but junk will often be tens of thousands), or it may seem like receivers are placed in the wrong order.

Results

Our device samples position data at a rate of 50 Hz allowing for quick updating of the receivers position in a real-time feel. The range of operation is about a meter. This is because we are using a transmitter to act as one of our receivers in order to save cost and its lowered sensitivity to the ultrasound signal limits our range.

To test the accuracy of each receiver, we recorded while holding the transmitter still for 12 seconds.

Receiver Mean Distance Maximum Variation Standard Deviation
#1: Bottom Left
54.3 cm
7.1 cm
1.5 cm
#2: Top Left
45.8 cm
2.6 cm
0.5 cm
#3: Top Right
45.8 cm
3.4 cm
0.6 cm

Because our design emits audio frequencies at 40 kHz, it will interfere with other designs that operate on the same frequency. The transmitter is extremely easy to use: all that is needed is to hold the device within range of the receivers and point it in their general direction.

Conclusions

Our device can function well in the way that we intended it to – as a 3-dimensional input device to a computer. However, jitter in the time delays recorded by the MCU were higher than we initially expected. Additionally, the range of approximately 1 meter was less than we expected.

If we were to complete an Ultra-Mouse v2.0, the following changes would be made:

  • Increase the strength of the transmitted signal
  • With increased transmission strength, decrease gain on receiver circuits to increase selectivity
  • Simplify receiver circuits to contain one instrumentation amplifier cascaded into a high speed op amp.
  • Run the sampling counter much faster by using a faster MCU so that more data is available for signal processing.
  • Use a battery and oscillator circuit to allow for a wireless transmitter.

If further development were considered to allow for the use multiple simultaneous inputs then we propose the following design changes:

  • Flip the circuit around so that the held units contain receivers and the mounted units contain transmitters.
  • Transmitters will pulse one at a time at a fixed rate
  • Receivers will calculate how the rate they observe deviates from the known fixed rate
  • Increase the number of transmitters from three to four so that precise synchrony between transmitters and receivers is not needed
  • Position information sent back to the input unit using the wireless Bluetooth protocol

In this way, a large number of 3d position units could be simultaneously used without interfering with each other. This could be useful if the device were incorporated into a gaming system to allow for multiplayer capability.

Ethical Considerations

The IEEE Code of Ethics can be found by following this link.

In the spirit of training ourselves to practice as ethical engineers, we have followed this code of ethics throughout the design, assembly, testing, and reporting of our project. As per our responsibility, we have taken precautions to ensure the safety of anyone using our device by preventing accidental electrical contact. Since our principle interests in the design of this device were to further our knowledge of design using microcontrollers and to complete the ECE 4760 course, no conflicts of interest exist. All of our claims on the devices performance were made based on real, un-altered data collected while testing our device.

We believe our device is an appropriate application of technology using ultra-sound and in its design we have improved our own and possibly a few friends understanding of what can created using electrical devices. Because our project fits within the skill-set of ECE 4760 and because both of us satisfy the requirements to enroll in this class, we felt qualified to undertake this project. All work, unless otherwise credited, is our own. Issues such as bribery, fair-treatment, and malicious action were not encountered throughout our completion of this project.

Legal Considerations

The safety of our users is both a legal and ethical consideration and so precautions have been taken to ensure no skin-to-wire contact when using the device. The only significant transmitted energy from this device is the 40 kHz sound wave emanating from the ultrasonic transmitter. Because this is not an electromagnetic wave and 40 kHz is outside the range of human hearing, its intensity is not regulated by the FCC. However, this noise will be detectable by some household pets including mice, cats, and certain species of dog. Before commercial deployment of such a device, research will have to be undertaken to rule out the possibility of induced hearing damage in these household pets when near the transmitter of our device.

Intellectual Property Considerations

The only material we used not created by us is the Tiger_3d demo which is distributed for free use by Microsoft. It would probably not be possible to patent our project in its current state because 3d ultrasonic positioning has been patented before however a more specific implementation (such as the suggested changes to include multiple receivers) would be significantly different from previous ideas and might be able to receive a patent.

Appendix

Videos







Bill of Materials

Part Quantity Unit Cost Total Source
--
--
--
$42.07
--
Ultrasonic TX/RX Pair
2
$7.95
$15.90
Jameco P/N 139492
Solderless Whiteboard
1
--
--
Previously Owned
Custom PC Board
1
$4.00
$4.00
ECE 4760 Lab
BUZ73 N-Channel Power MOSFET
1
--
--
ECE 4760 Lab
Push-Button Switch (unused)
1
--
--
Scrap Pile
Through-Hole Resistors & Capacitors
many!
--
--
ECE 4760 Lab
Wire
lots!
--
--
ECE 4760 Lab
High Performance Op-Amps
9
$0.75
$6.75
Digikey NJR P/N NJU7031D
RS-232 Serial Port
1
$1.00
$1.00
ECE 4760 Lab
MAX233CPP
1
--
--
Sampled
16 MHz Crystal Oscillator
1
--
--
ECE 4760 Lab
LM340T5 Voltage Regulator
1
$0.42
$0.42
ECE 4760 Lab
Atmel ATMega644
1
$8.00
$8.00
ECE 4760 Lab
8 pin DIP Socket
1
$0.50
$0.50
ECE 4760 Lab
40 pin DIP Socket
1
$0.50
$0.50
ECE 4760 Lab
9V Power supply
1
$5.00
$5.00
ECE 4760 Lab
Copper Wire Board
1
--
--
Scrap Pile

Schematics

Receiver Circuit

Transmitter Circuit





Syntax-Highlighted Code Listing

To download all of the project files at once, click here. You can also download some examples of data as plain-text (comma delimited). Included is a MATLAB script to graph the data with and without median filtering.

API source files (for the PC):

You'll need a C++ compiler that can create Win32 applications, such as Microsoft Visual C++ Express Edition.

ultramouse3d_api.h
ultramouse3d_api.cpp
ultramouse3d_api.zip [download]

Demo Project: Tiger3D

This project is based on the Microsoft Direct3D9 Tutorial #5 sample code. It makes use of the DirectX SDK, and requires a compatible graphics card to run (most computers have one).

tiger3d.cpp
ultramouse3d_tigerdemo.zip [download]

For the ATMega644

The code for Atmel's microcontroller was written to compile with AVR Studio 4.

ultramouse_mcu.c
ultramouse3d_mcu.zip [download]



References & Datasheets