Full source code can be found in Appendix A.

 

Microcontroller Software

PWM

To regulate the vibration motor on the stylus, we used pulse width modulation (PWM). Initially, we had 3 motors on each corner of the triangular frame to keep each spool taut. This required us having to do 4 PWMs, 3 on timer0 and 1 on timer2, each with a connection opto-isolator circuit. We were going to have PWM each motor to a low value which would supply a low voltage that would create a low torque used to reel in the excess string and keep lines taut. However, we were unable to find a suitable motor that was powerful enough to produce a sufficiently large torque for force feedback that had low enough internal friction that could be easily backdrivable. Instead, we moved to a passive weight torque system that uses a counterweight that provides the torque needed to keep strings wound an taut.

The motor control signal as calculated by the controller is outputted via pulse width modulation (PWM) on PORTB.3, enabled in the intialization method.  PWM is, essentially, a way of converting an disrete digital output signal into an analog signal.  During each PWM cycle (256 clock cycles), the output port can vary (in "fast" mode) the fraction of the 256 cycle it is high, known as it duty cycle.  Integrated over each PWM cycle, the descrete digital wave becomes an analog signal, proportional to the number of cycles spent high.  This can be expressed as the following equation:

Where n is the number of cycles during each PWM cycle that the output is high.  A visual to help understand PWM is provided below:

Visual representation of a PWM signal.
(source: http://www.societyofrobots.com/schematics_h-bridgedes.shtml)

The PWM output is then passed through the following circuit to electrically isolate the microcontroller from the motor powering circuit.  This is essential because the motor acts as a huge inductor and at the end of each duty cycle when the "switch" is turned off, there is a sizeable back electromagnetic field (EMF) across the motor terminals which produces a significant amount of noise, enough to force continual microcontroller resests.  The capacitor across the motor filters out the high frequency noise and the diode provides a path to ground out the induced current when the electronic gate is opened.

Optoisolator and motor power circuit.
(taken from: http://people.ece.cornell.edu/land/courses/ece4760/labs/f2012/lab4.html)

The schematics for the 4N35 and BUZ73 integrated circuits are shown below and will aid in proper circuit assembly.

Integrated circuit schematics.
(taken from: http://people.ece.cornell.edu/land/courses/ece4760/labs/f2012/lab4.html)

This optoisolator circuit is the actual unit that decouples the microcontroller and motor ground and prevents any noise crosstalk and works by internal emitter and phototransitor communication.

Push Buttons and User Interface

We debounced 2 push buttons that were used to interact with the entire functional system. Both buttons were connected via a 30-ohm resistor to ports D.5 and D.6. The first button was used to indicate a switch between 2 modes: 1. drawing cubes 2. drawing spheres. The second button was used to indicate to draw the object in virtual space in MATLAB. The object drawn is dependent on the current mode (cube or sphere). The following state machine diagram highlights debouncing the buttons to check for a valid press:

State machine diagram for debouncing button presses.

In main, stateMachine() is called every 30ms (tracked by a counter in the 1ms timer 1 interrupt). A valid press is one which is held for 2 cycles of 30ms (60ms) while the converse is true for a valid release.

USART

To send data packets to and from the MCU and MATLAB, we used serial port communication. To establish communication, we made MATLAB send a dummy packet each time, before each message was sent from the MCU. The dummy packet indicated whether connection was established and whether or not the stylus was within the boundaries of the virtual MATLAB Object. On the MCU, we sent each message in the following encoded format:

printf("%dx %dy %dz %dd %dm\r",adcVal,adcVal1,adcVal2,drawPress,choosePress);

Each message from the MCU had information regarding the 3 spool length in triangular 3D coordinate system and user messages indicated via the 2 buttons (to draw a shape and the chosen shape draw mode). To increase speed of boundary detection especially when the stylus is moved quickly, we increased the baud rate to by setting the UBBR0L register accordingly to 8, corresponding to a baud rate of 115200. Further details are given in the MATLAB section.

ADC Sampling

To extrapolate 3D coordinates in the triangular-based spool system, we made use of three 10-turn potentiometers, each with a spool of thread wound on it. Each potentiometer reading was ADC sampled and internally scaled to a 10 bit value (0 to 1023). Since we were using 10-bit precision, we had to read the low byte (ADCL) before reading the high byte (ADCH) register during each ADC Read. The timer1 clear on match interrupt triggered every millisecond, is used to do an ADC Read on the previous sampled value and to start a new ADC sample conversion. Since, we needed to ADC Sample across 3 ports (ports A.0, A.1 and A.2) and could only use a single ADC channel at any time, we cycled periodically through each to give all three spool distances in 3D triangular space. To convert the 10-bit ADC value to the length of thread unrolled we used the following conversion formula:

MATLAB Software

There are two software packages that were written to form an adaptable and agile development platform for futher advacement: one, called Surfaces.m, is designed to allow the user to feel complex curved surfaces, with a fixed base on the z-plane. The other, called Objects.m, allows users to place spheres and cubes in three-space, then trace them with a stylus. In both packages, the haptic feedback is given by a vibrating motor at the base of the stylus. The primary function of MATLAB here is twofold; first and foremost, to use its extensive collection of graphics libraries to give a strong visual element to our design; second, to handle many of the more complicated calculations (such as boundary detection and vector arithmetic) that would have proved too computationally intensive to perform on the controller. To convert from the awkward coordinate system given by ADC values of respective radii of the spools, we used a technique called trilateration (used in tracking location). If you have information about the distance of a point from three other points, then you can take the point to be intersection of three spheres centered at those three other points. In our system, the three other points and the three corners of our triangular base. The three spheres can intersect twice once above once below. But we know that movement is always below the triangular base.  All the serial communication libraries and graphics libraries were built off of Bruce Land’s supplied code, listed in the References Section.

Overall View - Surfaces.m

User interface in Surface mode.

In Surfaces.m, we take advantage of the face that MATLAB allows surface plots known as "zplots" to be stored as two-dimensional arrays. These arrays are essentially x-by-y coordinates (corresponding to indices), the value of which is the z-coordinate in the plane. This format lends itself to extremely efficient x-y-z lookup. From a top-down perspective, we are simply creating an interesting surface shape, storing it as a zplot, and then taking the x-y-z coordinates of the mechanical frame and doing a trivial lookup to see if a boundary has been crossed. If it has, send a packet to the controller telling it to activate PWM and vibrate the motor. If not, then do nothing. No matter what, keep looking and updating the x-y-z coordinate, until the program exits. The user’s cursor is represented by a small circle, but this is unfortunately an unrealistic representation, as it is in 2D – but it is the only type allowed by the zplot.

Overall View - Objects.m

User interface in Objects mode.

The goal of this program was to be able to not only feel pre-existing objects but to create new ones altogether. This program is slightly more complicated because it requires a different kind of x-y-z lookup logic. These objects are created with the CSG Object library, and are stored in a much different way than a zplot. Instead, every CSG object has within itself a 3-by-x matrix (where x is dependent on the resolution of the object) that contains all the surface points of the object in a -1 to 1 scale in all three dimensions. To update the cursor, we needed animation – so we created a function whose job it is to continuously clear the entire plane, and then redraw the static elements in the same spot and the cursor element in the original spot plus the change in x-y-z. Thus, the illusion of an animated, updated cursor is created. By moving this cursor representation around the screen in real time, we give the user a better sense of location in three-space. These x-y-z points are taken and compared with the objects in two ways: for the cubes, we simply check if the point is within all size faces, which only requires knowledge of the min’s and max’s of each dimension on the surface matrix. For the spheres, we take the distance of the x-y-z coordinate from the origin of the sphere, and compare it to the radius.

The "mins/maxs" and "radii" are actually arrays, and every additional object is an additional array entry: animation function is called one per pass through the main loop, and for every animation call, all the boundaries (the entire array) are checked. MATLAB is able to do this iterative calculation quickly enough compared to the bottleneck (the actual rendering) that it does not affect performance. The drawing is done by pressing a button on the stylus, and a second button determines whether the user is in "Cube Mode" or "Sphere Mode."

Graphics Libraries

The graphics used in these programs were built off of libraries taken from Bruce Land titled "Constructive Solid Geometry," it is based on the principle that complex three-dimensional shapes can be formed by combining the 3D fields of elementary shapes stored as “positive outside, negative inside.” There are many library functions that take advantage of the ease with which these models can be combined; basic, overlapping CSGcubes can, for instance, be subtracted from each other with CSGsubtract(cube1, cube2). The only parameters needed to build the basic CSGcube, CSGcylinder, and CSGsphere are the x-y-z center points, the size (or radius), the resolution (which is maxed out at res = 50), and, in the special case of the cylinder, the height’s axis.

When a CSG object is created, it is saved as the aforementioned "negative inside, positive outside" matrix. For rudimentary objects, this is a very simple representation, where a cube of resolution 3 may for instance appear as:

Combining objects obscures this simple representation, and creates non-1 values. Therefore, another way to look at objects had to be used. As it turns out, when a CSG object is passed through CSGtoSurface (which is necessary to render it to the screen), the resulting object is stored as a structure containing, among other things, a matrix of surface points. By checking against this matrix, we were able to determine boundaries corresponding to real-world positioning. Finally, this CSGtoSurface is passed to renderpatch(object) and drawn to the screen. We used a simple stationary face lighting in order to show the dimensions (not including lighting would make the image appear one-dimensional), and we turned on 3D-rotating of the plot.

Serial Communication:

Connection between the ATmega and MATLAB has already been discussed from the controller side; here, we will briefly mention the nuances of serial connection from the MATLAB end. One of the first commands we call is fclose(instrfind); this is similar to other close commands such as close all and clear all, but it specifically eliminates any unwanted, bogus serial connections. The actual setup of a new serial connection is a two-step process: first, the communications variable (called s in our codes) is defined by serial(communications channel, baudrate, terminator character). Baud rate determines how fast packets are sent over the channel. For higher performance – at the cost of accuracy – we chose a baudrate of 115200. According to the chart of baud rates, this introduces an error of 8.5%. This was deemed an appropriate tradeoff in terms of bit accuracy, and may have contributed (among the inherent inaccuracy of running ADC at full-speed, ten-bit mode) to the ever slight tremor of the cursor at rest – more on this later. The terminator character, for us, was chosen to be the "/r" seen in the C serial code.

Taken from Page 200 of the ATmega1284 datasheet

With the serial package defined as s, the next step is to establish connection with fopen(s). This command hijacks the communications port specifiedss earlier. Since this microcontroller only has one such port (through USART), it was impossible to debug with UART once the connection was made. This was not an issue, since we could simply print our visual debugging statements through MATLAB instead.

"Are you ready?" "Yes, I’m ready." "Are you sure?" "Yes, just do it already!"

MATLAB is extremely finicky about its serial communications protocol. In establishing connections and engaging in reliable data transfers across networks, it is common to use what has been coined a "handshake" between the involved parties to assure a packet is getting through. MATLAB demands the first move; it will not allow reliable packet reading on its own end until it is sure a channel has been opened, and the only way to be sure of that is for it to be the first to send  and solicit a garbage response before getting to business (ie: the "handshake"). Communications are sent from MATLAB using fprintf, so no matter the functionality, MATLAB codes written for this project send a garbage 's' character first (which corresponds to "I'm here! And don't vibrate yet!") and receive some preliminary garbage in return (which is why an fscanf is done, then the stored data is immediately overwritten). Now, MATLAB is ready for real communications!

Everything is a state machine, after all

Because the size of a packet is very limited, we decided to design our system such that there were two separate, minimalist packets which MATLAB could send the microcontroller to let it know whether our cursor was within a boundary or not. If MATLAB sends the controller an 's', it knows to keep sending coordinates but it will not PWM the motor; likewise, if MATLAB recognizes that we are within a boundary, it sends a 't', from which the controller knows to activate the PWM. The reading-writing-reading dynamic must be held constant until the program is terminated, so it is contained as the first instruction within a while(1) loop.

The next thing that happens is also the same no matter which file you are exploring: the incoming packet, read from fscanf, contains certain valuable data that must be converted from a string to a number in order to be useable. The general format we followed was to send packets from the microcontroller as a single, large string that contained everything we needed separated by certain markers. For instance, we could send "13x212y251z," and this would then be converted into an x, y, and z integer coordinate by taking the strfind of those special character markers and using them as indices of substrings. This is a very general example; for our code, we also used other special markers for items such as "sphere mode vs cube mode" and "are you drawing now?"

Iterate, iterate, and…… iterate!

Like the hardware and C code, our MATLAB implementation saw a number of updates and revisions throughout its design cycle. For one thing, the idea to draw to the screen in real-time wasn’t even something we were committed to doing until we realized that some students had done it (using the same libraries, but for a different purpose and having a different implementation) for a previous final project (see Haptic Glove). Our original idea was to create objects in SolidWorks, then somehow port those objects to the microcontroller. We later discovered that SolidWorks objects can be transferred to MATLAB, where they are presumably stored in a matrix form that could be susceptible to some form of boundary checking. This was perhaps one of our more ambitious ideas, and it ultimately did not come to fruition due only to a shortage of time needed to work on integrating those libraries.

We decided a happy middle ground was to simple design objects in MATLAB and render them directly from that code. Even then, our design saw many different versions. Our current packet sending-receiving system is the product of massive amounts of iteration. Because we lacked access to UART during the testing phase, we relied solely on MATLAB output to know whether we were on the right track or not (before our graphics were working). For instance, our very first triumph was being able to turn a packet string of x-y-z coordinates from the microcontroller into x-y-z integers in MATLAB, and printing them to the screen while being manipulated by a single spool in the z-direction. Once we had this working, we knew the rest of the project was doable.

At one time, there was only a single "draw" button, and the packet looked like "_x_y_z_draw." The only object available to draw was a cube, but because of the simplicity of this packet, holding down the single button allowed continuous drawing. When we changed the packet to "_x_y_z_d_m," we had changed the core functionality of the drawing function by turning the C code portion of the button press into a two-button-optimized state machine that sent a "pulse" rather than allowing holding the button for a prolonged period of time. Now, you could no longer hold and draw, but rather "place" objects to the screen. The functionality had changed from "drawing 3D objects" to "placing 3D objects on the canvas."

Finally, feeling surfaces was something we added much later than the 3D shapes program. It was something we knew we wanted to do from the beginning, and we always had an idea of how it might be possible in theory, but we didn't actually commit to implementing it until we were sure we had mastered the 3D shapes program.