| design + hardware + software + results + next + code + wires | ANDREW FIORE |
|
The concept behind the Zen touchpad simple: the user glides his finger over a trackpad surface, like those found on laptop computers, and the stroke appears on a two-bit, graphical liquid crystal display. Several seconds after the user makes a stroke, the image vanishes, leaving a clean slate on which to draw more. My inspiration for the Zen touchpad was the "Zen tablets" often found in art and gift shops. These tablets consist of a rice paper over some kind of absorbant surface, so that when the user paints with a brush wet only with water, the rice paper becomes translucent and sticks to the darker surface below, allowing it to show through. These brush strokes fade to invisibility as the rice paper dries. The progression of the drying process over the course of the stroke is, in fact, visible on Zen tablets -- that is, the beginning of the stroke vanishes first, and the process continues down the length of the stroke until it is entirely gone. The Zen touchpad can achieve the same effect by keeping a time-to-live datum for each segment of the user's finger movement on the touchpad. If the touchpad is sensing the movement with fine enough temporal granularity, the times-to-live might even be maintained at an LCD-pixel level.
Touchpads generally offer relative and absolute finger-tracking modes. For the Zen touchpad, absolute tracking is most appropriate, because we want locations on the touchpad relative to its edges and size to correspond to the same locations on the LCD, also relative to its edges and size. The Zen touchpad's interface is as minimal as an interface can be: the user interacts by simple touch and gets immediate, directly related visual feedback. To transform the user's touch from touchpad-space to LCD-space, we map the x and y values of the touch first into the range [0 .. 1] by dividing each by the maximum x and y values that the touchpad generates, then multiply the resulting values by the x and y dimensions in pixels of the LCD, yielding absolute screen coordinates for a touch. Given those touch coordinates, we use a 5x5-pixel brush centered around the touch to paint that spot on the LCD, e.g.: 
The particular brush shown here would have a somewhat calligraphic effect because it makes narrow strokes going northeast-southwest and wide strokes going northwest-southeast. A round brush would work just as well -- and in fact, multiple brushes can be stored as 25-element arrays in program memory (flash) on the microcontroller and loaded as needed. Brushes can be cycled through by clicking the left trackpad button, but it's not necessary for the user even to know about this feature. Each touch has a time-to-live that starts at a predefined value in the 1-10 second range (the user could cycle through several options by clicking the right button) and ticks down. When its time-to-live expires, the brush impressions created by that touch would vanish from the display, yielding the visual effect that the user's finger stroke is drying or fading with time. To enhance the fading effect, the outer pixels of the brush may be assigned a shorter time-to-live. That is, the inner 3x3 square of pixels in a 5x5 brush might get time-to-live t while the remaining outer strip of pixels would get time-to-live t/2. This gives the impression that the user's stroke is thinning before vanishing. A major problem with assigning times-to-live to individual touches (by which I mean absolute locations reported by the touchpad -- there will be very many of these for each user's touch) is that when their brush impressions overlap on the LCD, a naive algorithm will clear pixels when the oldest touch which has affected them dies rather than preserving the pixels until the most recent touch dies. Given a short enough time-to-live (that is, a fast enough fade) and a small enough brush size, this may be visually acceptable. On the other hand, with large enough brush and a slow enough fade, the tail of the stroke as it fades will appear to have brush-shaped bites taken out of it as previous brush impressions fade, taking all their pixels with them, including those that belong also to a subsequent impression. The best solution to this problem, then, requires external SRAM on a chip like the Atmel 8515. With the additional memory, we could make a 1-byte-per-pixel time-to-live buffer that would function something like a Z-buffer in three-dimensional graphics. When a brush impression is drawn on the LCD, the appropriate pixels are darkened, but instead of simply assigning the whole brush impression a time-to-live, we examine each pixel's entry in the time-to-live buffer and assign to it the greater of its current value and the new value that the fresh brush impression wants to assign. That way, no pixel will expire until the most recent touch associated with it dictates that it should.
The Zen touchpad is driven by the Atmel AT90S8515, an 8-bit microcontroller. Connected to it is an Alps Electronics touchpad (product number 808-874649-002-A 68S09726 Rev. B), which operates by the PS/2 protocol for input devices. The display is an LCD driven by the Toshiba T6963C controller. The PS/2 standard uses four lines for its bus-like communication between host (microcontroller or PC) and device (mouse or touchpad): CLK, DATA, VCC, and GND. Professor Land and I deduced based on the circuit layout and data sheet that, on my particular Alps touchpad, which has a small, 4-pin connector on its underside, pin 1 is VCC, pin 2 is DATA, pin 3 is CLK, and pin 4 is GND. Because the protocol requires that DATA and CLK be floating high via pull-up resistors to VCC rather than driven high by host or device, I enable the Atmel's internal pull-up resistors on its input pins. PS/2 specifies 10Kohm pull-up resistors; the Atmel's fall between 35Kohm and 120Kohm, but testing has shown that communication succeeds regardless. For this prototype, I used the STK200 prototyping board for the 8535. I connected the touchpad to port C on the board, which corresponds directly to the port C input/output pins on the 8535 package itself. I also connected the serial port of the STK200 to my PC so that I could use the 8535's on-board UART to send PS/2 data and debugging signals to a terminal emulation program for inspection.
The only software I had time to implement for this project was the PS/2 interface itself, which proved far more involved and challenging a task than I had anticipated, due in no small part to insufficient documentation. Alps Electronics' own documentation for their touchpad explains the data format used for "advanced mode," which is basically absolute position reporting, as opposed to "mouse compatibility mode," in which the touchpad reports relative position like a PS/2 mouse. I spent quite a long time muddling through the Alps data sheet with no success until I found the equivalent data sheet for Synaptics touchpads. The touchpad-specific information in this document was no help, because different manufacturers implement different data formats for absolute position reporting, but starting on page 27, Synaptics' data sheet offers a clear, detailed explanation of the PS/2 protocol, including timing diagrams. I highly recommend it to anyone who needs to implement PS/2. The software implementation of a PS/2 interface is not too complicated given a quality description of the protocol. To facilitate reading and writing bytes from and to the touchpad, I first wrote some helper functions and macros, including routines that wait for the clock (pin C0) and data (pin C1) lines to float high or be pulled low and a delay subroutine that simply spins for a given number of microseconds. I used an interrupt only to time the delay for the latter procedure. Subroutines getByte and putByte form the meat of the interface. Probably the most delicate point to keep in mind in dealing with a PS/2 device is the state of the bus. A PS/2 bus has three states:
The basic packet structure in PS/2 is 11 bits: a start bit (always low), 8 data bits, an odd parity bit (which is set if the data byte contains an even number of 1s), and a stop bit (always high). This format is used for both sending data to and receiving data from the touchpad, or any other mouse-compatible PS/2 device. The precise mechanism for sending and receiving packets is detailed in the Synaptics data sheet. Basic operation of the touchpad involves waiting for the power-up bytes (AA 00) and sending an enable command (F4). Most PS/2 mouse devices turn on in streaming mode, which means they continuously send packets of data as the user's actions produce it, but they may still be disabled so that the data doesn't immediately confuse an unprepared host. Thus, sending an enable byte is a key part of initialization. After that, touches and button clicks will generate triplets of data packets. These triplets are described on page 41 of the Synaptics data sheet. In short, the second byte contains change in the X direction, the third contains change in the Y direction, and the first byte contains bits corresponding to byte 3 overflow, byte 2 overflow, byte 3 sign, byte 2 sign, a constant 1, then the middle button, the right button, and finally the left button. In absolute position reporting mode, touchpads generally send six bytes at a time rather than three. Each, of course, is still wrapped in the same packet described above, and the host is free to put the bus in Inhibit mode between bytes. My PS/2 routines also include methods for sending bytes received from the touchpad through the UART for display on a PC terminal. The bytes are displayed with associated numbers 1 through 9. The numbers reset from 9 back to 1 continually because their purpose is simply to distinguish bytes on screen, not to provide globally unique identification. The code for my PS/2 implementation.
The PS/2 interface is fully functional. I have tested it with a number of standard commands, including enable, disable, status, and set mode. The Alps touchpad acknowledges these commands with the FA byte and acts on them appropriately (beginning to send data packets after an enable in streaming mode, for example).
The next logical step in this process would be to write the LCD controls (simple enough, since another student at Cornell has done it recently). Beyond that, it would be interesting and challenging to adapt this application -- that is, the Zen touchpad -- to larger displays and to color or grayscale displays. This would make the fading process much more realistic looking, but it would also require a much more powerful microcontroller. When I conceived of this project, I thought it would be wonderful as a large-scale art installation -- perhaps with a 50" or 100" flat screen. The touch surface ideally would be more like 8.5" x 11". Such a device might be installed as a mood device at a club, or simply on a city streetcorner so that passers-by could make transitory art for a large audience. My main regret about the current implementation, of course, is that it remains unfinished. Had I begun earlier and not underestimated the intricacies of the PS/2 protocol, I might have a working prototype now.
The code for my PS/2 implementation.
|
