Introduction
The Haptic Glove is a wearable device that simulates tactile sensations of virtual objects!
We created a glove that provides tactile feedback of virtual objects. When activated, it provides real feedback about the tactile sensation of holding a virtual object. Feedback includes temperature, weight, the size (diameter) of the object, and visual feedback will be provided using a Matlab interface. While this is just a demo, potential applications for a haptic glove includes use in virtual reality, gaming (like playing tennis the Wii, but you actually get to feel the ball hits that sweet spot!), online shopping, and physical therapy!
High Level Design
Inspiration
Although many people don't realize it, tactile feedback is as important as any of the other four major senses. It's not just about feeling a soft cloth or a furry pet, but the tactile sensations help us accomplish everyday tasks. Ever try tying your shoelaces while your hands are numb? Your hands haven't lost their dexterity, but without the tactile sensation, the task suddenly becomes a lot more difficult. We live in a world of strong dependence on computers and video games... with very little "real" feeling.
The original goal of this project was to make a haptic light saber (virtual sword). The idea was to have a sword (saber) fight wihtout the actual use of physical objects, but still being able to feel when your light saber hits another, when you get hit by one, and vice versa. Due to time and budget constraints however, that project became too overwhelming and we settled for the much humbler glove. But still, it brings virtual objects to life.
Background Math
Most of the calculations in this lab was done emperically. This includes the activation resistance of the flex sensor (the resistance that corresponds to object size), and the distance between the hall effect sensor and the table (based on the voltage output of the sensor).
For the feel of the temperature, we computed the heat flux of the surface of the objects based on the material of the virtual object. As an example, a glass bottle and a PVC pipe may be at the same temperature, but due to the difference in material conductivity, the perceived temperature of the two would not be the same, but this can be accounted for by computing the appropriate heat flux of each object. Heat flux is defined as the thermal conducitivy of the object multiplied by the temperature and ...
We ran some calculations to determine the strengh of the magnetic field that we would need to generate in order to mimick the effects of gravity, but that aspect of the project ended up not being implemented.
Calculations were done to determine the color of the object as presented in Matlab. To keep things simple, only two color scales were used: red to yellow for hot and blue to green for cold. The color scales were determined by setting a temperature threshold (around 30C). This threshold represents a user's skin temperature and objects that are higher temperature than this would be perceived as warm whereas objects at a lower temperature would be perceived as cold. Cold objects ran on a blue to green scale by setting a fixed value of blue and red and scaling the amount of green based on the temperature so that the end result is a more blue color for a colder object, and a warmer green color for a warmer object. On the other end of the scale, red and blue were held constant and green was changed to generate yellow for a warm object, orange for warmer, and red for a hot object.
Some calculations were involved in drawing the hands and the portraying the position of the object on the Matlab display. The input from the hall effect sensor was scaled by a factor of 0.01 to translate physical coordinates into screen coordinates and the size of the object (diameter) was scaled by a factor of 0.1 to fit onto the screen.
Logical Structure
We broke down the project into four major parts: size, temperature, gravity, and Matlab interface. The Matlab part consists of an MCU serial interface (to obtain object properties) and the graphics libraries obtained from Prof. Bruce Land. These graphics libraries have the ability to draw objects from basic 3D shapes as well as animate them in various ways. We use this as the visual feedback of our system where the user can see the object that he/she is holding, get information about the temperature of the object (based on the color), and see it be picked up.
Hardware/Software Tradeoffs
We had a lot of things that we had to compromise during the course of this project, and a lot of difficulties. Mostly, it involved difficulties in implementing the features using available hardware. The biggest tradeoff that we had to face was getting the gravity to work. We wanted a glove with a relatively free range of motion and something that didn't have to be tied down to the table, but finding something to simulate gravity was extremely difficult. We eventually decided on trying an electromagnet, but found it almost impossible to source a current strong enough to generate B-fields that were comparable to the pull of gravity. Our other idea was to use a servo motor and a spring and use a flex sensor to detect the dispalcement from the table, but the motion constraint that that puts the user under was not ideal either. In the end, we decided that our best option is to just forgo the weight simulation (much to our sadness).
We also had a bit of struggle with the temperature simulation. We chose to use two peltier devices (one on the index finger and the other on the thumb), to simulate temperature. Peltier devices convert electrical current to heat flux and has the ability to both cool and heat (one side will be cool and the other will be hot). We did not implement both due to the lack of appropriate hardware for an H-bridge, but the reality is that implementing both would be very difficult. The insulation on the peltier devices are not perfect, and the side that is heated severely overpowers the side that's cooled. As such, the device cannot sustain staying cool for long, and when turned off after simulating a cold object, the device gets hot enough to cause a great deal of discomfort on the user. Due to safety considerations, we decided to use the peltier devices for heating only, which we found to be silly, since we could have just done that with resistors for a much lower price.
We also faced the tradeoff between a powerful solenoid, or a a solenoid that's a manageable size, and along those lines, we had to tradeoff between having a "one-size-fits-all" glove or a tight-fitting glove that can only be used by people with hands of a certain size.
Design Standards
The major considerations we had to account for in our design involved heat (the danger of causing burns on the user's hand), physical forces used to simulate gravity (which may break the user's hand), and magnetic field radiation standards. Due to the weak nature of our solenoid motor (with forces of up to 6 Newtons) and the maximum temperature of the peltier device (-20C and 60C, both of which are outside the range of temperatures that can cause first degree burns, although 60C can be quit uncomfortable), we did not have much to worry about. The magnet that we used for determining distance was also far below danger levels as specified by the Radiation Protection Act of 1958.
Intellectual Property Considerations
Haptic devices are a popular field of research and haptic gloves that have the ability to interact with virtual environments are not new. Many companies market haptic devices including gloves (one of the most popular is the SensAble Technologies PHANTOM series). However, none of our implementation details (hardware, software, or the materials of the glove) used their patents or design or any other haptic glove patents that we could find. Our code for the Matlat visual feedback came from Bruce Land's Matlab graphics library.
Software and Hardware Design
The Software
Matlab Visualization
The code for the Matlab visual feedback was obtained from Bruce Land's website. The code used the graphics libary that he had already created to make objects from simple shapes and then animate them. For our simulation, all objects were cylindrical (cups), were placed on a table, and had the option of being either full or empty. The Matlab was interfaced with the MCU and obtained parameters about the object properties and what the user is doing with it to draw the object on screen. Two cylinders (stick-looking things) were drawn to simulate hands and appears to hold onto the object when the user is gripping it. The object then moves up and down on the screen depending on how much the user is moving it. We did not implement motion in the x and y axes or rotational motion.
The Hardware
The Glove
We used a soccer goalkeeper glove for our project due to its durability and relatively small range of motion.
Temperature Simulation
The temperature simulation used two thermoelectric (peltier) devices. These devices take in an input and translates the current into heat flux. One side of the device will be heated whereas the other will be cooled. The devices were connected to a 5V power supply and the MCU and were placed on the glove's index finger and thumb. We used the MCU's internal reference to use as our reference for the MCU to determine the heat flux that we want.
Size Simulation
We simulated the size (diameter) of the objects that we were picking up using a solenoid motor and two retractable ID holders. When the object is gripped, the solenoid activates and clamps the strings (attached to the fingers) to prevent the fingers from further clenching. A flex sensor was placed at the finger and the thumb and detected the closing of the hand so that the solenoid can activate when the hand is gripping at the desired diameter. This is accomplished by the changing resistance of the flex sensor as it is extended. We measured that change in resistance and translated it into a diameter. The solenoid motor was separated from the MCU using an optoisolator, which we found that may not have been necessary.
Distance from the Table
The distance between the user's hand and the table was measured using a hall effect sensor and a hard drive magnet. Hall effect sensors work by translating the strength of a magnetic field to a voltage output. The MCU reads in the voltage and determines the distance that the hand is from the table and sends the information to Matlab for the visual feedback of the object being picked up.
For more details, please see the hardware schematic.
Problems Encountered
There were a lot of problems and undesired outcomes for just about every aspect of the project. For starters, we had difficulties mounting the equipment onto our original glove (a common gardening glove) due to the loose nature of it. The glove allowed too much movement, meaning that the solenoid could not clamp the ID holders properly and the holders themselves had trouble staying on the glove. We ended up getting rid of those gloves and using a soccer goalkeeper's glove which was much stiffer and provided much better mounting space for the hardware.
The peltier devices did not work as perfectly as we had hoped that they would. The two surfaces of the device are separated by a weakly insulating material, which means that the hot and cold sides do not always stay that way. In fact, we found that the hote side severely overpowered the cold side and made for simulation of both heating and cooling impossible. In addition, after a hosrt period of cooling, the other side would get so hot and transfer the heat to the cool side that it actually became uncomfortable for the wearer. As a result, we decided to use the peltier devices for heating only.
We originally intended to simulate gravity by using an electromagnet, but the weak forces produced by the best magnet we could make made that unrealistic. We also considered the option of using a motor with slip, or a servo motor with springs, but those are both difficult to program, severely limits the range of motion of the hand, and run the possibility of injuring the user.
Size simulation did not work perfectly since the solenoid we had was so weak. However, a stronger solenoid motor meant that the size and weight of it would increase as well, which makes mounting onto the hand inpractical.
Results
Speed and Accuracy
Safety
As mentioned previously, the biggest safety considerations we had was the wrenching out the user's hand using a motor or causing discomfort to the user when the peltier gets too hot. We solved those by not using a motor for gravity simulation (and not doing gravity simulation at all) and using the peltier to heat rather than cool so that we could control the heating of it to prevent injuries.
Interference with Others
Our design did not involve any wireless components, and therefore did not interefere with anyone else's work in that manner. It was also sound free and did not disturb any other users.
Usability
Due to constraints of the glove (tight-fitting, stiff), we had to choose a glove that could only fit the hand of one of the group members. We chose Corey as the model, and since he has bigger hands, Ti and others who have small hands will not be able to use this glove, at least not to its full abilities.
Conclusion
Our Expectations
This project worked mostly according to our expectations. We were rather upset that we could not get the gravity simulation to work and that the solenoid motor was too weak that it didn't always work properly. However, getting the visual feedback on Matlab to work far exceeded oru expecations, and we are very happy about that. If we were to do this project again, we would have put more thought into planning out the design of the glove, what kind of glove we need, and where to mount the parts for optimal performance. We would also have been more careful in the design of the project and done more research about our components before we decided to implement them. This would have avoided the problem withthe peltier devices, and we may have been able to find a better compromise between the size and the power of the solenoid motor. We would also put more consideration into how to implement the gravity, which turned out to be a lot more difficult than we had expected. Ideally, if we were to do this again, we would also find ways to make the glove wireless to allow full freedom of motion and incorporate that into our visual feedback as well.
IP Considerations
We did not use any code from public domains, but did use Bruce Land's code for the Matlab visual feedback. We do not believe this project has patent opportunities since many patents and products exist for haptic gloves. We doubt, though, that any of them uses friendly soccer gloves or are held together with as much glue and tape as ours, so perhaps we could patent our design as the "questionable glove".
Ethical Considerations
We adhered to all the points of the IEEE Code of Ethics for our design. In every aspect of it, we considered safety to be our top priority and made absolute certainty that users or anyone in the vicinity might get injured from our project. We were careful in using soldering equipment, the hot glue gun, and the peltier devices. When we have had to use the hot glue gun on the glove while it was on Corey's hand, we made sure to put insulating material between him and the glue to make sure he did not sustain any burns.
We did not, as part of the project, accept any briberies (although nobody offered us any anyways).
We did our best to estimate our abilities and what we could accomplish with the available materials and time frame, and did so very honestly. We did find throughout the course of the project, that we often tended to overestimate ourselves, and underestimate the amount of work and time that goes into many tasks. However, we were very open to suggestions and criticism, and found that this experience greatly helped us impove our understanding of many technologies and greatly impoved our technical competence.
Our project was very different from projects of other groups, but when the opportunity came up, we were willing to help other students on their tasks or problems, and likewise, we received a lot of help ourselves. We also tried to be as considerate as possible of other groups when working on our project, which mostly involved finding space to solder or use the hot glue gun.
Appendices
This project had a budget of $75 including many borrowed parts from the lab but not including surplus or donated parts.
Program Listings
Two programs were written for this project. The code is available in plain text.
Schematics
Hardware and software schematics are provided in PDF form.
Cost Details
This project had a budget of $75 including many borrowed parts from the lab but not including surplus or donated parts.
| Parts List | Unit Price | Qty | Total | Source |
|---|---|---|---|---|
| Total | -- | -- | $63.11 | -- |
| Flex Sensors | $0.00 | 2 | $0.00 | Surplus |
| Right Soccer Keeper Glove | $7.00 | 1 | $7.00 | Dick's Sporting Goods |
| Guide Straws | $0.00 | 2 | $0.00 | Surplus |
| STK-500 with Mega-32 | $15.00 | 1 | $15.00 | Lab |
| Connectors | $0.00 | 4 | $0.00 | Lab |
| Peltier (allelectronics # PJT-7) | $15.18 | 2 | $30.36 | Digikey (102-1663-ND) |
| Hall Effect Sensor | $0.50 | 1 | $0.50 | All Electronics (#SS-19T) |
| Header Pins | $0.05 | 45 | $2.25 | Lab |
| Jumper | $0.00 | 1 | $0.00 | Lab |
| 12V, 200W Computer Power Supply | $0.00 | 1 | $0.00 | ACCEL |
| 5V, 1W Power Supply | $0.00 | 1 | $0.00 | Lab |
| Transistors (Buz-73) | $0.00 | 2 | $0.00 | Lab |
| Resitors, Capacitors, Wires, Diode | $0.00 | -- | $0.00 | Lab |
| Power Interfaces | $0.00 | 2 | $0.00 | Lab |
| Retractable ID Holders | $1.50 | 2 | $3.00 | Lowes |
Tasks
Both partners collaborated on all aspects of the project, but Corey Siegel was responsible for most of the software, Ti Zhao was responsible for most of the Matlab work and the design of the glove, and both partners are equally responsible for all aspects of the hardware.
References
More Information
The links above and below link to the details of our design as well as sources of interest references.
Related Resources
The glove (a modified soccer goalie glove) with the peltier devices plugged in. The Hall effect sensor is barely visible on the bottom of the glove. The round black object on the wrist is one of the retractable strings for the fingers.
Another shot of the finished glove.
Our TA Rob trying it out.
The circuit, comprised of the STK-500 (with ATMega32), small solderboard, medium solderboard, and glove.
This is what Matlab displays when a warm object is not being held.
A warm object is being held some distance above the table surface.
A cool object is denoted with blue instead of red/orange. A neutral object is colored yellow.