Snake Arm Glove Project

 

Karan Khanna (kk364)

Amit Penmetcha (ap328)

 

Introduction

For our project, we designed a glove that can be used to control the Cornell University robotic Snake arm– thereby enabling a surgeon to remotely operate the snakearm as a colonoscope in conjunction with a vision guide system (aka TV goggles). The glove allows for movement of the arm's segments as a whole and individually.

 

IMG_0167

 

More about the SnakeArm

A snake arm emulates the movement of a snake, with one point of attachment where the arm is anchored. It has multiple degrees of freedom, high flexibility, and thin diameter.

The current generation snakearm has been designed to be used in surgical applications – particularly colonoscopy. The need for a friendly control method emerged earlier in the course of our team efforts. The motion sensing glove is designed to meet this need by controlling the servo motors on the snakearm indirectly through the PC based on its movement.  The glove is composed of three types of sensors – flex sensors, a two axis accelerometer and several hall effect sensors. The flex sensors are used to select modes of operation, the accelometer is used to generally move the snakearm while the hall effect sensors are designed to enable precise control of the head

 

The vision guide system is the part that makes this project more appealing to surgeons. There is a camera mounted on the head of the snakearm itself and the camera is connected to TV goggles. So, the surgeon is able to control the snakearm with the glove while using the vision guide system to see inside the colon of the person being operation on. This is very much like playing a video game and this glove based system has been designed to enable remote colonoscopy surgeries.

 

For more information on the current snake arm capability, please see this youtube video.

http://www.youtube.com/watch?v=_ByB35gTuuI

 

II. High Level Design

Our idea came from our involvement with the snake arm team and the wish to control it in a more logical fashion. Having it mimic the movement of a person's arm would allow them to have a more complete and powerful control over the arm allowing them to move it exactly as they please. The snake arm team is currently developing the snake arm for a surgical application, thus, accurate control is very important.

The project consists of the Mega32 serving as an interface between the hardware and the snake arm interface. Using an accelerometer we determined the user's hand motion on the x and y plane, which controlled the base of the snake arm. The tilt determined the arm's movement vertically, while the pitch/roll determined the movement of the arm horizontally. The flex sensors were used to activate and deactivate the hall effect sensors, which were used to control the head of the snake arm. The flex sensors and accelerometers went into the ADC of the Mega32, while the hall effect sensors go into PORTB. The inputs from the hardware components are calibrated and reformatted into two bytes that are sent across the serial port to a java interface that manipulates the snake arm.

One of the questions we asked ourselves earlier in the project was whether we should use send the data from the glove through the ATmega32 to the PC (hence snakearm GUI) or connect our ATmega32 directly to the snakearm servo motors and send out PWM signals to move them. However, keeping in mind that the snakearm is quite complex – just four segments of it amount to 12 motors and each motor has to moved in a perfect puppetering action with regard to the other motors to move a segment in a simple direction. Since a control system was already implemented for the snakearm in Java, it made no sense not use it and try to reinvent the wheel.

 

III. Program/Hardware Design

Hardware Details

The hardware we used consisted of the STK500 programming board, Mega32 microcontroller, a two axis accelerometer (Freescale MMA6261Q 1.5g accelerometer), 5 flexsensors (FLX01), and 8 Allegro 1101 Hall Effect Sensors. We also used a Freescale 9V battery to 3V converter, which served as the supply voltage to the accelerometer. The accelerometer's x and y axis outputs tied into Pin A.0 and A.1. The flex sensors and hall effect sensors were both inputted into LM258/LM358 opamps and used a non-inverted style setup to amplify their voltage The hall effect sensors outputted digital signals that were amplified to 3.6 V, which is logic high. The sensors detect fluctuations in magnetic fields, so when a magnet is held next to one, the sensor goes low. The flex sensors also went into opamps to get a gain into a range of .5 - 2 volts, based on how much the sensor was bent. The greater the degree of bending the lower the output voltage. The output voltage is determined based on the equation Vin * R1 / (R1 + R2), where R1 is the other input resistor to the non-inverting terminal.

Figure 2: ATMega32 Connections

Figure 3: Flex Sensor circuit diagram

Figure 4: Accelerometer Circuit Diagram

Figure 5: Hall Effect Sensor Circuit Diagram

 

Figure 6: Hall effect sensor direction diagram

 with respect to sensor number

IMG_0164

Hall Effect Sensors

                                                                         

Glove with fingers stretched out:                                                                                                        Glove in a fist form: Accelerometer

Accelerometer on and hall effect sensors off                                                                                        off and hall effect sensors on


Program Details

We had one task that handled all of the inputs to the ADC and activity of the hall effect sensors. We first read the x-axis on the accelerometer from the ADC and then the y-axis(on the second iteration of the task) by incrementing the ADMUX variable. In the case that this was the first reading, these inputs were used to calibrate the readings from then on. We then got readings on the flex sensors in one iteration, by using a for loop that constantly incremented the ADMUX and polled the next port. We compared the reading on the flex sensor to a reference value. If it was less than that we determined that the flex sensor was being bent significantly. We used an array to store the pattern of flex sensors that were being flexed.

To actually get readings from the ADC, we used an ADC interrupt that gave us the latest conversion whenever it was ready. We used a sleep command when making the ADC read the next port, so that we could give it enough time to make the conversion to an accurate value. We noticed early on that we were getting random values from the ADC and after consulting the websites for previous mouse type projects we found that the sleep command could be used to help give the ADC ample time to make the conversion.

The id of the hall effect sensor that was activated was only transmitted when all of the flex sensors were being bent. Another interrupt was used to determine which of the flex sensors were being activated.

Sending the data across the serial port was done in three bytes. The first byte was merely a buffer byte, so that the Java interface could determine the order in which the bytes were being sent. The second byte consisted of the flex sensor array and whether or not the hall effect sensors had be activated. The third byte consisted of the reading from the accelerometer or the identity of the hall effect sensor that was being used. The upper nibble of the byte stored a 3 bit value that ranged from 0 to 4, which determined how quickly the user wanted to turn on the x-axis. The msb bit was used to determine direction. The lower nibble contained identical information on the y-axis. The byte would contain the ID of the hall effect sensor that was activated if they were being used. Our java interface opened a line of communication with the microcontroller and polled the serial port for data. We used queues to handle the quick stream of data and extracted the necessary data from them, before sending them to the snake arm interface. The snake arm is controlled by a program that communicates to the servo motors via the ATiny26. The program gets feedback from the ATiny26, so we obtained the current position of the snake arm and updated to where we would like it to move based on our readings from the accelerometer and hall effect sensors. This last part of interfacing was done with the assistance of one of the CS members of the snake arm team. The code for the microcontroller and serial interface with microcontroller is provided. However the snake arm interface could not be provided and is not really necessary for the user to see anyway.

1st byte - 1100000000 (buffer)
2nd byte - 2 bits - not used, 5 bits (Flex sensor array) 1 bit (Hall Effect or Accelerometer)
3rd byte - 4 bits x axis acceleration 4 bits y axis acceleration or 8 bit id of hall effect activated


These three bytes are sent through the UART (serial) interface to the computer where a Java program picks them up (GloveCommunicator.java) and interprets the data. A program runs as a new thread so that it takes up less CPU power while polling the COM port as necessary. Next, this data from this program (sensor positions) are extracted by the CU SnakeArm GUI which runs them through its programmed control system and moves the servo motors controlling the snakearm accordingly.

 

Looking back at the program implementation sector of our project, the serial communication and interfacing with the snakearm GUI and control system proved to be quite tricky. There were numerous errors that we encountered from bad values, inconsistent formats, etc. Moreover, integration this into an existing yet functioning platform proved to be demanding.

 

IV. Results of the Design

We adhered to the standard format of communication over the serial port. We used a baud rate of 9600 bps, sent 8 data bit packets, had 1 stop bit, and no flow control or parity bits. Although we were receiving at this rate on the computer end, we only handled data every half a second, dropping all other packets. The reason for this was to avoid overdriving the servo motor. The response went down as a result, but is still acceptable. This was needed for the safety of the snake arm. We also enforced safety in the glove by keeping the flex sensors within one of the layers of the glove and keeping the accelerometer on small breadboard on top of the glove to give it a proper shielding from any static that could form on the glove. Our project does not interfere with any other devices in any way. Overall, the snakearm does respond to our glove and the sensor readings on the GUI correspond properly to our sensors. However, the movement is not very smooth. This is a restriction from the existing control system which we did not create or modify in this project.

 

V. Conclusions

 

Our results mostly met our expectations. We were able to communicate the hardware and our software talks to the snake arm extremely effectively. Synching up our project with the snake arm interface is almost flawless, and we have maintained an efficient scheme for running a constant poll on the serial port that does not slow the CPU. We had to force a delay between the users control movements and the snake arm's response. We purposely drop many of the users commands, so that the snake arm is not overloaded with movement requests, which results in a substantial delay. Considering the snake arm is small and can break easily, this was a necessary flaw we had to keep in the project. The movement of the snake arm matches the general movements of the glove, but the overall snake arm's control algorithm has yet to be perfected. As a result, the snake arm is able to move in the general direction that the user picks, based on both the accelerometer and the hall effect sensors, but the snake arm starts to bend in the direction it is moving, causing it to form awkward positions that are not consistent with the user's commands. The glove serves as a suitable replacement for the current form of control, thus, we accomplished our primary goal of having a solid control scheme for the snake arm. In the future we would like to use a 3 axis accelerometer and or a different scheme that allows the user to move his/her entire arm in which the snake arm will follow. However this relies on our fine tuning the current control algorithm for the snake arm. Also the hall effect sensors were quite difficult to use, as we had to have the magnet at an exact orientation to activate them. In the future we will try using some other mechanism to give us an additional level of control over the snake arm. The other good part about our design is that it is very modular. Reassigning the function of a particular sensor is easy since all the sensor readings are sent to the Java program.

Intellectual Property and Ethical Standards

Our project fully complies with the IEEE Code of Ethics. Our glove controller was designed with safety in mind and is environmentally friendly and safe. The snake arm is meant to be used for beneficial purposes. We generated all of our code on our own, but when troubleshooting, we used the sleep command mentioned by a previous year's project. As mentioned previously, we used the snake arm's current interface and control algorithm and received help from a member of the CS team to help us communicate with the snake arm, due to our lack of knowledge of how the interface worked. We did research on how to communicate across the serial port using Java, and wrote our program based on the tutorials available and the style that the snake arm team wished us to follow. This idea is not completely new, however we have made a device for a fairly unique purpose. We did not accept any forms of bribery nor did we offer any bribes. We welcomed any help from Professor Land, TAs, or any other students, although we tried to remain as independent as possible. We offered advice to others who were also using accelerometers and helped those who were in need.

 

Legal Considerations

We did not use RF transmitters or any other components that can cause interference with other devices, thus there were no legal issues that we had to consider.

 

 

Appendix:

 

Cost:

Part

Cost

Part Number

Stk500 + Mega32

Free (SnakeArm team provided)

Not needed

Flex Sensor (5)

$12.95 * 5 = $64.75

Imagesco: FLX-01

Accelerometer

Free (Sampled from FreeScale)

FreeScale: MMA7260QT

OpAmps

11*$1.00 (Lab Fee) = $11.00

LM358

Glove

$1.00

Not Needed

Hall Effect Sensors (8)

Free (sampled from Allegro)

Allegro A1101EUA-T

Jumper Cables

2*$1.00 = 2.00

Not Needed

Several Breadboards

Free (SnakeArm provided)

Not Needed

Freescale 9V to 3V battery converter

Free(sampled from freescale)

Not Needed

Total

$78.75

 

Note: since this is a project team related project, our budget was waived.

 

Breakup of Work

Testing and designing hardware

(Flexsensors, Accelerometer, Hall Effect Sensors): Karan & Amit

Microcontroller software in C: Karan & Amit

Java Serial Interface: Karan & Amit

Integration with SnakeArm: Karan & Amit with help from Bryant Williams

Webpage: Karan & Amit

 

Code

Microcontroller C

Java Serial Communication

 

Data Sheets

LM358 Opamp

A1101 Hall effect sensor

FLX-01 Flex sensor

MMA6260Q Accelerometer

 

Links

Cornell University Snake Arm Project team

Previous year’s 476 project