Haptic teleoperation device for MRI guided biopsies
This is the last project I worked on during my PhD. The device was specifically designed for prostate biopsies inside an MRI machine. It has haptic feedback, it’s teleoperated, and it’s completely passive. This device utilizes MR-safe components, therefore, it does not interfere with the magnetic field produced by the MRI machine.
The robot has 2 degrees-of-freedom which are required for a prostate biopsy procedure. The two motions are: translation along the needle insertion axis, and rotation about the insertion axis, which is used to steer the needle tip as it passes through tissue mainly to keep it straight.
The input and output side of the robot are identical. In order to achieve the compounded linear and rotary motion for the needle, I created a rotating linear slide with a hexagonal linear rail and bearing carts to allow for smooth linear motion.
I analyzed the robot’s transparency by doing position tracking of both degrees-of-freedom. An encoder was attached to the location indicated on the picture on both the input and output side. Also, the robot was scanned in the MRI machine to evaluate it’s MR compatibility.
Non-metallic hydrostatic actuators
These actuators are made out of plastic and composite materials. Most of the components on the actuators are 3D printed with the exception of fasteners, rods, and hydraulic fittings. When compared to other actuators, they are incredibly light-weight and cost effective. They are based on pair of rolling diaphragm seals and can be used for any application that requires non ferrous materials and haptic feedback. This actuator technology is the foundation of all the robots on this page.
The linear actuator has a middle output, a stroke of 25 mm and weights 80 g. It’s made of 3D printed materials for its body, garolite G-10 rods for its shafts and standoffs, and RENY fasteners.
Removing air from the hydraulic line can be challenging in hydrostatic systems. Eliminating air in the transmission is critical to haptic feedback, because air functions as a compressible spring, reducing stiffness. To aid in the filling and air bleeding process I added an extra port to the water-side bonnet. This bleed port allows water to be cycled through the entire water transmission line, facilitating the removal of air bubbles.
The rotary actuator has a range of motion of 135 degree and weights 100g. It has a custom rex shaft output. This feature allows for positive engagement of clamped pieces.
Both actuators were scanned in an MRI machine to see if they would interfere with the scanner by producing imaging artifacts. Figure A represents the actuator and tissue phantom scanned at the same time. Figure B represents a control scan with just the tissue phantom. After the scans it was concluded that there were negligible imaging artifacts.
XPRIZE AVATAR 3rd place finalist
Team northeastern place 3rd in the worldwide XPRIZE competition winning 1 million dollars. We created a telepresence robot with intuitive controls for the operator. The robot uses the same technology as the haptic teleoperation device and haptic testbed to replicate high fidelity sense of touch.
The competition tasks included interacting with a person to acquire directions, picking up a bottle and inserting it in a circular hole, using a drill to remove a screw, and sensing the roughness of many rocks and selecting the roughest one.
The robot has 2 custom hands with 3 fingers each, two panda arms, and a base with mechanum wheels.
The control side has 2 exo-gloves to control the hands and arms of the robot, a foot plate to control the locomotion, and 2 display screens to observe the environment.
Haptic testbed
This robot was constructed for remote needle biopsy inside the MRI room, it is made out of non-ferrous materials, and has excellent haptic feedback. The system has been configured for fully passive benchtop testing. The system’s haptic capabilities and high transparency arise from its high series stiffness, lightweight, and low friction. The device relies on the MR-conditional hydrostatic rotary actuators that use rolling diaphragms similar to the non-metallic hydrostatic actuators. The robot also provides bilateral actuation which makes it safer for human contact.
The actuator used on this robot is made of non-ferrous material. It has machined aluminum parts for its body, titanium and brass fasteners, ceramic ball bearings, and Kevlar reinforced belts. The actuator has 135 degrees of motions and weights 120g. The timing belt pulley has an integrated 8 mm output shaft.
The robot has 3 degrees-of-freedom with a range of motion of 440mm in the needle insertion direction. Each degree-of-freedom requires 2 actuators, one for the input and one for the output. All of its actuators are placed close to the base to reduce inertia. In order to do this, I created a parallel mechanism made out of carbon fiber and 3D printed parts.
The robot was equipped with counterweights to make it more user friendly and less fatigue prone. Also, having counterweights makes it safer for clinical use as the physician can let go of the device and the needle won’t move.
I analyzed the transparency o the system by recording the forced applied with a torque sensor and reading the pressure recorded by a pressure sensor. Additionally, I performed a needle puncture test on a phantom tissue with and without a flesh-like membrane.
This robot can be used for more than just medical applications. It can be used for other tasks that requires high fidelity haptic feedback in remote manipulation, for example straight razor shaving.
In straight razor shaving, haptic feedback is essential because the barber must be able to feel the client’s skin and react to the contours of it immediately. The barber requires precise motion and force control in order to not break the client’s skin. Shaving requires a teleoperated device with haptic feedback, backdrivability, and bidirectionality. I decided to use the 3-DOF haptic testbed for straight razor shaving to test its haptic feedback capabilities.
User Study
The purpose of this user study was to explore the role of haptic feedback in two different applications: (1) remote palpation, and (2) remote needle insertion. An operator’s capacity to recognize crucial cues that appear during biopsy operations, such as input force spikes as the needle punctures an organ or tumor were assessed, as well as an operator’s capacity to recognize palpation cues, such as recognizing force changes as the end effector presses different tissues. The user study evaluates the teleoperated device by comparing the tasks the users were asked to perform in two modalities: by hand (holding the output) and through the haptic device (holding the input). The participant never directly touches or sees the phantom.
The first task in the user study was determining the stiffness of three different soft tissue phantoms and arranging them in order of increasing hardness. The second task was identifying a hard nodule inside a soft phantom. All the phantoms were made from silicone rubber. All the phantom have a diameter of 60 mm and a height of 20 mm. The nodule can be found 10 mm under the surface of the phantom.
Needle insertion phantoms were developed to simulate the forces experienced during prostate biopsy procedures. The phantom consisted of three parts: two parts of the same material, separated by a membrane. I chose a 7% concentration of bovine gelatin for the phantom. The membrane was made of a 0.2 mm thick silicone layer with an embedded tissue. The membranes were placed in the gelatin tissue phantom at 2 depths: 50 mm and 75 mm. In addition, they are installed in between the gelatin phantoms under minor tension to make a defined plane.
In the last task of the user study participants were instructed to insert a biopsy needle into phantom tissue with an embedded membrane. The phantom is occluded from the user at all times. The users were asked to state verbally when they had detected and pierced the membrane.
The palpation tasks yielded similar results for both modalities. All the users were able to identify the nodule at least once. Only one participant was not able to identify the nodule in one of their trials
The graph is the result from one participant palpating the different stiffness phantoms using the teleoperation device. Displacement and force are measured on the z-axis. The first phantom provided to them was the stiffest. In the graph, it can be seen that the 00-70 shore hardness phantom deflects the least and was subjected to the greatest applied force. While waiting for the next phantom, the participant lifts the end effector with no force applied. Therefore, displacement increases as force goes to zero.
The needle insertion task, where the user was asked to locate the membrane embedded in a bovine gelatin phantom tissue, had similar results in the two study modalities. 60% of the time, participants correctly identified the membrane by hand (holding the output side of the device), while correctly identifying it 57% of the time through the device.
Participants were asked to perform the tests in two modalities by hand and through the device. The users were asked to wear noise canceling headphones and look away during the tasks so they could focus on the haptic feedback only.
XPRIZE AVATAR 3rd place semi-finalist
Team Northeastern place 3rd in the semifinal, becoming the top ranking US team. We developed a simple yet capable system for the competition. Our robot performed all of the tasks significantly faster than all other teams. This prompted many judges to come to our booth to try out the robot for themselves.
The competition tasks included solving a simple kid’s puzzle, interacting with a person, feeling and describing the texture of a vase, and picking up a wine glass to cheers with another person.
We developed a haptic 2-DOF gripper with linear hydrostatic actuators that attaches to a UR5 arm and it’s mounted on a husky base. The use interface has a modified gripper that attaches to the user’s hand and a simple arm with encoders at the joints to control the UR5. The user also has 3 camera views of the robot, one top down to aid with manipulation, a straight isometric for reference, and a wide angle view for locomotion.
