Publications

2020

Guido Caccianiga ; Andrea Mariani ; Elena De Momi ; Gabriela Cantarero ; Jeremy D. Brown
An Evaluation of Inanimate and Virtual Reality Training for Psychomotor Skill Development in Robot-Assisted Minimally Invasive Surgery
IEEE Transactions on Medical Robotics and Bionics ( Volume: 2 , Issue: 2 , May 2020 )

Robot-assisted minimally invasive surgery (RAMIS) is gaining widespread adoption in many surgical specialties, despite the lack of a standardized training curriculum. Current training approaches rely heavily on virtual reality simulators, in particular for basic psychomotor and visuomotor skill development. It is not clear, however, whether training in virtual reality is equivalent to inanimate model training. In this manuscript, we seek to compare virtual reality training to inanimate model training, with regard to skill learning and skill transfer. Using a custom-developed needle-driving training task with inanimate and virtual analogs, we investigated the extent to which N=18 participants improved their skill on a given platform post-training, and transferred that skill to the opposite platform. Results indicate that the two approaches are not equivalent, with more salient skill transfer after inanimate training than virtual training. These findings support the claim that training with real physical models is the gold standard, and suggest more inanimate model training be incorporated into training curricula for early psychomotor skill development.

@ARTICLE{9080087, author={G. {Caccianiga} and A. {Mariani} and E. {De Momi} and G. {Cantarero} and J. D. {Brown}}, journal={IEEE Transactions on Medical Robotics and Bionics}, title={An Evaluation of Inanimate and Virtual Reality Training for Psychomotor Skill Development in Robot-Assisted Minimally Invasive Surgery}, year={2020}, volume={2}, number={2}, pages={118-129},}

Mohit Singhala; Jeremy D. Brown
Prefatory study of the effects of exploration dynamics on stiffness perception
2020 IEEE Haptics Symposium (HAPTICS)

The utility of telerobotic systems is driven in large part by the quality of feedback they provide to the operator. While the dynamic interaction between a robot and the environment can often be sensed or modeled, the dynamic coupling at the human-robot interface is often overlooked. Improving dexterous manipulation through telerobots will require careful consideration of human haptic perception as it relates to human exploration dynamics at the telerobotic interface. In this manuscript, we use exploration velocity as a means of controlling the operator’s exploration dynamics, and present results from two stiffness discrimination experiments designed to investigate the effects of exploration velocity on stiffness perception. The results indicate that stiffness percepts vary differently for different exploration velocities on an individual level, however, no consistent trends were found across all participants. These results suggest that exploration dynamics can affect the quality of haptic interactions through telerobotic interfaces, and also reflect the need to study the underlying mechanisms that cause our perception to vary with our choice of exploration strategy.

@INPROCEEDINGS{9086307, author={M. {Singhala} and J. D. {Brown}}, booktitle={2020 IEEE Haptics Symposium (HAPTICS)}, title={Prefatory study of the effects of exploration dynamics on stiffness perception*}, year={2020}, volume={}, number={}, pages={128-133},}

2019

Neha Thomas, Garrett Ung, Colette McGarvey & Jeremy D. Brown
Comparison of vibrotactile and joint-torque feedback in a myoelectric upper-limb prosthesis
Published: 11 June 2019 in the Journal of NeuroEngineering and Rehabilitation

https://doi.org/10.1186/s12984-019-0545-5

2017

Brown, Jeremy D. ; Fernandez, Joshua N. ; Cohen, Sean P. ; Kuchenbecker, Katherine J.
A wrist-squeezing force-feedback system for robotic surgery training
2017 IEEE World Haptics Conference (WHC), 107-112

Over time, surgical trainees learn to compensate for the lack of haptic feedback in commercial robotic minimally invasive surgical systems. Incorporating touch cues into robotic surgery training could potentially shorten this learning process if the benefits of haptic feedback were sustained after it is removed. In this paper, we develop a wrist-squeezing haptic feedback system and evaluate whether it holds the potential to train novice da Vinci users to reduce the force they exert on a bimanual inanimate training task. Subjects were randomly divided into two groups according to a multiple baseline experimental design. Each of the ten participants moved a ring along a curved wire nine times while the haptic feedback was conditionally withheld, provided, and withheld again. The realtime tactile feedback of applied force magnitude significantly reduced the integral of the force produced by the da Vinci tools on the task materials, and this result remained even when the haptic feedback was removed. Overall, our findings suggest that wrist-squeezing force feedback can play an essential role in helping novice trainees learn to minimize the force they exert with a surgical robot.

@INPROCEEDINGS{7989885, author={J. D. {Brown} and J. N. {Fernandez} and S. P. {Cohen} and K. J. {Kuchenbecker}}, booktitle={2017 IEEE World Haptics Conference (WHC)}, title={A wrist-squeezing force-feedback system for robotic surgery training}, year={2017}, volume={}, number={}, pages={107-112},}

Brown, Jeremy D. ; O’Brien, Conor E. ; Leung, Sarah C. ;Dumon, Kristoffel R. ; Lee, David I. ; Kuchenbecher, Katherine J.
Using Contact Forces and Robot Arm Accelerations to Automatically Rate Surgeon Skill at Peg Transfer
IEEE Transactions on Biomedical Engineering ( Volume: 64 , Issue: 9 , Sept. 2017 )

Objective: Most trainees begin learning robotic minimally invasive surgery by performing inanimate practice tasks with clinical robots such as the Intuitive Surgical da Vinci. Expert surgeons are commonly asked to evaluate these performances using standardized five-point rating scales, but doing such ratings is time consuming, tedious, and somewhat subjective. This paper presents an automatic skill evaluation system that analyzes only the contact force with the task materials, the broad-bandwidth accelerations of the robotic instruments and camera, and the task completion time. Methods: We recruited N = 38 participants of varying skill in robotic surgery to perform three trials of peg transfer with a da Vinci Standard robot instrumented with our Smart Task Board. After calibration, three individuals rated these trials on five domains of the Global Evaluative Assessment of Robotic Skill (GEARS) structured assessment tool, providing ground-truth labels for regression and classification machine learning algorithms that predict GEARS scores based on the recorded force, acceleration, and time signals. Results: Both machine learning approaches produced scores on the reserved testing sets that were in good to excellent agreement with the human raters, even when the force information was not considered. Furthermore, regression predicted GEARS scores more accurately and efficiently than classification. Conclusion: A surgeon’s skill at robotic peg transfer can be reliably rated via regression using features gathered from force, acceleration, and time sensors external to the robot. Significance: We expect improved trainee learning as a result of providing these automatic skill ratings during inanimate task practice on a surgical robot.

@ARTICLE{7765041, author={J. D. {Brown} and C. E. {O’Brien} and S. C. {Leung} and K. R. {Dumon} and D. I. {Lee} and K. J. {Kuchenbecker}}, journal={IEEE Transactions on Biomedical Engineering}, title={Using Contact Forces and Robot Arm Accelerations to Automatically Rate Surgeon Skill at Peg Transfer}, year={2017}, volume={64}, number={9}, pages={2263-2275},}

2016

Brown, Jeremy D; O’Brien, Conor E; Leung, Sarah C; Dumon, Kristoffel R; Lee, David I; Kuchenbecker, Katherine J
Using Physical Interaction Data of Peg Transfer Task to Automatically Rate Trainee’s Skill in Robotic Surgery. [Journal Article]
Accepted for publication in IEEE Transactions on Biomedical Engineering, 2016.

@article{BrownTBME2015,
title = {Using Physical Interaction Data of Peg Transfer Task to Automatically Rate Trainee’s Skill in Robotic Surgery.},
author = { Jeremy D. Brown and Conor E. O’Brien and Sarah C. Leung and Kristoffel R. Dumon and David I. Lee and Katherine J. Kuchenbecker},
url = {http://ieeexplore.ieee.org/document/7765041/},
year = {2016},
date = {2016-12-01},
journal = {Accepted for publication in IEEE Transactions on Biomedical Engineering},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

Brown, Jeremy D; Ibrahim, Mary; Chase, Elyse D Z; Pacchierotti, Claudio; Kuchenbecker, Katherine J
Data-driven comparison of four cutaneous displays for pinching palpation in robotic surgery Inproceedings
2016 IEEE Haptics Symposium (HAPTICS), pp. 147–154, IEEE, 2016, ISSN: 23247355.

@inproceedings{BrownHS2016,
title = {Data-driven comparison of four cutaneous displays for pinching palpation in robotic surgery},
author = { Jeremy D. Brown and Mary Ibrahim and Elyse D. Z. Chase and Claudio Pacchierotti and Katherine J. Kuchenbecker},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7463169},
doi = {10.1109/HAPTICS.2016.7463169},
issn = {23247355},
year = {2016},
date = {2016-04-01},
booktitle = {2016 IEEE Haptics Symposium (HAPTICS)},
pages = {147–154},
publisher = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

Brown, Jeremy D; Kunz, Timothy; Gardner, Duane; Shelley, Mackenzie K; Davis, Alicia J; Gillespie, Brent R; Gillespie, Brent
An Empirical Evaluation of Force Feedback in Body-Powered Prostheses Journal Article
IEEE Transactions on Neural Systems and Rehabilitation Engineering, PP (99), 2016, ISSN: 1558-0210.

Myoelectric prostheses have many advantages over body-powered prostheses, yet the absence of sensory feedback in myoelectric devices is one reason body-powered devices are often preferred by amputees. While considerable progress has been made in the mechanical design and control of myoelectric prostheses, research on haptic feedback has not had a similar impact. In this study, we seek to develop a fundamental understanding of the utility of force feedback (and vision) in the functional operation of a body-powered upper-limb prosthesis. Using a custom body-powered prosthesis in which force feedback can be conditionally removed, we asked N=10 non-amputee participants to identify objects based on stiffness in four separate conditions with and without visual and/or force feedback. Results indicate that the combination of visual and force feedback allows for the best accuracy, followed by force feedback only, then visual feedback only. In addition, combining force feedback with visual feedback does not significantly affect identification timing compared to visual feedback alone. These findings suggest that consideration should be given to the development of force feedback displays for myoelectric prostheses that function like a Bowden cable, coupling the amputee’s control input to the resulting feedback.

@article{Brown2016b,
title = {An Empirical Evaluation of Force Feedback in Body-Powered Prostheses},
author = { Jeremy D. Brown and Timothy Kunz and Duane Gardner and Mackenzie K. Shelley and Alicia J. Davis and R. Brent Gillespie and Brent Gillespie},
url = {http://www.ncbi.nlm.nih.gov/pubmed/27101614 http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7452633&newsearch=true&queryText=An Empirical Evaluation of Force Feedback in Body-Powered Prostheses},
doi = {10.1109/TNSRE.2016.2554061},
issn = {1558-0210},
year = {2016},
date = {2016-04-01},
journal = {IEEE Transactions on Neural Systems and Rehabilitation Engineering},
volume = {PP},
number = {99},
abstract = {Myoelectric prostheses have many advantages over body-powered prostheses, yet the absence of sensory feedback in myoelectric devices is one reason body-powered devices are often preferred by amputees. While considerable progress has been made in the mechanical design and control of myoelectric prostheses, research on haptic feedback has not had a similar impact. In this study, we seek to develop a fundamental understanding of the utility of force feedback (and vision) in the functional operation of a body-powered upper-limb prosthesis. Using a custom body-powered prosthesis in which force feedback can be conditionally removed, we asked N=10 non-amputee participants to identify objects based on stiffness in four separate conditions with and without visual and/or force feedback. Results indicate that the combination of visual and force feedback allows for the best accuracy, followed by force feedback only, then visual feedback only. In addition, combining force feedback with visual feedback does not significantly affect identification timing compared to visual feedback alone. These findings suggest that consideration should be given to the development of force feedback displays for myoelectric prostheses that function like a Bowden cable, coupling the amputee’s control input to the resulting feedback.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

Brown, Jeremy; Shelley, Mackenzie; Gardner, Duane; Gansallo, Emmanuel A; Gillespie, R B
Non-colocated Kinesthetic Display Limits Compliance Discrimination in the Absence of Terminal Force Cues Journal Article
IEEE Transactions on Haptics, PP (99), pp. 1–1, 2016, ISSN: 1939-1412.

An important goal of haptic display is to make available the action/reaction relationships that define interactions between the body and the physical world. While in physical world interactions reaction cues invariably impinge on the same part of the body involved in action (reaction and action are colocated), a haptic interface is quite capable of rendering feedback to a separate body part than that used for producing exploratory actions (non-colocated action and reaction). This most commonly occurs with the use of vibrotactile display, in which a cutaneous cue has been substituted for a kinesthetic cue (a kind of sensory substitution). In this paper we investigate whether non-colocated force and displacement cues degrade the perception of compliance. Using a custom non-colocated kinesthetic display in which one hand controls displacement and the other senses force, we ask participants to discriminate between two virtual springs with matched terminal force and adjustable non-linearity. An additional condition includes one hand controlling displacement while the other senses force encoded in a vibrotactile cue. Results show that when the terminal force cue is unavailable, and even when sensory substitution is not involved, non-colocated kinesthetic displays degrade compliance discrimination relative to colocated kinesthetic displays. Compliance discrimination is also degraded with vibrotactile display of force. These findings suggest that non-colocated kinesthetic displays and likewise cutaneous sensory substitution displays should be avoided when discrimination of compliance is necessary for task success.

@article{Brown2016c,
title = {Non-colocated Kinesthetic Display Limits Compliance Discrimination in the Absence of Terminal Force Cues},
author = { Jeremy Brown and Mackenzie Shelley and Duane Gardner and Emmanuel A Gansallo and R.B. Gillespie},
url = {http://www.ncbi.nlm.nih.gov/pubmed/27101616 http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7452650},
doi = {10.1109/TOH.2016.2554120},
issn = {1939-1412},
year = {2016},
date = {2016-01-01},
journal = {IEEE Transactions on Haptics},
volume = {PP},
number = {99},
pages = {1–1},
abstract = {An important goal of haptic display is to make available the action/reaction relationships that define interactions between the body and the physical world. While in physical world interactions reaction cues invariably impinge on the same part of the body involved in action (reaction and action are colocated), a haptic interface is quite capable of rendering feedback to a separate body part than that used for producing exploratory actions (non-colocated action and reaction). This most commonly occurs with the use of vibrotactile display, in which a cutaneous cue has been substituted for a kinesthetic cue (a kind of sensory substitution). In this paper we investigate whether non-colocated force and displacement cues degrade the perception of compliance. Using a custom non-colocated kinesthetic display in which one hand controls displacement and the other senses force, we ask participants to discriminate between two virtual springs with matched terminal force and adjustable non-linearity. An additional condition includes one hand controlling displacement while the other senses force encoded in a vibrotactile cue. Results show that when the terminal force cue is unavailable, and even when sensory substitution is not involved, non-colocated kinesthetic displays degrade compliance discrimination relative to colocated kinesthetic displays. Compliance discrimination is also degraded with vibrotactile display of force. These findings suggest that non-colocated kinesthetic displays and likewise cutaneous sensory substitution displays should be avoided when discrimination of compliance is necessary for task success.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

2015

Brown, Jeremy D; O’Brien, Conor E; Miyasaka, Kiyoyuki W; Dumon, Kristoffel R; Kuchenbecker, Katherine J
Analysis of the Instrument Vibrations and Contact Forces Caused by an Expert Robotic Surgeon Doing FRS Tasks Inproceedings
8th Annual Hamlyn Symposium on Medical Robotics, pp. 75–77, 2015.

@inproceedings{Brown2015,
title = {Analysis of the Instrument Vibrations and Contact Forces Caused by an Expert Robotic Surgeon Doing FRS Tasks},
author = { Jeremy D. Brown and Conor E. O’Brien and Kiyoyuki W. Miyasaka and Kristoffel R. Dumon and Katherine J. Kuchenbecker},
year = {2015},
date = {2015-01-01},
booktitle = {8th Annual Hamlyn Symposium on Medical Robotics},
pages = {75–77},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

Brown, Jeremy D; Paek, Andrew; Syed, Mashaal; O’Malley, Marcia K; Shewokis, Patricia A; Contreras-Vidal, Jose L; Davis, Alicia J; Gillespie, Brent R
An exploration of grip force regulation with a low-impedance myoelectric prosthesis featuring referred haptic feedback Journal Article
Journal of NeuroEngineering and Rehabilitation, 12 (104), 2015, ISSN: 1743-0003.

@article{Brown2015a,
title = {An exploration of grip force regulation with a low-impedance myoelectric prosthesis featuring referred haptic feedback},
author = { Jeremy D. Brown and Andrew Paek and Mashaal Syed and Marcia K. O’Malley and Patricia A. Shewokis and Jose L. Contreras-Vidal and Alicia J. Davis and R. Brent Gillespie},
url = {http://www.jneuroengrehab.com/content/12/1/104},
doi = {10.1186/s12984-015-0098-1},
issn = {1743-0003},
year = {2015},
date = {2015-01-01},
journal = {Journal of NeuroEngineering and Rehabilitation},
volume = {12},
number = {104},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

2014

Gillespie, Brent R; Kim, Dongwon; Suchoski, Jacob M; Yu, Bo; Brown, Jeremy D
Series elasticity for free free-space motion for free Inproceedings
Proc. IEEE Haptics Symposium, pp. 609–615, IEEE, 2014, ISBN: 978-1-4799-3131-6.

Series elastic actuators are used to significant advantage in many robot designs but have not found their way into the design of haptic devices. We use a pneumatic circuit to realize both a flexible power transmission as well as the elastic element in a series elastic actuator. The pneumatic circuit effectively hides the impedance of a high friction, high mass ball-screw actuator, while a low friction, low mass pneumatic cylinder is used at the end-effector. We offer a comparative study of an impedance-control device, admittance-control device, and a device incorporating a series elastic actuator, investigating both the open-loop and the closed-loop impedance displayed to the user. While all hardware and control designs offer an ability to shape the impedance within their operational bandwidths, the series elastic design has the particular advantage of low impedance (a very compliant spring) outside of that bandwidth. Thus, a haptic device featuring series elastic actuation is capable of providing both the low impedances required in free-space and the high impedance required for rendering stiff virtual walls.

@inproceedings{Brown2015,
title = {Analysis of the Instrument Vibrations and Contact Forces Caused by an Expert Robotic Surgeon Doing FRS Tasks},
author = { Jeremy D. Brown and Conor E. O’Brien and Kiyoyuki W. Miyasaka and Kristoffel R. Dumon and Katherine J. Kuchenbecker},
year = {2015},
date = {2015-01-01},
booktitle = {8th Annual Hamlyn Symposium on Medical Robotics},
pages = {75–77},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

2013

Brown, Jeremy D; Paek, Andrew; Syed, Mashaal; O’Malley, M K; Shewokis, P A; Contreras-Vidal, J L; Gillespie, Brent R
Understandin[1] J. D. Brown, A. Paek, M. Syed, M. K. O’Malley, P. A. Shewokis, J. L. Contreras-Vidal, and R. B. Gillespie, “Understanding the Role of Haptic Feedback in a Teleoperated / Prosthetic Grasp and Lift Task,” in Proc. IEEE World Haptics Conferen Inproceedings
Proc. IEEE World Haptics Conference, pp. 271–276, 2013.

@inproceedings{Brown2013,
title = {Understandin[1] J. D. Brown, A. Paek, M. Syed, M. K. O’Malley, P. A. Shewokis, J. L. Contreras-Vidal, and R. B. Gillespie, “Understanding the Role of Haptic Feedback in a Teleoperated / Prosthetic Grasp and Lift Task,” in Proc. IEEE World Haptics Conferen},
author = { Jeremy D Brown and Andrew Paek and Mashaal Syed and M K O’Malley and P A Shewokis and J L Contreras-Vidal and R. Brent Gillespie},
year = {2013},
date = {2013-01-01},
booktitle = {Proc. IEEE World Haptics Conference},
pages = {271–276},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

Paek, Andrew Y; Brown, Jeremy D; Gillespie, Brent R; O’Malley, Marcia K; Shewokis, Patricia A; Contreras-Vidal, Jose L
Reconstructing surface EMG from scalp EEG during myoelectric control of a closed looped prosthetic device.
Proc. IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5602–5, 2013, ISSN: 1557-170X.

Series elastic actuators are used to significant advantage in many robot designs but have not found their way into the design of haptic devices. We use a pneumatic circuit to realize both a flexible power transmission as well as the elastic element in a series elastic actuator. The pneumatic circuit effectively hides the impedance of a high friction, high mass ball-screw actuator, while a low friction, low mass pneumatic cylinder is used at the end-effector. We offer a comparative study of an impedance-control device, admittance-control device, and a device incorporating a series elastic actuator, investigating both the open-loop and the closed-loop impedance displayed to the user. While all hardware and control designs offer an ability to shape the impedance within their operational bandwidths, the series elastic design has the particular advantage of low impedance (a very compliant spring) outside of that bandwidth. Thus, a haptic device featuring series elastic actuation is capable of providing both the low impedances required in free-space and the high impedance required for rendering stiff virtual walls.

@inproceedings{Paek2013,
title = {Reconstructing surface EMG from scalp EEG during myoelectric control of a closed looped prosthetic device.},
author = { Andrew Y Paek and Jeremy D Brown and R Brent Gillespie and Marcia K O’Malley and Patricia A Shewokis and Jose L Contreras-Vidal},
url = {http://www.ncbi.nlm.nih.gov/pubmed/24111007},
doi = {10.1109/EMBC.2013.6610820},
issn = {1557-170X},
year = {2013},
date = {2013-01-01},
booktitle = {Proc. IEEE Engineering in Medicine and Biology Society (EMBC)},
pages = {5602–5},
abstract = {In this study, seven able-bodied human subjects controlled a robotic gripper with surface electromyography (sEMG) activity from the biceps. While subjects controlled the gripper, they felt the forces measured by the robotic gripper through an exoskeleton fitted on their non-dominant left arm. Subjects were instructed to identify objects with the force feedback provided by the exoskeleton. While subjects operated the robotic gripper, scalp electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) were recorded. We developed neural decoders that used scalp EEG to reconstruct the sEMG used to control the robotic gripper. The neural decoders used a genetic algorithm embedded in a linear model with memory to reconstruct the sEMG from a plurality of EEG channels. The performance of the decoders, measured with Pearson correlation coefficients (median r-value = 0.59, maximum r-value = 0.91) was found to be comparable to previous studies that reconstructed sEMG linear envelopes from neural activity recorded with invasive techniques. These results show the feasibility of developing EEG-based neural interfaces that in turn could be used to control a robotic device.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

2012

Brown, Jeremy D; Gillespie, Brent R; Gardner, Duane; a. Gansallo, Emmanuel
Co-location of force and action improves identification of force-displacement features 
Proc. IEEE Haptics Symposium, pp. 187–193, 2012, ISBN: 978-1-4673-0809-0.

@inproceedings{Brown2012,
title = {Co-location of force and action improves identification of force-displacement features},
author = { Jeremy D. Brown and R. Brent Gillespie and Duane Gardner and Emmanuel a. Gansallo},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6183789},
doi = {10.1109/HAPTIC.2012.6183789},
isbn = {978-1-4673-0809-0},
year = {2012},
date = {2012-01-01},
booktitle = {Proc. IEEE Haptics Symposium},
pages = {187–193},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

2011

Brown, J D; Gillespie, R B

The effect of force/motion coupling on motor and cognitive performance

Proc. IEEE World Haptics Conference, pp. 197–202, 2011, ISBN: 978-1-4577-0299-0.

Haptic cues take on meaning as a function of the context in which they are experienced. In interaction with objects in the physical environment, the context always includes a mechanical contact, at which point force and motion variables can be identified, and across which power may flow. In interaction with objects in a virtual or remote environment, it is not necessary for the contact across which haptic responses are rendered to be the same as the contact at which exploratory actions are applied. In this paper, we ask whether force/motion coupling has a significant impact on manual performance or cognitive load. We conducted an experiment in which n=7 participants attempted, while acting through a teleoperator, to discriminate three objects by their stiffness under two conditions. In one condition physical force/motion coupling was present, in the other it was not. To assess cognitive load, we engaged participants in a simultaneous cognitive task that included a response time measure. Results indicated no difference in manual discrimination performance. After rejecting the datasets of three of our participants based on inconsistent strategy by condition, we observed a small, non-significant trend toward lower cognitive load in the condition with physical coupling. Establishing a robust trend will require additional participants. While results are preliminary, we offer our paradigm as an important direction for new inquiry into the distinctions and interrelationships between information and its presentation in various haptic interface applications. Our work is aimed in particular at developing haptic feedback for use in prosthetic applications.

@inproceedings{Brown2011,
title = {The effect of force/motion coupling on motor and cognitive performance},
author = { J D Brown and R B Gillespie},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5945485},
doi = {10.1109/WHC.2011.5945485},
isbn = {978-1-4577-0299-0},
year = {2011},
date = {2011-01-01},
booktitle = {Proc. IEEE World Haptics Conference},
pages = {197–202},
abstract = {Haptic cues take on meaning as a function of the context in which they are experienced. In interaction with objects in the physical environment, the context always includes a mechanical contact, at which point force and motion variables can be identified, and across which power may flow. In interaction with objects in a virtual or remote environment, it is not necessary for the contact across which haptic responses are rendered to be the same as the contact at which exploratory actions are applied. In this paper, we ask whether force/motion coupling has a significant impact on manual performance or cognitive load. We conducted an experiment in which n=7 participants attempted, while acting through a teleoperator, to discriminate three objects by their stiffness under two conditions. In one condition physical force/motion coupling was present, in the other it was not. To assess cognitive load, we engaged participants in a simultaneous cognitive task that included a response time measure. Results indicated no difference in manual discrimination performance. After rejecting the datasets of three of our participants based on inconsistent strategy by condition, we observed a small, non-significant trend toward lower cognitive load in the condition with physical coupling. Establishing a robust trend will require additional participants. While results are preliminary, we offer our paradigm as an important direction for new inquiry into the distinctions and interrelationships between information and its presentation in various haptic interface applications. Our work is aimed in particular at developing haptic feedback for use in prosthetic applications.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

2010

Gillespie, R B; Contreras-Vidal, J L; Shewokis, P A; O’Malley, M K; Brown, J D; Agashe, H; Gentili, R; Davis, A
Toward improved sensorimotor integration and learning using upper-limb prosthetic devices
Proc. IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5077–5080, 2010, ISBN: 978-1-4244-4123-5.

To harness the increased dexterity and sensing capabilities in advanced prosthetic device designs, amputees will require interfaces supported by novel forms of sensory feedback and novel control paradigms. We are using a motorized elbow brace to feed back grasp forces to the user in the form of extension torques about the elbow. This force display complements myoelectric control of grip closure in which EMG signals are drawn from the biceps muscle. We expect that the action/reaction coupling experienced by the biceps muscle will produce an intuitive paradigm for object manipulation, and we hope to uncover neural correlates to support this hypothesis. In this paper we present results from an experiment in which 7 able-bodied persons attempted to distinguish three objects by stiffness while grasping them under myoelectric control and feeling reaction forces displayed to their elbow. In four conditions (with and without force display, and using biceps myoelectric signals ipsilateral and contralateral to the force display,) ability to correctly identify objects was significantly increased with sensory feedback.

@inproceedings{Gillespie2010,
title = {Toward improved sensorimotor integration and learning using upper-limb prosthetic devices},
author = { R B Gillespie and J L Contreras-Vidal and P A Shewokis and M K O’Malley and J D Brown and H Agashe and R Gentili and A Davis},
doi = {10.1109/IEMBS.2010.5626206},
isbn = {978-1-4244-4123-5},
year = {2010},
date = {2010-01-01},
booktitle = {Proc. IEEE Engineering in Medicine and Biology Society (EMBC)},
pages = {5077–5080},
abstract = {To harness the increased dexterity and sensing capabilities in advanced prosthetic device designs, amputees will require interfaces supported by novel forms of sensory feedback and novel control paradigms. We are using a motorized elbow brace to feed back grasp forces to the user in the form of extension torques about the elbow. This force display complements myoelectric control of grip closure in which EMG signals are drawn from the biceps muscle. We expect that the action/reaction coupling experienced by the biceps muscle will produce an intuitive paradigm for object manipulation, and we hope to uncover neural correlates to support this hypothesis. In this paper we present results from an experiment in which 7 able-bodied persons attempted to distinguish three objects by stiffness while grasping them under myoelectric control and feeling reaction forces displayed to their elbow. In four conditions (with and without force display, and using biceps myoelectric signals ipsilateral and contralateral to the force display,) ability to correctly identify objects was significantly increased with sensory feedback.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}