[Prev][Next][Index][Thread]
Intent to submit
Subject: Special Issue of the journal Human-Computer Interaction
on the topic of "Multimodal Interfaces"
We intend to submit a paper for this special issue. An abstract
of the paper we intend to submit follows.
Thank you,
Zunaid Kazi
---- begin ----
Title:
Multimodal Control of an Assistive Robot
Authors:
Zunaid Kazi, Marcos Salganicoff, Matthew Beitler,
Shoupu Chen, Daniel Chester and Richard Foulds
Address:
Applied Science and Engineering Laboratories
Alfred I. duPont Institute/University of Delaware
Wilmington, Delaware USA
Abstract:
The Multimodal User Supervised Interface and Intelligent Control
(MUSIIC) system involves the integration of human-computer
interaction with reactive planning to operate a telerobot for use
as an assistive device. The system is intended to meet the needs
of individuals with physical disabilities to control an assistive
robot in an unstructured environment, rather than in a structured
workcell. We describe a novel approach for an intelligent
assistive telerobotic system for such an environment: speech and
gesture control integrated with a knowledge-driven reactive
planner and a stereo-vision system which builds a superquadric
shape representation of the scene.
Users of our system use deictic gestures (pointing, achieved by a
head mounted laser pointer) to indicate locations, and spoken
commands to identify objects and specific actions. The
combination of spoken language along with deictic gestures
performs a critical disambiguation function by binding the spoken
words in terms of nouns and actions to a locus in the physical
work space. The spoken input is used to supplant the need for a
general purpose object recognition module in the system.
Our approach is based on the assumption that the user's world is
unstructured, but that the objects within that world have
reasonably predictable properties. We reflect this arrangement by
providing a means of determining the three-dimensional shape and
pose of objects and surfaces, and an object-oriented knowledge
base and planning system which superimposes information about
common objects onto the three-dimensional world. A third major
aspect involves the multimodal user interface which interprets
the gesture and speech inputs with the objective of identifying
the portion of contour that is of interest to the user, as well
as allowing an easier and more flexible interface for robot
control. In addition, we are developing a simulation environment
that will enable us to investigate the modalities of the human-
computer interaction in a low risk fashion.
---- end ----
--
---------------------------------------------------------------------
Zunaid Kazi kazi@asel.udel.edu (or @cis or @strauss)
CompSci & Robotics http://www.asel.udel.edu/~kazi
---------------------------------------------------------------------