Intent to submit

Subject: Special Issue of the journal Human-Computer Interaction 
         on the topic of "Multimodal Interfaces"

We intend to submit a paper for this special issue. An abstract
of the paper we intend to submit follows.

Thank you,
Zunaid Kazi

---- begin ----


	Multimodal Control of an Assistive Robot


	Zunaid Kazi, Marcos Salganicoff, Matthew Beitler,
	Shoupu Chen, Daniel Chester and Richard Foulds


	Applied Science and Engineering Laboratories
	Alfred I. duPont Institute/University of Delaware
	Wilmington, Delaware USA


The Multimodal User Supervised Interface and Intelligent  Control
(MUSIIC)   system  involves  the  integration  of  human-computer
interaction with reactive planning to operate a telerobot for use
as  an assistive device. The system is intended to meet the needs
of individuals with physical disabilities to control an assistive
robot in an unstructured environment, rather than in a structured
workcell.  We  describe  a  novel  approach  for  an  intelligent
assistive  telerobotic system for such an environment: speech and
gesture  control  integrated  with  a  knowledge-driven  reactive
planner  and  a  stereo-vision system which builds a superquadric
shape representation of the scene.

Users of our system use deictic gestures (pointing, achieved by a
head  mounted  laser  pointer)  to indicate locations, and spoken
commands  to  identify  objects   and   specific   actions.   The
combination  of  spoken  language  along  with  deictic  gestures
performs a critical disambiguation function by binding the spoken
words  in  terms  of nouns and actions to a locus in the physical
work space. The spoken input is used to supplant the need  for  a
general purpose object recognition module in the system.

Our approach is based on the assumption that the user's world  is
unstructured,  but  that  the  objects  within  that  world  have
reasonably predictable properties. We reflect this arrangement by
providing  a means of determining the three-dimensional shape and
pose of objects and surfaces, and  an  object-oriented  knowledge
base  and  planning  system  which superimposes information about
common objects onto the three-dimensional world.  A  third  major
aspect  involves  the  multimodal user interface which interprets
the gesture and speech inputs with the objective  of  identifying
the  portion  of contour that is of interest to the user, as well
as allowing an easier  and  more  flexible  interface  for  robot
control.  In addition, we are developing a simulation environment
that will enable us to investigate the modalities of  the  human-
computer interaction in a low risk fashion.

---- end   ----
Zunaid Kazi                  kazi@asel.udel.edu (or @cis or @strauss)
CompSci & Robotics                     http://www.asel.udel.edu/~kazi