Multimodal User Supervised Interface and Intelligent Control


MUSIIC Manual : Body

CHAPTER 1 Starting Up

The Components

This section details briefly the various server-client components making up the MUSIIC System

Starting up the Server and Clients

This section details the various servers and clients that need to be started up before commencing.

Starting the Server

1. Log onto euler.asel.udel.edu
2. Change directory to /home/kazi/research/phd/imp/current
3. Run bbserver if it is not already running

Speech: Starting up rpcear

1. Go to CITY-PC
2. Start Windows
3. Start DragonDictate
4. Open a DOS window from Main
5. <altl><enter> to reduce the window size
6. Go to C:\MUSIIC
7. Run rpcear

Arm: Starting up rpcarm

1. Go to the Robot PC
2. Change director to C:\MUSIIC
3. Run rpcarm

Vision: Starting up rpceye

1. Log into poison.asel.udel.edu
2. Change directory to /usr/people/chester/spiedemo
3. Run rpceye3

Planner: Starting up the planner

1. Log onto a solaris machine
2. Change directory to /home/kazi/research/phd/imp/current

General RPC Control

1. Open an window on a solaris machine
2. Change directory to /home/kazi/research/phd/imp/current
3. Run bbclient



It is necessary to synchronize the vision and speech servers to allow speech and vision deictics to be mapped correctly. With the current implementation the two servers remain synchronized for about an hour without severe degradation of performance. To synchronize use the generic rpc server and send a message "synchronize" from control to speech. This is illustrated below:

Computer output is shown in fixed font. Your input in emphasized text. Carriage return is symbolized by <return>.

Select [Quit, Send, Receive]: s
Select source [Speech, Vision, Control, Arm, Graphics]: s
Select destination [Speech, Vision, Control, Arm, Graphics]: c
Type message (< 125 characters)
>> synchronize

Working with the Speech Mechanism

After having invoked the DragonDictate from Windows the voice bar will always be visible. The title of the bar should say MS-DOS Window. If it does not then click on the title bar of the MS-DOS Window. You should have the basic vocabulary necessary for running MUSIIC trained by DragonDictate, otherwise you will not be able to use MUSIIC.

Select the activate button on the VoiceBar to cause DragonDictate to listen to and recognize your voice. Of course, you must have put on the microphone headset before this.

A mini-parser will parse your input and will only send your voice input to the planner if it conforms to the parse structure.

Putting DragonDictate to Sleep

To prevent spurious commands and noise from degrading performance, it is recommended that you put the recognizer to sleep after each complete command. This is done by speaking "Gotosleep". DragonDictate will still be listening but will not attempt to recognize any speech input.

Waking up DragonDictate

To resume the recognizer's recognition process simply say "wakeup".

Initializing the Planner

Now that you are ready you must initialize the planner before actually doing anything with the world.

Loading in the plan library

In order to load in the library of plans that the planner knows about by the command. [All succeeding commands are speech and/or gesture ones]

load plans

Loading in the Domain

Shoupu's vision system takes a snapshot of the domain under consideration and does some preliminary object parsing. This information must be obtained by the planner to update it's object knowledge base [DomainBase]. This is done by the command:

load domain

In both cases the planner will respond both through audio as well as text.

Robot Preliminaries

Now that the plans and objects have been loaded in, you must get the robot ready for operation. Make sure the robot is powered off and the arm is in its nesting position. All commands to the robot are preceded by the key word robot.

Homing the Robot

The following command will home the robot.

robot home

The planner will acknowledge the receipt through both audio as well as text. During execution you will be prompted by the robot PC to do certain things at certain times. Please take this into account and remain near the robot PC.

Make sure that the emergency STOP switch is in your hands at all times.

Putting the robot to the "ready" position

This is exactly the same position as the home position, except that the gripper will be open. The command is:

robot ready

Now that the robot is ready, a fact which will be acknowledged by the planner, you are ready to begin. The next chapter outlines what you can do and gives a description of the commands available for your use.

CHAPTER 3 Robot Commands


This chapter will introduce you to some of the basic robot commands available for your use and how to use them. Command names will be given in bold followed by the arguments in fixed fonts:

command-name argument-one argument-two

Variables will be denoted by words preceded by a colon:


Variables or names that will be matched to a gesture deictic will be shown within square brackets:



put [that] [there]
put :object [there]

In the first format, you say the word put followed by that and simultaneously pointing to the object; then you say there and then say there while again pointing at the destination location.

In the second case, the only difference is that one names the object that will be manipulated.


[thats :object]

This command is co-synchronous with a speech deictic. This maps an object pointed at by the user to a name, for later reference, as in the previous example.


go [there]

This causes the robot gripper to move to wherever the user was pointing.


move :location

In this command the variable :location can take one of 6 values:

1. up
2. down
3. right
4. left
5. closer
6. away

In each case the robot arm is moved a unit distance of 5 mm.



This command will cause the robot to assume the ready position



This command will cause the robot to go back to its next and will turn the power off.





CHAPTER 4 Miscellany

Vision Commands

load domain and load plans have been described previously.

Shutting Down

Make sure the robot is in the nest before shutting MUSIIC down.

Using the generic client send "quit" messages from the control to each of the three servers.


[MUSIIC [Robotics] [ASEL]


Last Updated: March 5, by Zunaid Kazi <kazi@asel.udel.edu>