This section details briefly the various server-client components making up
the MUSIIC System
This section details the various servers and clients that need to be started
up before commencing.
It is necessary to synchronize the vision and speech servers to allow speech
and vision deictics to be mapped correctly. With the current implementation the
two servers remain synchronized for about an hour without severe degradation of
performance. To synchronize use the generic rpc server and send a message
"synchronize" from control to speech. This is illustrated below:
Computer output is shown in fixed font. Your input in emphasized text.
Carriage return is symbolized by <return>.
After having invoked the DragonDictate from Windows the voice bar will
always be visible. The title of the bar should say MS-DOS Window. If it does
not then click on the title bar of the MS-DOS Window. You should have the basic
vocabulary necessary for running MUSIIC trained by DragonDictate, otherwise you
will not be able to use MUSIIC.
Select the activate button on the VoiceBar to cause DragonDictate to listen
to and recognize your voice. Of course, you must have put on the microphone
headset before this.
To prevent spurious commands and noise from degrading performance, it is
recommended that you put the recognizer to sleep after each complete command.
This is done by speaking "Gotosleep". DragonDictate will still be listening but
will not attempt to recognize any speech input.
To resume the recognizer's recognition process simply say "wakeup".
Now that you are ready you must initialize the planner before actually doing
anything with the world.
In order to load in the library of plans that the planner knows about by the
command. [All succeeding commands are speech and/or gesture ones]
Shoupu's vision system takes a snapshot of the domain under consideration
and does some preliminary object parsing. This information must be obtained by
the planner to update it's object knowledge base [DomainBase]. This is done by
In both cases the planner will respond both through audio as well as text.
Now that the plans and objects have been loaded in, you must get the robot
ready for operation. Make sure the robot is powered off and the arm is in its
nesting position. All commands to the robot are preceded by the key word robot.
The following command will home the robot.
The planner will acknowledge the receipt through both audio as well as text.
During execution you will be prompted by the robot PC to do certain things at
certain times. Please take this into account and remain near the robot PC.
This is exactly the same position as the home position, except that the
gripper will be open. The command is:
Now that the robot is ready, a fact which will be acknowledged by the
planner, you are ready to begin. The next chapter outlines what you can do and
gives a description of the commands available for your use.
This chapter will introduce you to some of the basic robot commands
available for your use and how to use them. Command names will be given in bold
followed by the arguments in fixed fonts:
Variables will be denoted by words preceded by a colon:
Variables or names that will be matched to a gesture deictic will be shown
within square brackets:
In the first format, you say the word put followed by that and
simultaneously pointing to the object; then you say there and then say there
while again pointing at the destination location.
This command is co-synchronous with a speech deictic. This maps an object
pointed at by the user to a name, for later reference, as in the previous
This causes the robot gripper to move to wherever the user was pointing.
In this command the variable :location can take one of 6 values:
In each case the robot arm is moved a unit distance of 5 mm.
This command will cause the robot to assume the ready position
This command will cause the robot to go back to its next and will turn the
load domain and load plans have been described previously.
Make sure the robot is in the nest before shutting MUSIIC down.
Using the generic client send "quit" messages from the control to each of
the three servers.
Last Updated: March 5, by