This demo allows a user to issue natural language commands via a command line interface. The output is text printed out to the Terminal or Command Prompt, informing the user of what actions have been taken, such as:

>> Robot1, move to the big red box!
FED1_ProblemSolver: robot1_instance is moving to (6.0, 6.0, 0.0)

The same Robot Problem Solver that's used in the text demo can be used for the Morse/ROS demos -- just with different API calls.

This demo has the same requirements as the ecg-framework-code. Follow the guidelines there (including downloading the other ECG repositories, such as ecg_grammars), and once you've installed the dependencies, you can proceed. Make sure you also install ecg_framework_code, as that provides the core components necessary to run the robot version.

To download the ecg_robot_code repository, clone it in the same directory as ecg_framework_code and ecg_grammars:

git clone https://github.com/icsi-berkeley/ecg_robot_code.git

This repository contains app versions of the various core components in the framework (details on the "core" code can be found here).

Like with the ecg_framework_code, you'll want to manually add this to your path:

export PYTHONPATH = {PATH_TO_ROBOT_CODE}/src/main:$PYTHONPATH

Like ecg_framework_code, there is a bash script called analyzer.sh. Also like ecg_framework_code, it assumes the ecg_grammars repository is installed in the same directory, adjacent to ecg_robot_code. Unlike the framework version, however, this initializes the ECG Analyzer with the compRobots grammar. The compRobots grammar is distinct from the research grammar because it: 1) contains vocabulary specific to the robot application, and 2) imports more of the inactive packages defined in "core".

Run the Analyzer with the following command:

./analyzer.sh

Note that by default, the Jython command to run the Analyzer looks like this:

jython -m analyzer ../ecg_grammars/compRobots.prefs

Depending on your computer's processing power, you might need to run a variant of the command (also found in analyzer.sh), which allocates virtual memory to the Jython process:

jython -J-Xmx4g -m analyzer ../ecg_grammars/compRobots.prefs

You can change the number in the command to change how much memory you allocate, but beware that allocating too much of your machine's memory resources could affect the behavior of other applications.

Once the Analyzer is running, you can start the text-based system by running:

./setup.sh

This initializes the following Agents:

1. Robot Problem Solver: receives n-tuple, solves problem
2. UI-Agent: receives text, produces n-tuple with help from Analyzer and Specializer (also sets up a proxy connection to the localhost on which the Analyzer runs)
3. Text-Agent: prompts user for input, sends text to UI-Agent

If all goes well, you should be prompted with the following screen:

./setup.sh
>

Without a visualization of the simulated world, the demo is not as rich as the Morse demo. However, the user can still issue commands, assert information, and also query information about the six boxes (box1-box6) that exist in the world. World information is contained in a hard-coded JSON world model, which is used to produce the Problem Solver's context model.

Note that if a user references an object the Problem Solver doesn't know about, it will notify the user:

> is the black box big?
FED1_ProblemSolver: Sorry, I don't know what the black box is.

This kind of identification_failure procedure is also used to notify a user when the Solver doesn't know how to perform an action, etc.

FED1_ProblemSolver: I cannot solve the {} action.

An interesting first query can be asking about the locations of various objects. Objects can be referenced either by their proper name, such as...

> where is box1?
FED1_ProblemSolver: the position of the Box1 is (6.0, 6.0, 0.0).

...Or by a description, such as:

> where is the big red box?
FED1_ProblemSolver: the position of the red box is (6.0, 6.0, 0.0).

Both questions are actually about the same object, but the reply is tailored to the input language.

Where questions expect information about the location or position of an object. However, a user can also ask yes/no questions about qualities of certain objects:

> is the blue box big?
FED1_ProblemSolver: Yes.
> is the blue box near the green box?
FED1_ProblemSolver: No.
> is the blue box near the big red box?
FED1_ProblemSolver: Yes.

A user can also request information about object properties, such as:

> what color is box1? FED1_ProblemSolver: red

Finally, a user can ask to identify objects by their properties, such as:

> which box is blue?
FED1_ProblemSolver: box2
> which box is near the big red box?
FED1_ProblemSolver: There is more than one item matching this description.
FED1_ProblemSolver: box6, box2, box5
> which boxes are red?
FED1_ProblemSolver: box1, box4

The user can also make assertions about the state of the world, such as object properties. These are somewhat under-developed in the Problem Solver -- e.g., many more assertions can be converted into n-tuples than can be "solved" by the Problem Solver. Below are some examples of the types of assertions that can be made:

> the blue box weighs 5 pounds
> Robot1 has 10 gallons of fuel

Commands are probably the most interesting aspect of the system, since it involves actually carrying out an action. With the text-based demo, this action simply takes the form of changing the world model, and then reporting back the results of the action through a printed statement.

Moving is the main action that the robot is capable of. Note that in general, a Motion event has several important parameters that can be specified by language: heading/direction, speed, distance, and a source-path-goal (e.g. instructing the robot to move TO an object). These parameters all have defaults, so they don't have to be specified by language.

> Robot1, move north!
FED1_ProblemSolver: robot1_instance is moving to (0.0, 4.0, 0.0)
> Robot1, move 5 inches north!
FED1_ProblemSolver: robot1_instance is moving to (0.0, 9.0, 0.0)
> Robot1, move to the big red box!
FED1_ProblemSolver: robot1_instance is moving to (6.0, 6.0, 0.0)

The speed parameter will not affect the printed out statements, but it will appear in the n-tuple. Speed can be modulated with manner verbs like "dash" or "amble", or with adverbs like "quickly" or "slowly".

Pushing is another significant action. This involves more problem solving, since the robot has to get in a position to push the box, and then actually carry out the action.

> Robot1, push the big red box 4 inches north!
FED1_ProblemSolver: robot1_instance is moving to (6.0, 2.0, 0.0)
FED1_ProblemSolver: robot1_instance is moving to (6.0, 10.0, 0.0)

Note that if the robot is unable to carry out the action, the Problem Solver will notify the user:

> Robot1, push the blue box north!
FED1_ProblemSolver: box6 is in the way
> Robot1, push the big red box 1000 inches north!
FED1_ProblemSolver: not enough fuel

Another interesting aspect of the system is that it can process conditional statements, including conditional imperatives. In terms of the n-tuple, a conditional imperative can be composed with a Yes/No query (condition) and a command (imperative).

> Robot1, if the box near the green box is red, push it south!

Try out various yes/no questions as the conditions. Here, you can also ask questions about modality:

> Robot1, if you can move to the big red box, push it north!

You can also use while instead of if, which stipulates that the robot carry out the action as long as the condition is satisfied:

> Robot1, while you are not near box1, move 1 inch north!

If user input is under-specified, the Problem Solver can ask for additional information. In this case, a question is returned to the user:

> Robot1, move to the red box!
FED1_ProblemSolver: which red box?

The user can then supply more information:

> the one near the green box

The UI-Agent then incorporates the new information into the original n-tuple, and sends the modified n-tuple back to the Problem Solver.

If you'd like to add new behavior to the robot, you'll need to add new vocabulary. If you want to add an entirely new procedure or behavior (e.g. "swimming"), you'd need to add code to the Problem Solver (check out this documentation on how the solver is written for guidelines). However, if you just want to modulate existing behavior with new actions, such as change the speed at which the robot moves, it's as easy as adding a token to the grammar.

Tokens can be added using Token Tool. To add a simple token like "dash", see here for a brief tutorial. You'll need the ECG Workbench and accompanying tutorial.