ISIS World

Setting up and Installing IsisWorld

1 The IsisWorld Simulator for AI research

The IsisWorld simulator is available to researchers for building and evaluating problem solving and language learning systems with everyday problems that take place in a kitchen. Some documentation and arguments about theoretical underpinnings aimed toward the metareasoning community are available in this paper:

1.1 About IsisWorld

IsisWorld is a highly-customizable open source kitchen simulator written in the Python language. IsisWorld was created to as evaluation platform for evaluating and observing the behavior of commonsense reasoning systems. It has a number of unique features:

  • Source code and binaries are freely available for download and distribution
    All source code is immediately available and binaries for most common platforms are available for download. The Panda3D libraries and the ODE framework are covered by the BSD 3-clause license. The rest of the project is covered by the GPLv3.
  • Task evaluation framework is built-in
    It’s easy to write scenario templates and create custom evaluation scenarios. A scenario defines the default 3D environment and contains tasks with training settings and goal functions.
  • Environments are generated from a specification file according to commonsense defaults
    Environment descriptions are procedurally generated: the sizes and orientation of the kitchen are determined each time the environment is generated as an effort to prevent AI systems from “overfitting” the task.
  • Client-server architecture
    The agents in the simulator can be controlled by a client. Other than XML-RPC, no additional libraries are required.
  • Rigid-body physics simulation:
    Objects’ position and orientation can be controlled by the Open Dynamics Engine: an object location is a function of its mass, and linear and angular forces (including gravity). Custom callbacks and triggers can be defined.
  • Default objects with commonsense behaviors
    We are continuing to add new objects, actions and events to the environment.

2 Installing the simulator binaries

If you download and execute the standalone simulator binaries, you can control the agent using both the commands above or by a client using XML-RPC on port 8001. Note, this will not allow you to make changes to the simulator world, just the client. Standalone binaries are available for most platforms:

3 Change History

What’s new in the Iris World simulator?

  • Forthcoming features Release TBA
    • fixed broken ‘say’ command
    • loads objects with default properties in objects.py
    • object loader handles containment (“in”), surface contact (“on”) and part of relationships
    • kinematic and dynamic physical modeling using the Open Dynamics Engine
    • more generic Boxman models
  • 0.3.1 2010–04–09 -fixed bug accepting text
  • 0.3 2010–04–06
    • Added body orientation (h,p,r) to proprioceptive position frame.
    • Changed name from “Society of Mind Simulator” to Isis Simulator.
  • 0.2 2010–03–17
    • Initial public release.

4 Contributors

The simulator was brought to life through the work of:

We also thank the following people for contributing bug reports and fixes:

  • A. Christine Spang

5 Developing an agent to use the simulator

5.1 Perceptual and motor interfaces

By default, the simulator is paused, and no physical simulation can take place. The simulator can be unpaused and stepped, advanced incrementally, or run continuously. The character in the simulator, “Ralph”, can be controlled directly in the simulator using key commands or separately through a programmed agent (a client). The environment and agent are temporally decoupled. In this section, we review the agent-environment interface in terms of a perceptual frame and primitive actions.

5.1.1 Perceptual Frame

The agent can sense the environment, which returns a perceptual frame containing:

Object Information
All names of the items in the character’s field of vision; each item’s center’s relative x and y coordinates on the frustum: the 2D plane representing the projected 3D region of the character’s field of vision 1; the item’s area of the entire field of vision; the item’s distance; and the item’s global orientation.

Proprioceptive Information
The character’s global position (x,y,z) and orientation (h,p,r) in the environment as well as the relative positions and orientations of the character’s head and limbs with respect to its body; and the names of the items within each of the character’s hands (if any).

Linguistic Information:
A buffer representing all of the text entered by the user since the last time the sense command was executed.

5.1.2 Primitive actions

The character has a set of primitive actions that can be taken:

  • turn_left-start and turn_left-stop: Starts and stops rotating the character’s body along the vertical Z axis: subtracts from theheading.
  • turn_right-start and turn_right-stop: Starts and stops rotating the character’s body along the vertical Z axis: adds to theheading.
  • move_forward-start and move_forward-stop: Starts and stops moving the character’s location in the environment forward (in the direction the body faces) a fixed amount, and changes the model to an animated walking character.
  • move_backward-start and move_backward-stop: Starts and stops moving the character’s location in the environment backward a fixed amount, and changes the model to an animated walking character.
  • look_left-start and look_left-stop: Starts and stops rotating the character’s neck, with respect to the body, along the horizontal X axis: adds to the neck’s pitch. (look- actions are bounded to prevent uncanny exorcist-style head rotations.)
  • look_right-start and look_right-stop: Starts and stops rotating the character’s neck, with respect to the body, along the horizontal X axis: subtracts from the neck’s pitch.
  • look_up-start and look_up-stop: Starts and stops rotating the character’s neck, with respect to the body, along the vertical Z axis: adds to the neck’s heading.
  • look_down-start and look_down-stop: Starts and stops rotating the character’s neck, with respect to the body, along the vertical Z axis: subtracts from the neck’s heading.
  • say(statement): Presents the a statement string above the character’s head, which gradually disappears over the course of two seconds as the simulator steps forward in time.
  • pick_up_with_right_hand(object): Attaches the object to the character’s right hand, so long as the object is graspable and the actor has a free right hand.
  • pick_up_with_left_hand(object): Attaches the object to the character’s left hand, so long as the object is graspable and the actor has a free left hand.
  • drop_from_right_hand: Releases any object from the character’s right hand.
  • drop_from_left_hand: Releases any object from the character’s left hand.

Some actions like turn_left are interval actions and have separate -start and -stop commands. Once started, they continue until they are stopped. Other actions are atomic and instantaneous, like drop_from_right_hand.

5.2 Building an agent

A simple implementation of an agent in Python:

1
2
3
4
5
6
7
8
9
10
import xmlrpclib as xml
# connect to environment via XML-RPC
e = xml.ServerProxy('http://localhost:8001')
# sense world
perceptions = e.do('sense')
# do something
e.do('say',{'statement':"Hello world!"})
# simulator is paused by default
# run for X=0.02 seconds
e.do('step_simulation',{'seconds':0.02})

6 Developing the simulator

Source code is available on GitHub. To edit the repository, use Git:

git@github.com:dasmith/IsisWorld.git

Running the simulator from source requires the Panda3D SDK and Python 2.4–2.6. To install the Panda3D SDK, you should first try installing a precompiled binary. You can obtain these from:

If you need to compile Panda3D SDK from source, download the source and extract the tarball. The easiest way to compile is:

tar xvfz panda3d-1.7.0.tgz
cd panda3d-1.7.0
python makepanda/makepanda.py --everything

To verify that your installation works, try running one of the Sample / Tutorial files and seeing if a 3D visual appears.

7 Problems and solutions for Developers

Here is a list of problems

7.1 ImportError: No module named direct.directbase.DirectStart

This problem is probably that you are using a different version of Python than what Panda3D has configured itself for.

To determine which version of Python you are using, type:

python -V

Panda3D configures itself for Python 2.5. Somewhere in that directory is a “Panda3D.pth” file.

cat /path/to/old/python/2.5/site-packages/Panda3D.pth

Which should output “/installed/path/Panda3D/lib”, indicating the path to the required Panda3d libraries. But you now have to copyPanda3D.pth to the site-packages directory of default version of Python you are using. To see all paths to installed python (recognized by your $PATH variable), type:

which python

The first path output by this command, e.g., if it outputs

/another/path/2.6/bin/python

is the python executed by default when you type “python”. That’s the one who needs to have the Panda3D.pth file. Copy the file to the Python’s site-packages folder, e.g.,

sudo cp /path/to/old/python/2.5/site-packages/Panda3D.pth /another/path/2.6/lib/python2.6/site-packages/

Notice the “2.6/bin/python” from the output of which changed to “2.6/lib/python2.6/site-packages/”

7.2 Ubuntu install help

System: Ubuntu 9.04 or 9.10, 32 or 64 bit.

Problems:

  1. Missing 3rd party dependencies.
  2. Running the sample programs gives no errors but displays a grey window.

Solutions:

  1. A list of dependencies to install:
  2. Change /etc/Config.prc:

Thanks to Sarina Canelake for this information.

7.4 Running on Athena

To develop the simulator on Athena, you should compile Panda3D from source.

CONTACT US

We're not around right now. But you can send us an email and we'll get back to you, asap.

Sending

©2008-2017 MIT Mind Machine Project, all rights reserved.

Log in with your credentials

Forgot your details?