Awesome

MeshCat

MeshCat is a remotely-controllable 3D viewer, built on top of three.js. The MeshCat viewer runs in a browser and listens for geometry commands over WebSockets. This makes it easy to create a tree of objects and transformations by sending the appropriate commands over the websocket.

The MeshCat viewer is meant to be combined with an interface in the language of your choice. Current interfaces are:

• meshcat-python (Python 2.7 and 3.4+)
• MeshCat.jl (Julia)

API

MeshCat can be used programmatically from JS or over a WebSocket connection.

Programmatic API

To create a new MeshCat viewer, use the Viewer constructor:

let viewer = new MeshCat.Viewer(dom_element);

where dom_element is the div in which the viewer should live. The primary interface to the viewer is the handle_command function, which maps directly to the behaviors available over the WebSocket.

WebSocket API

Setting opacity

Objects can have their opacity changed in the obvious way, i.e.:

This would assign the opacity value 0.5 to all of the materials found rooted at "/path/to/my/geometry". (That means using the path "/path" could affect many geometries.) However, this will overwrite whatever opacity the geometry had inherently (i.e., from the material defined in the mesh file); a transparent object could become more opaque.

Meshcat offers a pseudo property "modulated_opacity". Meshcat remembers an object's inherent opacity and, by setting this value, sets the rendered opacity to be the product of the inherent and modulated opacity value. The corresponding command would be:

Setting "modulated_opacity" to 1 will restore the geometry's original opacity. This does not introduce a queryable modulated_opacity property on any material. This is what makes it a "pseudo" property. The same tree-based scope of effect applies to "modulated_opacity" as with "opacity" (and also the "color" property).

Meshcat always remembers the inherent opacity value. So, if you've overwritten the value (via setting "opacity" or "color"), you can restore it by setting the "modulated_opacity" value to 1.0.

Useful Paths

The default MeshCat scene comes with a few objects at pre-set paths. You can replace, delete, or transform these objects just like anything else in the scene.

Camera Control

The camera is just another object in the MeshCat scene, so you can move it around with <code>set_transform</code> commands like any other object. You can also use <code>set_target</code> to change the camera's target. Please note that replacing the camera with <code>set_object</code> is not currently supported (but we expect to implement this in the future).

Controlling the camera is slightly more complicated than moving a single object because the camera actually has two important poses: the origin about which the camera orbits when you click-and-drag with the mouse, and the position of the camera itself. In addition, cameras and controls in Three.js assume a coordinate system in which the Y axis is upward. In robotics, we typically have the Z axis pointing up, and that's what's done in MeshCat. To account for this, the actual camera lives inside a few path elements:

/Cameras/default/rotated/<object>

The <code>/rotated</code> path element exists to remind users that its transform has been rotated to a Y-up coordinate system for the camera inside.

There is one additional complication: the built-in orbit and pan controls (which allow the user to move the view with their mouse) use the translation of only the intrinsic transform of the camera object itself to determine the radius of the orbit. That means that, in practice, you can allow the user to orbit by setting the position property at the path /Cameras/default/rotated/<object> to a nonzero value like [2, 0, 0], or you can lock the orbit controls by setting the position property at that path to [0, 0, 0]. Remember that whatever translation you choose is in the rotated, Y-up coordinate system that the Three.js camera expects. We're sorry.

Examples

To move the camera's center of attention to the point [1, 2, 3], while still allowing the user to orbit and pan manually, we suggest setting the transform of the /Cameras/default path to whatever center point you want and setting the position property of /Cameras/default/rotated/<object> to [2, 0, 0]. That means sending two commands:

{
    type: "set_transform",
    path: "/Cameras/default",
    matrix: new Float32Array([1, 0, 0, 0,
                              0, 1, 0, 0,
                              0, 0, 1, 0,
                              1, 2, 3, 1])  // the translation here is the camera's
                                            // center of attention
},
{
    type: "set_property",
    path: "/Cameras/default/rotated/<object>",
    property: "position",
    value: [2, 0, 0] // the offset of the camera about its point of rotation
}

To move the camera itself to the point [1, 2, 3] and lock its controls, we suggest setting the transform of /Cameras/default to the exact camera pose and setting the position property of /Cameras/default/rotated/<object> to [0, 0, 0]:

{
    type: "set_transform",
    path: "/Cameras/default",
    matrix: new Float32Array([1, 0, 0, 0,
                              0, 1, 0, 0,
                              0, 0, 1, 0,
                              1, 2, 3, 1])  // the translation here is now the camera's
                                            // exact pose
},
{
    type: "set_property",
    path: "/Cameras/default/rotated/<object>",
    property: "position",
    value: [0, 0, 0] // set to zero to lock the camera controls
}

Developing MeshCat

The MeshCat javascript sources live in src/index.js. We use webpack to bundle up MeshCat with its Three.js dependencies into a single javascript bundle. If you want to edit the MeshCat source, you'll need to regenerate that bundle. Fortunately, it's pretty easy:

Option 1: Use Docker

If you have Docker + BuildKit installed, then you can run rebuild/rebuild.sh to hermetically regenerate the dist files. The very first time you run it, it might take a little time to download all of the relevant files. Subsequent runs should be much faster (~5-10 seconds).

This option offers the easiest way to rebuild, with the drawback that "watch" mode of webpack (to automatically rebuild every time you save) is unavailable.

To install Docker + BuildKit on Ubuntu, run:

sudo apt install docker.io docker-buildx

To run as non-root, you will probably also need to update the docker group permissions:

https://docs.docker.com/engine/install/linux-postinstall/

Option 2: Install Yarn + NPM tools

This option offers a convenient "watch" mode during an edit-test development cycle, but if you're not careful when installing the yarn+node+npm tools you might damage other software on your computer.

1. Install yarn. This should also install node and npm. Try running yarn -v and npm -v to make sure those programs are installed.
2. Run yarn
   • This will read the project.json file and install all the necessary javascript dependencies.
3. Run npm run build
   • This will run webpack and create the bundled output in dist/main.min.js. The build script will also watch for changes to the MeshCat source files and regenerate the bundle whenever those source files change.
4. Try it out! You can load the bundled main.min.js in your own application, or you can open up dist/index.html in your browser.

Note that due to caching, you may need to do a hard refresh (shift+F5 or ctrl+shift+R) in your browser to reload the updated javascript bundle.