Home

Awesome

GelSight Simulation

This repository contains the necessary code for executing the Simulated Environment and experiments as in our GelSight Simulation paper. These packages contain the drivers for running a GelSight sensor (in real world and simulation) and a FDM-Printer for carrying the described experiments (also in real world and simulation). The experiments includes the final experiments for the dataset alignment and the Sim2Real classification task, and should be executed using Python 3 (outside ROS). Visit danfergo.github.io/gelsight-simulation for more information about the work and links for downloading the datasets,

Index of Packages

PackageDescription
experimentsPython3 (outside ROS) scripts with the experiments described in the paper.
fdm_printerContains two drivers for running the FDM printer in the real world: a standard ROS subscriber/publisher and a ROS Control Harware Interface. The drivers work by issuing g-code commands to the printer.
fdm_printer_bringupIncludes the file for launching the printer either in simulation or the real world (sim:=true for simulation)
fdm_printer_descriptionThe URDF files and STL meshes describing the printer.
gelsight_descriptionThe URDF files and STL meshes describing the GelSight sensor. The modeling is based on sensor proposed here.
gelsight_gazeboThis package contains the driver that implements the proposed approach, to be used in simulation.
gelsight_simulationScripts and Materials used to carry the data collection process.

To run the collection of tactile images using the simulated setup.

roscore
roslaunch gelsight_simulation dc.launch sim:=true
rosrun gelsight_simulation data_collection.py

To run the experiments scripts, e.g.,

    python -m experiments.sim2real.train_nn

A big thanks to keras-visuals, for providing some helpful Keras callbacks for assessing our NN optimization.