Awesome
Smile software workspace
This repository contains the software for the project Smile that is the basis for these papers:
-
Joublin, F., Ceravola, A., Smirnov, P., Ocker, F., Deigmoeller, J., Belardinelli, A., Wang, C., Hasler, S., Tanneberg, D. & Gienger, M.: Copal: Corrective planning of robot actions with large language models. International Conference on Robotics and Automation (ICRA), 2024.
-
Tanneberg, D., Ocker, F., Hasler, S., Deigmoeller, J., Belardinelli, A., Wang, C., Wersing, H., Sendhoff, B. & Gienger, M. (2024). To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactions. arXiv preprint arXiv:2403.12533.
Checkout and update from git
Initial checkout of the workspace
For an initial checkout, do:
git clone https://github.com/HRI-EU/SmileWorkspace.git
cd SmileWorkspace
git submodule update --init --recursive
Updating the workspace
To update the workspace and all submodules, do:
cd SmileWorkspace
git fetch
git rebase
git submodule update --init --recursive
How to build
Linux
The code has been mainly developed under Ubuntu 20.04. These system packages are required to be installed before building the project:
- Libxml2 (MIT License, Ubuntu-package libxml2-dev)
- Qt5: LGPL (Also dual commercial license available, Ubuntu-package qt5-default)
- qwt (LGPL with additions, Ubuntu-package libqwt-qt5-dev)
- OpenSceneGraph (OSGPL, Ubuntu-package libopenscenegraph-dev)
- Bullet Physics (Zlib, Ubuntu-package libbullet-dev)
mkdir -p <build-directory>
cd <build-directory>
cmake <repo-top-directory> -DCMAKE_INSTALL_PREFIX=<install-dir>
make -j
make install
While it is not mandatory, we recommend adding build and install directories under the repositorie's top-level directory. This eases defining the paths:
cd <repo-top-directory>
mkdir -p build install
cd build
cmake ..-DCMAKE_INSTALL_PREFIX=../install
make -j
make install
Windows
It also compiles for Windows (Visual Studio), please follow the instructions here.
How to run
C++ test program
cd install
bin/TestLLMSim
You should now see a graphics window. If you hover the mouse over an object on the table, and type 'm' in the graphics window, you should be able to see the robot grasping it.
Python module test
This section explains how to run the simulator from a python console. First, start a python console from the install directory. In order to find the python module and the dependent library files, the path to the installation directory's library directory needs to be added. For Linux, the so files and the python module are located in the lib directory. For Windows, the dlls and the python module are located in the bin directory.
import platform
import sys
if platform.system() == "Linux":
sys.path.append("lib")
elif platform.system() == "Windows":
sys.path.append("bin")
Now we can import the pyAffaction module:
from pyAffaction import *
Now we can instantiate a simulator. The variables allow to configure it in various ways, please refer to the python module's documentation.
sim = LlmSim()
sim.noTextGui = True
sim.unittest = False
sim.useWebsocket = False
sim.speedUp = 3
sim.noLimits = False
sim.verbose = False
sim.init(True)
This brings up a graphics window (headless with sim.init(False)). To start the simulator's endless loop, type
sim.run()
You can do the above conveniently by calling
python -i ../src/AffAction/python/pySim.py
Now, you can send commands to it on the console, for instance
sim.plan("get glass_red", True)
To stop it, use
sim.stop()
To properly exit the python console, type
del b
quit()
Click here for more information and documentation of the python API.
Connect to simulator with python websocket client
See section Python virtual environment in src/AffAction/README.md
License
The project is licensed under the BSD-3-Clause License.
Licenses of dependencies
This repository includes several files and submodules with the licenses stated here. We include some mesh files of the kinova-ros and ros-kortex packages in the Meshes directory. These mesh files and these are licensed under the BSD-3-Clause License. We also include parts of the GeometricTools library (Wild Magic Engine version 5.17) through the WildMagic5p17 submodule. This code is licensed under the Boost Software License - Version 1.0. We include the pybind11 submodule to build the python wrappers. It is licensed under a BSD-3-Clause License. The websocketpp submodule is licensed unter a BSD-3-Clause License. The submodules AffAction, ESLib, Tropic, and Rcs come from ourselves and are licensed under the BSD-3-Clause License.