Home

Awesome

RoSÉ: A Hardware-Software Co-Simulation Infrastructure Enabling Pre-Silicon Full-Stack Robotics SoC Evaluation

Robotic systems such as unmanned autonomous drones and self-driving cars have been widely deployed in many scenarios and have the potential to revolutionize the future generation of computing. To improve the performance and energy efficiency of robotic platforms, significant research efforts are being devoted to developing hardware accelerators for workloads that form bottlenecks in the robotics software pipeline.

Although domain-specific accelerators (DSAs) can offer improved efficiency over general-purpose processors on isolated robotics benchmarks, system-level constraints such as data movement and contention over shared resources could significantly impact the achievable acceleration in an end-to-end fashion.

In addition, the closed-loop nature of robotic systems, where there is a tight interaction across different deployed environments, software stacks, and hardware architecture, further exacerbates the difficulties of evaluating robotic SoCs.

To address this limitation, we develop RoSÉ, an open-source, hardware-software co-simulation infrastructure for full-stack, pre-silicon hardware-in-the-loop evaluation of robotics SoCs, together with the full software stack and realistic robotic environments created to support robotics RoSE captures the complex interactions across hardware, algorithm, and environment, enabling new architectural research directions in hardware-software co-design for robotic UAV systems.

Specifically, RoSÉ integrates AirSim, a software robotic environment simulator, supporting UAV models and FireSim, an FPGA-based RTL simulator, to capture closed-loop interactions between environments, algorithms, and hardware. The sensor environment data is passed from AirSim to FireSim to trigger the hardware pipeline, and the computed commands are generated from the FireSim hardware simulation and fed back to AirSim. RoSÉ accurately synchronizes the timing and data transfer between the AirSim environment simulator and the FireSim hardware simulator without significant degradation in simulation speed.

Our evaluation demonstrates that RoSÉ holistically captures the closed-loop interactions between environment, algorithms, and hardware, opening up new opportunities for systematic hardware-software co-design for robotic UAV systems.

In summary, this project makes the following contributions:

In summary, RoSÉ enables robotics UAV researchers and architects to comprehensively evaluate robotics UAV SoCs. RoSÉ captures a full-stack simulation of a robot by integrating the simulation of a robot UAV's environment and its RTL, enabling design space exploration of robot environments, algorithms, hardware, and system parameters within a unified simulation environment.

By using RoSÉ, researchers can better study and analyze the tradeoffs of robotic UAV systems without the overhead of taping out an SoC. Building upon RoSÉ by introducing new applications, robot UAV environments, and architectures will enable the agile development of robotics SoCs across diverse domains.

RoSÉ Tutorial

This artifact appendix describes how to use RoSÉ to run end-to-end robotics simulations, and how to reproduce experimental results.

The instructions assume that a user already has robotic applications and hardware configurations developed, and provides reference examples used in the evaluation of this work. While RoSÉ can be used to develop new hardware and software, instructions to do so are outside of the scope of the artifact evaluation.

Tutorial Meta-Information Checklist

Description

(1) How to access:

• The artifact consists of the core RoSÉ repos- itory, modifications to Firesim, Chipyard, and ONNX Runtime:

Additionally, this evaluation builds upon the following infrastructures. For the purpose of the evaluation, binaries for simulators built from Unreal Engine and AirSim are provided.

(2) Dependencies - Hardware

To run a full simulation with RoSÉ, access to a GPU and FPGA is required, although these can be hosted on separate computers. For this artifact evaluation, instructions for running simulations on a locally-provisioned FPGA are provided. However, RoSÉ can also be used using AWS EC2 FPGA instances (e.g. f1.2xlarge). In this artifact we provide build scripts for generating bitstreams for locally-provisioned FPGAs.

Additionally, GPU access is needed in order to run robotics environment simulations with rendering. For this evaluation, AirSim binaries packaged using Unreal Engine are provided.

To use RoSÉ in the cloud, ne AWS EC2 c5.4xlarge instance (also referred to as “manager” instance), and one f1.2xlarge instance is required. The latter will be launched automatically by FireSim’s manager.

(3) Dependencies - Software

Use ssh or mosh on your local machine to remote access evaluation instances. All other requirements are automatically installed by scripts in the following sections.

Installation

To begin installation, clone the repository:

    git clone https://github.com/ucb-bar/RoSE.git
    cd RoSE
    git checkout main

FireSim Installation

Begin by installing FireSim by running the following commands within the RoSÉ repository.

    git submodule update --init ./soc/sim/firesim
    cd ./soc/sim/firesim
    ./scripts/machine-launch-script.sh --prefix [INSERT_CONDA_DIRECTORY_HERE]
    ./build-setup.sh
    source sourceme-manager.sh --skip-ssh-setup
    firesim managerinit --platform xilinx_alveo_u250

RoSÉ Installation

Begin by cloning RoSÉ in the project directory:

Next, within RoSÉ, run the setup script to set the proper environment variables. Make sure to run this script whenever starting a new interactive shell.

    source rose-setup.sh

After this is complete, run the following script to patch FireSim and Chipyard to support RoSÉ, and to instantiate submodules.

    bash soc/setup.sh

After this setup is complete, run the following script to build binaries for the trail-navigation controllers evaluated in Section IV for generating RISC-V Fedora images containing the controllers and ONNX models.

   bash soc/build.sh

Next, run the following script to install dependencies and configure parameters for the RoSÉ deployment scripts, using the IP address of the GPU system that will be used to run the provided AirSim binaries.

source deploy/setup.sh [AIRSIM IP]

Bitstream Generation

To build bitstreams for Rocket+Gemmini and BOOM+Gemmini configurations, run the following.

   bash soc/buildbitstreams.sh

DNN Training

This artifact provides pre-trained models for evaluation. To train new classifier DNNs using the provided datasets, run the following, selecting between the given ResNet configurations. Each training run will output an ONNX model named trail_dnn_resnet[xy].onnx.

    bash env/train/train_resnet.py (6|11|14|18|34|50)

Finally, the steps for building custom Unreal Engine maps are out of the scope of this evaluation. However, new environments can be built using the documentation provided at (https://microsoft.github.io/AirSim/build_linux/).

Experiment Workflow

Now that the environment has been set up and the target hardware and software have been built, one can run the experiments in this work by launching an AirSim simulation and running the following scripts. All the experiments can be executed by running run-all.sh. This will generate CSV files as well as videos recorded from the front-facing camera of the simulated UAV in deploy/hephaestus/logs/.

   bash deploy/scripts/run-all.sh

To run individual experiments corresponding to the figures in this work, the following scripts are also provided (which are all included in the main script).

• Figure 10:

   bash deploy/scripts/tunnel-exp.sh

• Figures 15, 16:

   bash deploy/scripts/rose-perf-sync-only.sh

   bash deploy/scripts/rose-perf-tunnel-exp.sh

• Figures 11, 14:

    bash deploy/scripts/rose-hw-sw-sweep.sh

• Figure 12:

   bash deploy/scripts/rose-velocity-sweep.sh

• Figure 13:

   bash deploy/scripts/rose-dynamic-exp.sh

Figures and Evaluation

After executing the prior experiments, figures can be generated using the CSV outputs by running the following command. The figures will be available as in deploy/figures/.

   python3 deploy/scripts/generate-figures.py

Experiment Customization

• Building New FPGA Images In addition to the provided SoC configurations, users can evaluate other designs. To evaluate new designs, refer to the Chipyard documentation, as well as the example RoSÉ-annotated configs found in soc/src/main/scala/RoSEConfigs.scala.

• Designing AirSim Environments If users install Unreal Engine as well as AirSim, it is possible to create new maps/environments for robot agents to interact with. By default, one can modify the Blocks environment provided by AirSim. Additional assets and maps can be designed by users, or obtained from the Unreal Marketplace.

• Changing Simulation Parameters RoSEprovides flags that can be used to select different simulation parameters. To view available parameters for deploying simulations, refer to deploy/hephaestus/runner.py. Example configurations include changing simulation granularity, or deploying a car vs a drone simulation.

Additionally, new controller ONNX models can be trained using the provided dataset and evaluated using the provided drone_test executable.

Citing RoSÉ

@inproceedings{rose-isca,
  title={RoS{\'E}: A Hardware-Software Co-Simulation Infrastructure Enabling Pre-Silicon Full-Stack Robotics SoC Evaluation},
  author={Nikiforov, Dima and Dong, Shengjun Kris and Zhang, Chengyi Lux and Kim, Seah and Nikolic, Borivoje and Shao, Yakun Sophia},
  booktitle={Proceedings of the 50th Annual International Symposium on Computer Architecture},
  pages={1--15},
  year={2023}
}