Home

Awesome

Competition: Fostering Global Cooperation to Mitigate Climate Change

PyTorch 1.9.0 Python 3.7 Warp drive 1.7.0 Ray 1.0.0 Paper Code Tutorial

<a href="https://www.kaggle.com/kernels/fork-version/105300459"><img src="https://www.kaggle.com/static/images/site-logo.svg" align="left" height="40" width="40" ></a>(Code Tutorial Notebook on Kaggle with free GPU available)

This is the code respository for the competition on modeling global cooperation in the RICE-N Integrated Assessment Model. This competition is co-organized by MILA and Salesforce Research.

The RICE-N IAM is an agent-based model that incorporates DICE climate-economic dynamics and multi-lateral negotiation protocols between several fictitious nations.

In this competition, you will design negotiation protocols and contracts between nations. You will use the simulation and agents to evaluate their impact on the climate and the economy.

We recommend that GPU users use warp_drive and CPU users use rllib.

Tutorial

Resources

Installation

Notice: we recommend using Linux or MacOS. For Windows users, we recommend to use virtual machine running Ubuntu 20.04 or Windows Subsystem for Linux.

You can get a copy of the code by cloning the repo using Git:

git clone https://github.com/mila-iqia/climate-cooperation-competition
cd climate-cooperation-competition

As an alternative, one can also use:

git clone https://e.coding.net/ai4climatecoop/ai4climatecoop/climate-cooperation-competition.git
cd climate-cooperation-competition

We recommend using a virtual environment (such as provided by virtualenv or Anaconda).

You can install the dependencies using pip:

pip install -r requirements.txt

Get Started

Then run the getting started Jupyter notebook, by starting Jupyter:

jupyter notebook

and then navigating to getting_started.ipynb.

It provides a quick walkthrough for registering for the competition and creating a valid submission.

Training with reinforcement learning

RL agents can be trained using the RICE-N simulation using one of these two frameworks:

  1. RLlib: The pythonic environment can be trained on your local CPU machine using open-source RL framework, RLlib.
  2. WarpDrive: WarpDrive is a GPU-based framework that allows for over 10x faster training compared to CPU-based training. It requires the simulation to be written out in CUDA C, and we also provide a starter version of the simulation environment written in CUDA C (rice_step.cu)

We also provide starter scripts to train the simulation you build with either of the above frameworks.

Note that we only allow these two options, since our backend submission evaluation process only supports these at the moment.

For training with RLlib, rllib (1.0.0), torch (1.9.0) and gym (0.21) packages are required.

For training with WarpDrive, the rl-warp-drive (>=1.6.5) package is needed.

Note that these requirements are automatically installed (or updated) when you run the corresponding training scripts.

Docker image (for GPU users)

We have also provided a sample dockerfile for your reference. It mainly uses a Nvidia PyTorch base image, and installs the pycuda package as well. Note: pycuda is only required if you would like to train using WarpDrive.

Docker image (for CPU users)

Thanks for the contribution from @muxspace. We also have an end-to-end docker environment ready for CPU users. Please refer to README_CPU.md for more details.

Customizing and running the simulation

See the Colab_Tutorial.ipynb. Open In Colab for details.

It provides examples on modifying the code to implement different negotiation protocols. It describes ways of changing the agent observations and action spaces corresponding to the proposed negotiation protocols and implementing the negotiation logic in the provided code.

The notebook has a walkthrough of how to train RL agents with the simulation code and how to visualize results from the simulation after running it with a set of agents.

For those who have limited access to Colab, please try to use free GPUs on Kaggle. Please notice that Kaggle platform requires mobile phone verification to be able to access the GPUs. One may find the settings to get GPUs and internet connect on the right hand side after clicking on the link above and login.

Training RL agents in your simulation

Once you build your simulation, you can use either of the following scripts to perform training.

USAGE: The training script (with RLlib) is invoked using (from the root directory)

    python scripts/train_with_rllib.py

USAGE: The training script (with WarpDrive) is invoked using

    python scripts/train_with_warpdrive.py

As training progresses, some key metrics (such as the mean episode reward) are printed on screen for your reference. At the end of training, a zipped submission file is automatically created and saved for your reference. The zipped file essentially comprises the following

Contributing

We are always looking for contributors from various domains to help us make this simulation more realistic.

If there are bugs or corner cases, please open a PR detailing the issue and consider submitting to Track 3!

Citation

To cite this code, please use the information in CITATION.cff and the following bibtex entry:

@software{Zhang_RICE-N_2022,
author = {Zhang, Tianyu and Srinivasa, Sunil and Williams, Andrew and Phade, Soham and Zhang, Yang and Gupta, Prateek and Bengio, Yoshua and Zheng, Stephan},
month = {7},
title = {{RICE-N}},
url = {https://github.com/mila-iqia/climate-cooperation-competition},
version = {1.0.0},
year = {2022}
}

@misc{https://doi.org/10.48550/arxiv.2208.07004,
  doi = {10.48550/ARXIV.2208.07004},
  url = {https://arxiv.org/abs/2208.07004},
  author = {Zhang, Tianyu and Williams, Andrew and Phade, Soham and Srinivasa, Sunil and Zhang, Yang and Gupta, Prateek and Bengio, Yoshua and Zheng, Stephan},
  title = {AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-N},  
  publisher = {arXiv},
  year = {2022}
}

License

For license information, see LICENSE.txt.