Home

Awesome

Force

About

Force is a library for deep reinforcement learning (RL) research, built on PyTorch and Gymnasium. It is under active development. Features at present:

The name Force was originally derived from the word reinforcement, but, in a fun coincidence, it is also related to the name of my PhD advisor via Newton's second law of motion.

Installation

Clone the repository, run the following commands from the root directory of the repository:

pip install -r requirements.txt
pip install -e .

Usage

An example of how to use the library can be found in scripts/train.py. For anything more complicated, you can easily define your own logic by subclassing Experiment.

The script can be called like so:

python scripts/train.py --root-dir ROOT_DIR --domain DOMAIN --config CONFIG_PATH

where the all-caps variables are substituted appropriately:

Multiple config files can be used by repeating the --config (abbr. -c) flag.

To override specific entries in the config, use the --set (abbr. -s) flag, which takes two arguments, the key and the value. The key name may contain dots to denote nesting, for example -s agent.init_alpha 0.5 if using a SAC agent.

You can optionally set a specific random seed by passing --seed SEED. Otherwise, a seed will be randomly chosen.

Viewing experiments

The GUI uses a client-server architecture because the experiment logs typically live on a remote machine. To launch the server, simply point it to the directory:

python force/workflow/result_server.py -d DIRECTORY

You can optionally set a specific port using the --port (-p) flag.

From the client side, you can view various info about each experiment and access the config and log files. URL query strings can be used to specify a quantity to plot (e.g. ?plot=eval/return_mean) and to filter by domain, algorithm, recency, or job status.