Awesome

TradeMaster: An RL Platform for Trading

TradeMaster is a first-of-its kind, best-in-class open-source platform for quantitative trading (QT) empowered by reinforcement learning (RL).

It covers the full pipeline for the design, implementation, evaluation and deployment of RL-based trading methods. It contains: 1) a toolkit for efficient data collection, preprocessing and analysis; 2) a high-fidelity data-driven market simulator for mainstream QT tasks (e.g., portfolio management and algorithmic trading); 3) standard implementations of over 10 novel FinRL methods; 4) a systematic evaluation benchmark called PRUDEX-Compass.

Outline

• TradeMaster: An RL Platform for Trading
  • Outline
  • Overview
  • Installation
  • Tutorial
  • Toolkit
  • Results and Visualization
  • Model Zoo
  • Dataset
  • External Data Source
  • How to Use Your Own Data
  • File Structure
  • Publications
  • Contact
  • Join Us
  • Competition

Overview

TradeMaster could be beneficial to a wide range of communities including leading trading firms, startups, financial service providers and personal investors. We hope TradeMaster can make a change for the whole pipeline of FinRL to prevent untrustworthy results and lead successful industry deployment.

Installation

We provide a video tutorial of using docker to build a proper environment of running this project.

To help you better understand the step discribed in the video, Here are the installation tutorials for different operating systems:

• Docker <font color='red'> (Recommended) </font>
• MacOS
• Linux
• Windows

Tutorial

We provide tutorials for users to get start with.

• Colab Version: Use Google Colab resource to run TradeMaster on Cloud

Toolkit

• CSDI for financial data imputation (link)
• Automatic market style recognition (link)

Results and Visualization

The evaluation module of TradeMaster is mainly based on PRUDEX-Compass, a systematic evaluation toolkit of FinRL methods with 6 axes and 17 measures. We show some results here:

PRUDEX-Compass provides an intuitive visual means to give readers a sense of comparability and positioning of FinRL methods. The inner level maps out the relative strength of FinRL methods in terms of each axis, whereas the outer level provides a compact way to visually assess which set-up and evaluation measures are practically reported to point out how comprehensive the evaluation are for FinRL algorithms.

PRIDE-Star is a star plot to evaluate profitability,risk-control and diversity. It contains the normalized score of 8 measures.

Rank distribution plot is a bar plot, where the i-th column in the rank distribution shows the probability that a given method is assigned rank i in the corresponding metrics.

Performance profile reports FinRL methods' score distribution of all runs across the different financial markets that are statistically unbiased and more robust to outliers.

For more information of the usage of this part, please refer to this tutorial and this project

Model Zoo

DeepScalper based on Pytorch (Shuo Sun et al, CIKM 22)

OPD based on Pytorch (Fang et al, AAAI 21)

DeepTrader based on Pytorch (Wang et al, AAAI 21)

SARL based on Pytorch (Yunan Ye et al, AAAI 20)

ETTO based on Pytorch (Lin et al, 20)

Investor-Imitator based on Pytorch (Yi Ding et al, KDD 18)

EIIE based on Pytorch (Jiang et al, 17)

Classic RL based on Pytorch and Ray: PPO A2C SAC DDPG DQN PG TD3

Dataset

OHLCV: open, high, low, and close prices; volume: corresponding trading volume

External Data Source

Users may download data from the following data source with personal account:

How to Use Your Own Data

TradeMaster supports financial data with open, high, low, close, volume (OHLCV) raw informations as:

We compute 10 technical indicators to describe the financial markets:

Users can adapt their data with prefered features by changing the data loading and feature calculation part with corresponding input and output size. We plan to support limit order book (LOB) and altervative data such as text and graph in the future.

File Structure

|-- agent
|   |-- ClassicRL
|   |-- DeepScalper
|   |-- DeepTrader
|   |-- EIIE
|   |-- Investor_Imitator
|   |-- SARL
|-- config
|   |-- input_config
|   |-- output_config
|-- data
|   |-- download_data.py
|   |-- preprocess.py
|   |-- data
|       |-- BTC
|       |-- dj30
|       |-- exchange
|       |-- sz50
|-- env
|   |-- AT
|   |-- OE
|   |-- PM
|-- experiment
|-- figure
|-- result
|-- tutorial
|   |-- ClassRL_for_PM_on_FX.ipynb
|   |-- DeepScalper_for_AT_on_Bitcoin.ipynb
|   |-- EIIE_for_PM_on_DJ30.ipynb
|   |-- IMIT_for_PM_on_DJ30.ipynb
|   |-- SARL_for_PM_on_DJ30.ipynb
|   |-- Visualization.ipynb
|-- visualization
|   |-- compass
|   |-- exen
|   |-- ocatgon
|   |-- performance_profile
|   |-- rank
|-- README.md
|-- requirement.txt

Publications

Deep Reinforcement Learning for Quantitative Trading: Challenges and Opportunities (IEEE Intelligent Systems 2022)

DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities (CIKM 2022)

Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management (AAAI 21)

Reinforcement Learning for Quantitative Trading (Survey)

PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets

Contact

• This repository is developed and maintained by AMI group at Nanyang Technological University
• If you want to make contributions to TradeMaster, please create pull requests.

Join Us

We have positions for software engineer, RA and postdoc. If you are interested in working at the intersection of RL and financial trading, feel free to send an email to shuo003@e.ntu.edu.sg with your CV.

Competition

TradeMaster Cup 2022