Home

Awesome

Reinforcement learning in 3D

Implemented DQN [3] and A3C [4] algorithm for ViZDoom [1] and DeepMind Lab [2] environments.

Small network on small screen resolution trained relatively fast on simple maps:

DQN, ViZDoom map simpler_basic

ViZDoom map simpler_basic

DQN, DeepMind Lab map seekavoid_arena_01

ViZDoom map simpler_basic

Dependencies

How to run

ViZDoom

DeepMind Lab

py_binary(
    name = "agent_dqn",
    srcs = ["*path_work_dir*/agent_dqn.py"],
    data = [":deepmind_lab.so"],
    main = "*path_work_dir*/agent_dqn.py",
)

py_binary(
    name = "agent_a3c",
    srcs = ["*path_work_dir*/agent_a3c.py"],
    data = [":deepmind_lab.so"],
    main = "*path_work_dir*/agent_a3c.py",
)

Thanks

A3C is a little bit tricky algorithm and there are a lot of it's implementations already. So as reference I used implementation by Arthur Juliani.

References

[1] Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, Wojciech Jaśkowski. ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning. arXiv:1605.02097, 2016.

[2] Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, Julian Schrittwieser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg, Stig Petersen. DeepMind Lab. arXiv:1612.03801, 2016.

[3] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller. Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602, 2013.

[4] Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu. Asynchronous Methods for Deep Reinforcement Learning. arXiv:1602.01783, 2016.