Awesome

BeliefGridValueIteration

An offline POMDP solver from "Computationally Feasible Bounds for Partially Observed Markov Decision Processes" (1991), by W. S. Lovejoy. It computes an upper bound on the value function by performing value iteration on a discretized belief space.

Installation

Install using the standard package manager:

using Pkg
Pkg.add("BeliefGridValueIteration")

Usage

using POMDPs
using POMDPModels # for the tiger pomdp problem
using BeliefGridValueIteration

pomdp = TigerPOMDP()

solver = BeliefGridValueIterationSolver(m = 2, verbose=true)

policy = solve(solver, pomdp)

# Evaluate the value at a given belief point
b0 = [0.5, 0.5]
value(policy, b0)

Documentation

Solver Options:

• m::Int64 = 1 Granularity of the belief grid for the triangulation
• precision::Float64 = 0.0 The solver stops when the desired convergence precision is reached
• max_iterations::Int64 = 100 Number of iteration of value iteration
• verbose::Bool = false whether or not the solver prints information

Requirements:

This should return a list of the following functions to be implemented for your POMDP to be solved by this solver:

@requirements_info BeliefGridValueIterationSolver() YourPOMDP()

Acknowledgements

The authors thank Tim Wheeler and Mykel Kochenderfer for providing a starter implementation of this code.