A comparison of action selection methods for implicit policy method reinforcement learning in continuous action-space

Nichols, Barry D. ORCID: https://orcid.org/0000-0002-6760-6037 (2016) A comparison of action selection methods for implicit policy method reinforcement learning in continuous action-space. In: International Joint Conference on Neural Networks (IJCNN 2016), 24-29 Jul 2016, Vancouver, Canada. (doi:10.1109/IJCNN.2016.7727688)

PDF - Final accepted version (with author's formatting)
Download (338kB) | Preview


In this paper I investigate methods of applying reinforcement learning to continuous state- and action-space problems without a policy function. I compare the performance of four methods, one of which is the discretisation of the action-space, and the other three are optimisation techniques applied to finding the greedy action without discretisation. The optimisation methods I apply are gradient descent, Nelder-Mead and Newton's Method. The action selection methods are applied in conjunction with the SARSA algorithm, with a multilayer perceptron utilized for the approximation of the value function. The approaches are applied to two simulated continuous state- and action-space control problems: Cart-Pole and double Cart-Pole. The results are compared both in terms of action selection time and the number of trials required to train on the benchmark problems.

Item Type: Conference or Workshop Item (Paper)
Research Areas: A. > School of Science and Technology
A. > School of Science and Technology > Computer Science
Item ID: 19845
Notes on copyright: Full text: © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Useful Links:
Depositing User: Barry Nichols
Date Deposited: 19 May 2016 10:14
Last Modified: 31 May 2019 01:27
ISBN: 9781509006205
URI: https://eprints.mdx.ac.uk/id/eprint/19845

Actions (login required)

Edit Item Edit Item

Full text downloads (NB count will be zero if no full text documents are attached to the record)

Downloads per month over the past year