On the utility of dreaming: a general model for how learning in artificial agents can benefit from data hallucination
Windridge, David 
ORCID: https://orcid.org/0000-0001-5507-8516, Svensson, Henrik and Thill, Serge ORCID: https://orcid.org/0000-0003-1177-4119
(2021)
On the utility of dreaming: a general model for how learning in artificial agents can benefit from data hallucination.
Adaptive Behavior, 29
(3)
.
pp. 267-280.
ISSN 1059-7123
[Article]
(doi:10.1177/1059712319896489)
|
PDF
- Published version (with publisher's formatting)
Available under License Creative Commons Attribution-NonCommercial 4.0. Download (785kB) | Preview |
Abstract
We consider the benefits of dream mechanisms – that is, the ability to simulate new experiences based on past ones – in a machine learning context. Specifically, we are interested in learning for artificial agents that act in the world, and operationalize “dreaming” as a mechanism by which such an agent can use its own model of the learning environment to generate new hypotheses and training data.
We first show that it is not necessarily a given that such a data-hallucination process is useful, since it can easily lead to a training set dominated by spurious imagined data until an ill-defined convergence point is reached. We then analyse a notably successful implementation of a machine learning-based dreaming mechanism by Ha and Schmidhuber (Ha, D., & Schmidhuber, J. (2018). World models. arXiv e-prints, arXiv:1803.10122). On that basis, we then develop a general framework by which an agent can generate simulated data to learn from in a manner that is beneficial to the agent. This, we argue, then forms a general method for an operationalized dream-like mechanism.
We finish by demonstrating the general conditions under which such mechanisms can be useful in machine learning, wherein the implicit simulator inference and extrapolation involved in dreaming act without reinforcing inference error even when inference is incomplete.
| Item Type: | Article |
|---|---|
| Keywords (uncontrolled): | Artificial dream mechanisms, data simulation, machine learning, reinforcement learning |
| Research Areas: | A. > School of Science and Technology > Computer Science |
| Item ID: | 28755 |
| Notes on copyright: | Copyright © The Author(s) 2020.
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). |
| Useful Links: | |
| Depositing User: | David Windridge |
| Date Deposited: | 15 Jan 2020 13:45 |
| Last Modified: | 09 Feb 2022 10:35 |
| URI: | https://eprints.mdx.ac.uk/id/eprint/28755 |
Actions (login required)
 |
View Item |
Statistics
Additional statistics are available via IRStats2.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)