On the utility of dreaming: a general model for how learning in artificial agents can benefit from data hallucination

Windridge, David ORCID: https://orcid.org/0000-0001-5507-8516, Svensson, Henrik and Thill, Serge ORCID: https://orcid.org/0000-0003-1177-4119 (2020) On the utility of dreaming: a general model for how learning in artificial agents can benefit from data hallucination. Adaptive Behavior . ISSN 1059-7123 (Published online first) (doi:10.1177/1059712319896489)

[img]
Preview
PDF - Published version (with publisher's formatting)
Available under License Creative Commons Attribution-NonCommercial.

Download (785kB) | Preview

Abstract

We consider the benefits of dream mechanisms – that is, the ability to simulate new experiences based on past ones – in a machine learning context. Specifically, we are interested in learning for artificial agents that act in the world, and operationalize “dreaming” as a mechanism by which such an agent can use its own model of the learning environment to generate new hypotheses and training data.

We first show that it is not necessarily a given that such a data-hallucination process is useful, since it can easily lead to a training set dominated by spurious imagined data until an ill-defined convergence point is reached. We then analyse a notably successful implementation of a machine learning-based dreaming mechanism by Ha and Schmidhuber (Ha, D., & Schmidhuber, J. (2018). World models. arXiv e-prints, arXiv:1803.10122). On that basis, we then develop a general framework by which an agent can generate simulated data to learn from in a manner that is beneficial to the agent. This, we argue, then forms a general method for an operationalized dream-like mechanism.

We finish by demonstrating the general conditions under which such mechanisms can be useful in machine learning, wherein the implicit simulator inference and extrapolation involved in dreaming act without reinforcing inference error even when inference is incomplete.

Item Type: Article
Keywords (uncontrolled): Artificial dream mechanisms, data simulation, machine learning, reinforcement learning
Research Areas: A. > School of Science and Technology > Computer Science
Item ID: 28755
Notes on copyright: Copyright © The Author(s) 2020.
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Useful Links:
Depositing User: David Windridge
Date Deposited: 15 Jan 2020 13:45
Last Modified: 24 Jan 2020 17:50
URI: https://eprints.mdx.ac.uk/id/eprint/28755

Actions (login required)

Edit Item Edit Item

Full text downloads (NB count will be zero if no full text documents are attached to the record)

Downloads per month over the past year