This title is printed to order. This book may have been self-published. If so, we cannot guarantee the quality of the content. In the main most books will have gone through the editing process however some may not. We therefore suggest that you be aware of this before ordering this book. If in doubt check either the author or publisher’s details as we are unable to accept any returns unless they are faulty. Please contact us if you have any questions.
Continual learning remains a long-standing challenge. Success requires continuously ingesting new knowledge while retaining old knowledge that remains useful. More generally, an agent needs to efficiently accumulate knowledge to develop increasingly sophisticated skills over a long lifetime. Existing incremental machine learning techniques fall short of these ambitions of continual learning, as a major challenge has been to develop scalable systems that judiciously control what information they ingest, retain, or forget.
An agent that accumulates knowledge to develop increasingly sophisticated skills over a long lifetime could advance the frontier of artificial intelligence capabilities. The design of such agents, which remains a longstanding challenge, is addressed by the subject of continual learning. This monograph clarifies and formalizes concepts of continual learning, introducing a framework and tools to stimulate further research. Also presented are a range of empirical case studies to illustrate the roles of forgetting, relearning, exploration, and auxiliary learning.
Metrics presented in previous literature for evaluating continual learning agents tend to focus on particular behaviors that are deemed desirable, such as avoiding catastrophic forgetting, retaining plasticity, relearning quickly, and maintaining low memory or compute footprints. In order to systematically reason about design choices and compare agents, a coherent, holistic objective that encompasses all such requirements would be helpful. To provide such an objective, in this book continual learning is cast as reinforcement learning with limited compute resources. In particular, the continual learning objective is posed to be the maximization of infinite-horizon average reward subject to a computational constraint. Continual supervised learning, for example, is a special case of general formulation where the reward is taken to be negative log-loss or accuracy. Among the implications of maximizing average reward are that remembering all information from the past is unnecessary, forgetting non-recurring information is not "catastrophic," and learning about how an environment changes over time is useful.
