tailieunhanh - A study on efficient transfer learning for reinforcement learning using sparse coding

In the experiments, we have adopted colored mazes as tasks and confirmed that our proposed method significantly improved in terms of jumpstart and of the reduction of the total learning cost, compared with normal Q-learning. | Journal of Automation and Control Engineering Vol. 4, No. 4, August 2016 A Study on Efficient Transfer Learning for Reinforcement Learning Using Sparse Coding Midori Saito and Ichiro Kobayashi Advanced Sciences, Graduate School of Humanities and Sciences, Ochanomizu University, Tokyo, Japan Email: {, koba}@ Abstract—By applying the knowledge previously obtained by reinforcement learning to new tasks, transfer learning has been successful in achieving efficient learning, rather than re-learning knowledge about action policies from scratch. However, in the case of applying transfer learning to reinforcement learning, it is not easy to determine which and how much the obtained knowledge should be transferred. With this background, in this study, we propose a novel method that enables to decide the knowledge and to determine the ratio of transference by adopting sparse coding in transfer learning. The transferred knowledge is represented as a linear combination of the accumulated knowledge by means of sparse coding. In the experiments, we have adopted colored mazes as tasks and confirmed that our proposed method significantly improved in terms of jumpstart and of the reduction of the total learning cost, compared with normal Q-learning. Index Terms—sparse coding, reinforcement learning, maze transfer are transferred. By this, we aim to make possible to efficiently transfer action policies to new target tasks. II. RELATED STUDIES This section presents the related studies to our study, in particular, the ones focus on transfer learning employed in reinforcement learning and also sparse coding employed in transfer learning. First, as for the transfer learning employed in reinforcement learning, it has been successful in generalizing information across multiple tasks. Even though between two different tasks, transferring the knowledge an agent has learned in a source task is useful in a target task [6]. Fernando et al. [3] employed Policy .