Developmental Machine Learning: Machines that learn like children … and help children learn better

Pierre-Yves Oudeyer, Inria and Ensta ParisTech, France
ICLR 2019 Keynote talk

Keywords: developmental machine learning, models of child development, autonomous exploration, curiosity, intrinsic motivation, automated curriculum learning, tool use, language acquisition, robotics, cognitive sciences.

Abstract:

Current approaches to AI and machine learning are still fundamentally limited in comparison with autonomous learning capabilities of children. What is remarkable is not that some children become world champions in certains games or specialties: it is rather their autonomy, flexibility and efficiency at learning many everyday skills under strongly limited resources of time, computation and energy. And they do not need the intervention of an engineer for each new task (e.g. they do not need someone to provide a new task specific reward function).

I will present a research program that has focused on computational modeling of child development and learning mechanisms in the last decade. I will discuss several developmental forces that guide exploration in large real world spaces, starting from the perspective of how algorithmic models can help us understand better how they work in humans, and in return how this opens new approaches to autonomous machine learning.

In particular, I will discuss models of curiosity-driven autonomous learning, enabling machines to sample and explore their own goals and their own learning strategies, self-organizing a learning curriculum without any external reward or supervision.

I will show how this has helped scientists understand better aspects of human development such as the emergence of developmental transitions between object manipulation, tool use and speech. I will also show how the use of real robotic platforms for evaluating these models has led to highly efficient unsupervised learning methods, enabling robots to discover and learn multiple skills in high-dimensions in a handful of hours. I will discuss how these techniques are now being integrated with modern deep learning methods.

Finally, I will show how these models and techniques can be successfully applied in the domain of educational technologies, enabling to personalize sequences of exercises for human learners, while maximizing both learning efficiency and intrinsic motivation. I will illustrate this with a large-scale experiment recently performed in primary schools, enabling children of all levels to improve their skills and motivation in learning aspects of mathematics.

Web: http://www.pyoudeyer.com

Key references:

[1] Colas, C., Sigaud, O., Oudeyer, P-Y. (2019) CURIOUS : Intrinsically Motivated Modular, Multi- Goal Reinforcement Learning, Proceedings of International Conference on Machine Learning (ICML 2019). https://arxiv.org/abs/1810.06284

[2] Gottlieb, J.,Oudeyer, P-Y. (2018) Towards a neuroscience of active sampling and curiosity, Nature Reviews Neuroscience, 19(758–770).

[3] Laversanne-Finot, A., Péré, A., Oudeyer, P-Y. (2018) Curiosity Driven Exploration of Learned Disentangled Goal Spaces, in Proceedings of Conference on Robot Learning, CoRL 2018.
https://arxiv.org/abs/1807.01521

[4] Colas, C., Sigaud, O., Oudeyer, P. Y. (2018). GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms. In Proceedings of International Conference on Machine Learning (ICML 2018), arXiv preprint arXiv:1802.05054. https://arxiv.org/abs/1802.05054

[5] Péré, A., Forestier, S., Sigaud, O., Oudeyer, P-Y. (2018) Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration, in proceedings of International Conference on Learning Representations (ICLR 2018), Vancouver, CA.

[6] Forestier S, Oudeyer P-Y. (2017) Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning. https://arxiv.org/abs/1708.02190

[7] Clement, B., Roy, D., Oudeyer, P-Y. *, Lopes, M. * (2015) Multi-Armed Bandits for Intelligent Tutoring Systems, Journal of Educational Data Mining (JEDM), Vol 7, No 2. http://www.pyoudeyer.com/JEDMClementetal15.pdf

[8] Baranes, A., Oudeyer, P-Y. (2013) Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots, Robotics and Autonomous Systems, 61(1), pp. 49-73. http://www.pyoudeyer.com/ActiveGoalExploration-RAS-2013.pdf

[9] Oudeyer P-Y, Kaplan , F. and Hafner, V. (2007) Intrinsic Motivation Systems for Autonomous Mental Development, IEEE Transactions on Evolutionary Computation, 11(2), pp. 265–286. http://www.pyoudeyer.com/ims.pdf

[10] Oudeyer, P-Y. (2006) Self-Organization in the Evolution of Speech, Oxford University Press

source