Abstract
A classifier system for the reinforcement learning control of autonomous mobile robots is proposed. The classifier system contains action selection, rules reproduction, and credit assignment mechanisms. An important feature of the classifier system is that it operates with continuous sensor and action spaces. The system is applied to the control of mobile robots. The local controllers use independent classifiers specified at the wheel-level. The controllers work autonomously, and with respect to each other represent dynamic systems connected through the external environment. The feasibility of the proposed system is tested in an experiment with a Khepera robot. It is shown that some patterns of global behavior can emerge from locally organized classifiers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Connel J, Mahadevan S (eds) (1993) Robot learning. Kluwer, Boston
Kaelbling L, Littman M, Moore A (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285
Asada M, Noda S, Hosoda K (1997) Action-based state space construction for robot learning (in Japanese). J Robotics Soc Jpn 15:76–82
Watkins C (1989) Learning from delayed rewards. PhD Thesis, University of Cambridge
Sutton R (1990) Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Proceedings of the 7th International Conference on Machine Learning, pp 216–224
Naruse K, Leu M (1997) Autonomous vehicle navigation by layered learning and planning. In: Proceedings of the 29th CIRP International Seminar on Manufacturing Systems, Osaka, pp 87–92
Deng K, Moore A (1995) Multiresolution instance-based learning. In: Proceedings of the International Joint Conference on Artificial Intelligence
Moore A, Atkeson C (1995) The parti-game algorithm for variable resolution reinforcement learning in multidimensional state space. Mach Learn 21:1–36
McCallum R (1996) Hidden state and reinforcement learning with instance-based state identification. IEEE Trans System Man Cybern Part B, 26:464–473
Murao H, Kitamura S (1997) An incremental quantization method of the continuous sensor space for learning agents. Mem Fac Eng Kobe Univ 44:155–164
Murao H, Kitamura S (1998) An adaptive state space design for reinforcement learning. In: Sugisaka M (ed) Proceedings of the International Sympasium on Artificial Life and Robotics (AROB 3), vol 1, pp 85–88
Goldberg D (1989) Genetic algorithms in search, optimization and machine learning, Addison-Wesley, Reading
Holland J, Holyoak K, Nisbett R et al. (1989) Induction. Processes of inference, learning, and discovery. MIT Press, Cambridge
Holland J (1995) Hidden order. How adaptation builds complexity. Addison-Wesley, New York
Nakamura Y, Ohnishi S, Okhura K et al. (1997) Instance-based reinforcement learning for robot path finding in continuous space. In: Proceedings of the IEEE International Conference on System, Man, and Cybernetics, pp 1228–1234
Kuroyama K, Svinin M, Nakamura Y et al. (1998) Learning control of autonomous robots with the use of an instance-based classifier generator in the continuous state space. In: Sugisaka M (ed) Proceedings of the International Sympasium on Artificial Life and Robotics (AROB 3), vol 1, pp 89–92
Holland J (1985) Properties of the bucket brigade algorithm. In: Proceedings of the 1st International Conference on Genetic Algorithms and their Applications, pp 1–7
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Svinin, M., Kuroyama, K., Ueda, K. et al. Learning control of autonomous robots using an instance-based classifier generator in continuous state space. Artif Life Robotics 3, 90–96 (1999). https://doi.org/10.1007/BF02481253
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02481253