ISSN:
0885-6125
Keywords:
neural networks
;
concept learning
;
online algorithms
;
variational optimization
Source:
Springer Online Journal Archives 1860-2000
Topics:
Computer Science
Notes:
Abstract We review the application of statistical mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule.
Type of Medium:
Electronic Resource
URL:
http://dx.doi.org/10.1023/A:1007428731714
Permalink