On-line learning of unrealizable tasks

Abstract

The dynamics of on-line learning is investigated for structurally unrealizable tasks in the context of two-layer neural networks with an arbitrary number of hidden neurons. Within a statistical mechanics framework, a closed set of differential equations describing the learning dynamics can be derived, for the general case of unrealizable isotropic tasks. In the asymptotic regime one can solve the dynamics analytically in the limit of large number of hidden neurons, providing an analytical expression for the residual generalization error, the optimal and critical asymptotic training parameters, and the corresponding prefactor of the generalization error decay.

Publication DOI: https://doi.org/10.1103/PhysRevE.60.5902
Divisions: College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright of the American Physical Society
Uncontrolled Keywords: on-line learning,neural networks,neurons,asymptotic regime one,residual generalization error,asymptotic training parameters,generalization error decay,Physics and Astronomy(all),Condensed Matter Physics,Statistical and Nonlinear Physics,Mathematical Physics
Publication ISSN: 1550-2376
Last Modified: 05 Feb 2024 08:09
Date Deposited: 30 Jul 2009 10:50
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
http://link.aps ... hysRevE.60.5902 (Publisher URL)
PURE Output Type: Article
Published Date: 1999-11
Authors: Scarpetta, Silvia
Saad, David (ORCID Profile 0000-0001-9821-2623)

Download

[img]

Version: Published Version


Export / Share Citation


Statistics

Additional statistics for this record