GRADIENT DESCENT LEARNING IN PERCEPTRONS - A REVIEW OF ITS POSSIBILITIES

Abstract

We present a streamlined formalism which reduces the calculation of the generalization error for a perceptron, trained on random examples generated by a teacher perceptron, to a matter of simple algebra. The method is valid whenever the student perceptron can be identified as the unique minimum of a specific cost function. The asymptotic generalization error is calculated explicitly for a broad class of cost functions, and a specific cost function is singled out that leads to a generalization error extremely close to the one of the Bayes classifier.