Unsupervised learning by examples: On-line versus off-line

Abstract

We study both on-line and off-line unsupervised learning from p random patterns which are uniformly distributed on the N-sphere with the exception of a single symmetry breaking orientation B, along which they may be arbitrarily distributed. Supervised learning from the same kind of patterns is included as a special case. In the thermodynamic limit N --> infinity with alpha = p/N fixed we calculate the overlap R(alpha)= B . J/\J\\B\ between the unknown ''true'' B and the optimal ''Bayes'' hypothesis J with particular emphasis on the small and large cu asymptotics and the phenomenon of retarded learning. Finally, we identify a cost function whose minimum reproduces the off-line Bayes overlap.