Learning in the hypercube: A stepping stone to the binary perceptron

Abstract

The learning problem for storing random patterns in a perceptron with binary weights can be facilitated by pretraining an appropriate precursor network with continuous weights. Unlike previous studies which compare the performance of different continuous-weight perceptrons on the hypersphere (spherical constraint), we also consider weight vectors constrained to the volume of the hypercube (cubical constraint). We compare the performance of the maximally stable networks on the hypersphere and in the hypercube, and show that the latter is superior for predicting the weights of the maximally stable binary perceptron. We further determine an upper bound for the fraction of binary weights that any precursor is able to predict correctly, and introduce a precursor in the hypercube that closely approaches this upper bound. We finally demonstrate the value of this hypercube precursor by carrying out simulations for a perceptron with up to 100 weights.