Generalization error in a self-similar committee machine

Abstract

We derive the form and properties of the generalization error for a self-similar feedforward network (constructed through an iterative procedure of test set partitioning combined with majority voting). We discuss the universality of these results and conjecture that the generalization error rate epsilon(alpha) as a function of the amount of training alpha never decreases faster than epsilon(alpha) - 1/2 is similar to square-root alpha in the limit alpha --> 0. We show how the self-similar architecture can be used to improve the generalization performance.