Feature and Label Association Based on Granulation Entropy for Deep Neural Networks

Abstract

Pooling layers help reduce redundancy and the number of parameters before building a multilayered neural network that performs the remaining processing operations. Usually, pooling operators in deep learning models use an explicit topological organization, which is not always possible to obtain on multi-label data. In a previous paper, we proposed a pooling architecture based on association to deal with this issue. The association was defined by means of Pearson's correlation. However, features must exhibit a certain degree of correlation with each other, which might not hold in all situations. In this paper, we propose a new method that replaces the correlation measure with another one that computes the entropy in the information granules that are generated from two features or labels. Numerical simulations have shown that our proposal is superior in those datasets with low correlation. This means that it induces a significant reduction in the number of parameters of neural networks, without affecting their accuracy.