Agregação de variáveis nas redes neurais artificiais baseada em medidas de informação mútua

Abstract

Artificial neural networks constitute a class of models that lies at the core of mo dern machine learning. A notable attribute of these models is, in many applications, their large number of parameters, which gives rise to the need for either implicit or explicit regularization in order to achieve strong predictive performance. One way to incorporate implicit regularization is by constraining which variables are allowed to interact within artificial neurons, as exemplified by the convolutional and the pooling layers of convolutio nal neural networks (CNNs), where each variable interacts only with its spatial neighbors. Within this context, the central hypothesis motivating this dissertation is that, in certain scenarios, it is beneficial to impose such constraints so that computations performed over subsets of the input variables involve those exhibiting high statistical dependence. In this regard, one of the main contributions of this dissertation is a proof of concept demonstra ting that conventionally defined CNNs provide a practical example of the application of this principle. Furthermore, experimental evidence is presented showing that extending this principle to tabular data through the use of locally connected layers is a promising approach for representation learning and autoencoding. Throughout the dissertation, it is frequently necessary to quantify the statistical dependence between pairs of variables using mutual information measures. Accordingly, this document also presents an in-depth study of the mutual information estimation problem, which remains an active research hotspot. As a result, the outcomes of this study include the proposal of a novel estima tion method for discrete variables and the adoption of a mutual information estimator for continuous sources, which is employed throughout the dissertation.