Deep clustering self-organizing maps with relevance learning

Abstract

In recent years, with the advancement of the internet, there has been an increase in data available. Alongside various data sources like sensors began to generate data of all kinds. Extracting useful information from data is a challenging problem in machine learning. The emerging focus on machine learning research has been the field of deep learning, which aims to learn multiple layers of abstraction that can be used to interpret data and perform complex tasks. In fact, the successful results of deep learning rely on the supervised field, which needs a large amount of labeled data. Unsupervised deep learning models have been proposed to deal with data without the requirement of annotations, which incorporate the data itself as a clue to guide the learning process. In that way, this thesis presents Deep Clustering Self-Organizing Map with Relevance Learning (DCSOM-RL), an unsupervised learning model capable of working with complex data, such as images and sound, while learning representations more suitable for clustering in latent spaces. The proposed approach combines deep learning architectures such as Autoencoders with a SOM layer with time-varying topology. The results show that the prototypes identified by DCSOM-RL represent frequent variations observed in the input data. For instance, the different ways to represent the input data. It can also bring insights about similarities between different categories or feature representations and which dimensions of the latent space capture important information. The neighborhood learned by the DCSOM-RL leads to smoother regions of transition between categories in the latent space. Although it does not present state of the art results in terms of evaluation metrics, the qualitative analysis shows that the model presents unique properties not available in other methods of Deep Clustering methods.