Assessment of Unsupervised Models: In pursuit of an evaluation measure

Abstract

One of the categories of machine learning is unsupervised learning. The assessment of models of this category is particularly challenging since the user lacks evidence regarding the original data set and the correct conclusions that must be made from the data set. This thesis is a pursuit for finding proper measures that can aid us in gaining insight and understanding the models. We introduce measures to evaluate the model's performance generally and with respect to specific subsets. Hence, we find out what parts of the data were learned well by the model and what parts were overlooked. We also introduce a procedure for producing synthetic data with controlled levels of randomness to examine the models with varying amounts of noise in the data. Finally, we apply our methods to various data sets and numerous models learned from them and conclude eminence of some of these models. We also point out the reasons for the poor performance of the models on some of the synthetic data sets.