Individualized Smile Detection Using Transfer Learning

Abstract

In current machine learning approaches to visual smile detection the training data generally consists of a variety of different faces displaying a smile or neutral expression, and the target data introduces a new face to classify. Although this approach has shown promising results towards correctly classifying the newly introduced target face, we believe that performance can be improved by acknowledging that there are differences in smile-styles and facial differences throughout the training process. In this research we aim to do so by applying transfer learning. This entails first training a classifier on a data set showing a wide variety of smiling and non-smiling faces, as is the general approach. Afterwards we apply transfer learning, by means of an Adaptive SVM, where a small number of labeled instances from the target face is provided as to make the classifier more specific to the target face. To make transfer learning effective we use low-level geometric features which we expect to capture the difference between smiles and no-smile in a variety of different faces and smile styles. We evaluate our approach by comparing performance against alternative strategies. We find that using a traditional classifier trained on an aggregated data set containing the general and target data outperforms our baseline and our suggested transfer learning approach for each of the test videos. A big problem for the transfer learning approach seems to be the quality of the labeled data of the target face used during training. The aggregated approach seems less effected by it, making it a preferred approach for real-life applications where labeled target data will be sparse and difficult to select. However, its performance improvements does not outweigh its efforts in current experiments, and further research should focus on the choice of target data to use during the training process.