Improving online rating systems using machine learning

Abstract

The online marketplace relies heavily on reviews to establish trust between the customer and the seller. These reviews are often both in written and star-rating form. In this paper, I use reviews scraped from the site werkspot.nl, a Dutch online market platform for small construction work, to analyse the sentiment reflected in the written review, and to predict the rated amount of stars. This study aims to investigate if the rating system can be improved by recommending star ratings based on the written review. I train a deep learning-based Bi-directional Short and Long Term Memory (LSTM) model to predict both the multi-class (5-star rating) and the binary sentiment (positive or negative) rating. The results suggest that this method has the potential to be of use for this application, but there are some issues to be resolved before this is possible. For instance, both models suffer from overfitting on the training data, resulting in lower model performance scores when testing the model on untrained data. One of the likely reasons for this is the extreme disparity between high and low ratings in the data. I elaborate more on this further in this paper. Overall, the main finding is that while the training results for both binary and multi-class are good, improvement is necessary to implement the model in real-world applications.