A comparison between Bayesian penalized regression priors: lasso and regularized horseshoe

Abstract

A comparison is performed between Bayesian penalized regression priors: the lasso and regularized horseshoe using the statistical programming language R. This study aims to provide researchers with insights into the use of these priors to deal with high-dimensional data. Therefore, the shrinkage behavior of the Bayesian lasso and regularized horseshoe models, using different hyperparameter settings, were compared. Furthermore, variable selection was executed for the models. Lastly, the predictive performances were evaluated based on their Root Mean Square Error (RMSE). Results showed that researchers have to take several factors into consideration. First consideration concerns which prior is best suited on their data. The Bayesian lasso showed more variation in shrinkage behavior and is easy to implement, while regularized horseshoe prior is more robust to their specific hyperparameter settings and is complex to implement. Second, researchers should consider a variable selection method. This paper shows that an RMSE plot is a suitable tool for variable selection. In conclusion, there were no significant differences in predictive performances found between the Bayesian lasso and regularized horseshoe.