The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media

Abstract

AI systems for depression detection on social media have been continuously improving their performance, showing that meaningful patterns can be found in the data. While many machine learning models used to detect depression are opaque, models predicting depression symptoms can often provide more explainability. Previous research has shown that some depression detection datasets with data collected from social media exhibit gender biases, but no studies have investigated gender bias for a dataset annotated with depression symptoms yet. Therefore, this thesis aims to investigate the extent to which gender bias is present in the BDI-Sen dataset, evaluate classifier performance across different genders, and whether existing gender bias can be mitigated. A statistical analysis reveals that the dataset shows some gender bias, reflecting gender differences in depression symptoms. Analysis of mentalBERT classifiers trained on the dataset identifies several biases across the different symptoms, particularly in terms of predictive equality, with the majority of the bias favoring males. To address these biases, data augmentation strategies such as synonym replacement, back-translation, and oversampling were applied. These methods helped reduce the bias but did not remove it completely. Future research could implement different bias mitigation techniques to reduce the bias, and investigate gender bias in depression symptom detection on larger datasets or datasets annotated for symptoms from a different questionnaire, such as the PHQ-9.