Algorithmic fairness in ECG analysis: exploring the impact of sex imbalance on neural network performance

Abstract

In cardiovascular research, deep convolutional neural networks (CNNs) have demonstrated efficacy in predicting disease outcomes and supporting triage assessments based on ECG data. CNN networks have been shown to be able to infer demographic characteristics such as sex from ECG data, raising concerns about potential algorithmic bias if sex-specific features are used as shortcuts to predict outcomes, possibly exacerbated by existing biases present in research data. This study investigates the algorithmic fairness of CNN models by examining the impact of sex-stratified outcome prevalence in sex-imbalanced datasets on model performance. The study finds that a balanced dataset does not guarantee exactly equal performance between sexes. The model did achieve a fairly consistent AUC performance across sexes with severe sex-imbalance training data and outcome prevalence in datasets. Calibration error metrics revealed larger performance discrepancies when the prevalence of predicted outcomes was imbalanced by sex. These results highlight the need to address severe sex imbalances in outcome prevalence to mitigate potential biases. Additionally, we emphasise the importance of rigorous reporting on both sex representation in the data and comprehensive sex-stratified performance metrics for the fairness of ECG-trained CNN algorithms.