Natural Language Processing for Early Diagnosis of Childhood Epilepsy

Abstract

Background: Accurate and timely diagnosis is crucial in epilepsy treatment. Diagnostic delay in epilepsy results in unnecessary risk exposure to psychosocial distress, morbidity, or mortality. Language is an indispensable source of information for diagnosing epilepsy. Natural language processing, a branch of artificial intelligence, analyses language to extract information and identify patterns. This study assessed the diagnostic value of natural language processing to facilitate the early diagnosis of childhood epilepsy. Methods: A dataset of 1561 letters from first consultations was available from the University Medical Center Utrecht and Martini Hospital Groningen. Natural language processing was applied to analyse textual data and classify the letters as either 'epilepsy' or 'no epilepsy'. The Naïve Bayes model was employed for text classification. Data was divided into training and test sets to evaluate performance and generalisability. Training sets identified predictive features, consisting of keywords indicative of 'epilepsy' or 'no epilepsy'. The model's output was compared to the clinician's final diagnosis (gold standard). Results: Model accuracy ranges from 0.66 to 0.68. Balanced accuracy varies from 0.67 to 0.72 for ‘epilepsy’ and 0.68 to 0.73 for ‘no epilepsy’. F1 score varies from 0.50 to 0.57 for 'epilepsy' and 0.76 to 0.80 for 'no epilepsy'. AUROC varies from 0.74 to 0.78 for ‘epilepsy’ and 0.73 to 0.77 for ‘no epilepsy’. AUPRC varies from 0.52 to 0.63 for ‘epilepsy’ and 0.79 to 0.81 for ‘no epilepsy’. Conclusion: All models demonstrated moderate to good performance, with better performance in diagnosing ‘no epilepsy’. Improvements are required to enhance accuracy and generalisability.