Domain-specific text classification: determining medical outcomes using free text in electronic patient records

Abstract

Background In modern healthcare, outcome measures are considered of high value. They are used to make discharge decisions, to reflect on a specific treatment and to decide whether changes in the treatment plan need to be made. In psychiatry, these outcome measures are often only obtained at the start and end of a hospital admission, which does not cover the complete picture. However, it is also desirable to get a picture from the time between admission an discharge. Method A support vector machine model was created that classifies daily written nurse reports in either being written at the start of admission or in the last days before discharge. In order to attain a measurement of patient well-being we make the assumption that patients suffer from serious mental problems at the beginning of admission and that their symptoms are reduced or at least have stabilized when they are discharged, thereby linking time of the report to mental state of well-being. For unseen nurse reports written in the days between admission and discharge the model predicts whether the report has been written in the last days before discharge with a certain probability. From these probability rates a line chart is created that follows the course of an admission. Higher probability rates are possibly able to show patient improvement, while lower probability rates may indicate patient impairment. Model results were compared with findings in the literature and with reflections made by human annotators who rated patient improvement during admission. Results The model was able to predict whether a report was written at the start or at the end of admission with a 92\% accuracy. The mean line chart shows a decelerating curve that follows the findings in the literature. Moments where the model found exceptionally high or low probability rates were indicated as apparent patient improvement or impairment four out of eight times by annotators. Overall, annotators observed more of these moments than the model did. Words that the model found indicating either improvement (high probability rates) or impairment (low probability rates) were indicated by annotators for 14.8\% of the words. Discussion The results suggest that the method we tested to find patterns of patient improvement and impairment has potential, although the model's explanations did not comply with human explanations. Since this project was a first attempt at getting insight in outcome measures through nurse reports, improvements should be made to the model and other methods should be explored.