View Item 
        •   Utrecht University Student Theses Repository Home
        • UU Theses Repository
        • Theses
        • View Item
        •   Utrecht University Student Theses Repository Home
        • UU Theses Repository
        • Theses
        • View Item
        JavaScript is disabled for your browser. Some features of this site may not work without it.

        Browse

        All of UU Student Theses RepositoryBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

        Extended Video2Report Database: Object Detection & Medical Action Recognition for Medical Consultations.

        Thumbnail
        View/Open
        Thesis Bas Kuijpers - Video2Report.pdf (64.95Mb)
        Publication date
        2022
        Author
        Kuijpers, Bas
        Metadata
        Show full item record
        Summary
        Many healthcare professionals are burdened with a large administrative load, and questions arise whether the current approach is sustainable, despite consensus that reporting leads to a better quality of healthcare. A portion of the administration can be automated by analyzing and documenting the events that take place during medical appointments. Computer vision is used to detect medical actions by analyzing the poses of both patient and care provider, as well as the detection of medical objects. OpenPose (pose estimation) and Faster R-CNN Resnet 101 (object detection) are used and the output of both these models are processed and analyzed with machine learning models, Random Forest and Long-Short Term Memory. The Video2Report dataset containing videos of medical actions has been extended with various new actions and other action classes have been complemented with additional recordings. An image dataset containing medical objects was collected (2117 images) and annotated (2956 annotations). Our experiments with object detection models did not result in improvements, possibly caused by a scarcity of images resembling the actual usage scenario. The best performing model proved to be Random Forest with a cross-validated test score of 75.43%. LSTM models reached an accuracy of 63.08%.
        URI
        https://studenttheses.uu.nl/handle/20.500.12932/481
        Collections
        • Theses
        Utrecht university logo