View Item 
        •   Utrecht University Student Theses Repository Home
        • UU Theses Repository
        • Theses
        • View Item
        •   Utrecht University Student Theses Repository Home
        • UU Theses Repository
        • Theses
        • View Item
        JavaScript is disabled for your browser. Some features of this site may not work without it.

        Browse

        All of UU Student Theses RepositoryBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

        Exploring the use of large language models to improve one-to-one communication training scenarios

        Thumbnail
        View/Open
        Thesis_Jens_Hartkamp_MAI.pdf (3.838Mb)
        Publication date
        2024
        Author
        Hartkamp, Jens
        Metadata
        Show full item record
        Summary
        This paper explores the automation of communication training scenario writing by using AI generated statements. Two different research questions were explored, the first attempts to improve the quality of the training scenarios by replacing so-called non-functioning “Distractor” options. These options are pitfalls to make a user think more about the best practice response in a certain scenario. Distractors were first evaluated where non-functioning distractors were identified and replaced by AI generated statements. The original and generated distractors were implemented in communication training scenarios which were used in a bachelor course. The differ ences in how often these options were chosen were analyzed using a Mann Whitney Utest. There were no significant differences found, although the outcome was very close to the 0.05 significance cutoff with a p-value of 0.057. For the second research question we attempted to generate statements based on the desired parameter set tings. The output was evaluated by experts who concluded the results are promising, but not ready for automation without human evaluation. The expert grades of the AI generated options were significantly worse than the rating of the human written dis tractors meaning more work has to be done before fully automated parameter based answering option generated is viable
        URI
        https://studenttheses.uu.nl/handle/20.500.12932/48081
        Collections
        • Theses
        Utrecht university logo