Show simple item record

dc.rights.licenseCC-BY-NC-ND
dc.contributor.advisorDu, Y.
dc.contributor.authorAlbers, J.M.
dc.date.accessioned2021-09-08T18:00:12Z
dc.date.available2021-09-08T18:00:12Z
dc.date.issued2021
dc.identifier.urihttps://studenttheses.uu.nl/handle/20.500.12932/1176
dc.description.abstractIn this research I examined the differences in different contextualized word embedding models for predicting the graded effect of context in word similarity. Each model was tested on a task in which the degree and direction of change of word similarity of two word pairs had to be predicted. This was later compared to human-annotated scores. I found that the BERT architecture works exceptionally well compared to other Transformer based models on the English language. I also found that the multilingual BERT models offers obtains the best score on the low resource languages Finnish, Hungarian and Slovenian. Furthermore I found that stacked embeddings, in which multiple models get combined, offer room for improvement for already performing models. At last I recommend further research in which more models can get compared and the further examination of stacked embeddings.
dc.description.sponsorshipUtrecht University
dc.format.extent566003
dc.format.mimetypeapplication/pdf
dc.language.isoen
dc.titleComparing Contextualised Embeddings for Predicting the (Graded) Effect of Context in Word Similarity
dc.type.contentBachelor Thesis
dc.rights.accessrightsOpen Access
dc.subject.keywordsNLP, contextualized word embedding, BERT, GPT-2, Flair, CoSimLex, word similarity, context
dc.subject.courseuuKunstmatige Intelligentie


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record