| dc.description.abstract | This thesis, developed in collaboration with Nick Ramsey’s BCI lab at UMC Utrecht, investigated the feasibility of decoding semantic representations of speech from Stereotactic EEG data, recorded from epilepsy patients during a natural speech production task. These representations were either encoded as static or dynamic word embeddings, generated through the use of the Word2Vec and BERT models respectively, or as automatically generated semantic categories, derived from either these embedding, or from WordNet, a large lexical database. Furthermore, auto encoders were utilized in order to reduce the dimensionality of the word embeddings, whilst keeping important semantic information intact, which in turn reduced the complexity of decoding such embeddings. Additionally, auto encoders were applied to the sEEG data, for the purposes of de-noising, automatic feature selection, and dimensionality reduction. A preliminary motor decoding task, in the form of syllable classification was also included, to determine whether the sEEG data held any decoding potential in the first pace, as this task demonstrated a more salient relationship with the sEEG data. Lastly, data from 2 individual subjects was combined, after having been compressed by the auto encoders, in order to investigate whether limitations regarding the sparse distribution of sEEG electrode placements could be mitigated, through the incorporation of more data. The results of this study have demonstrated that auto encoders could successfully compress both the brain data and the semantic vectors, whilst keeping important information intact. Furthermore, the results indicated that while motor information could be decoded to a certain extent, semantic information could not be decoded from the available sEEG data. This was likely caused by a combination of the sEEG data’s inability to sufficiently encapsulate semantic processing, given its sparse and distributed nature, as well as a lack of clear separability between the different semantic representations. Despite these findings with respect to semantic decoding, the efficacy of the auto encoders could hold great potential for future semantic speech decoding studies. Future work should focus on generating more separable semantic representations, incorporating data from more subjects, and on designing tools that can properly quantify the relationship between different electrode activations and semantic processing. | |
| dc.subject | This thesis, developed in collaboration with Nick Ramsey’s BCI lab at UMC Utrecht, investigated the feasibility of decoding semantic representations of speech from Stereotactic EEG data, recorded from epilepsy patients during a natural speech production task. These representations were either encoded as static or dynamic word embeddings, generated through the use of the Word2Vec and BERT models respectively, or as automatically generated semantic categories. | |
