The Tense Debate: Cognitive Modelling English Past Tense Inflection with Encoder-Decoder Neural Models

Abstract

A longstanding debate surrounds modelling English past tense inflection. In this thesis, we investigated neural models, specifically Encoder-Decoders, as cognitive models of English past tense inflection. While recent studies showed that Encoder-Decoders achieve strong improvements over the initial connectionist model of Rumelhart and McClelland (1986), it has also been reported that they still failed to accurately capture human speaker inflections of novel forms. Our results reveal that data representativeness and model configuration choices influence model performance on real and nonce verbs. Importantly, we found improved correlations with human nonce verb inflections when the problem of overfitting on training data was mitigated, for instance, by using fewer training epochs. However, this also resulted in lower accuracies on real irregular verbs. A key finding is that this problem could be overcome by augmenting the training data with token frequency. This led to near-perfect performance on training verbs, including high accuracies on the irregular class, while obtaining almost equally strong correlations with human data. This highlights the relevance of token frequency, challenging previous assumptions. Additionally, we investigated a multi- task training setup, wherein the model also classifies verbs as regular or irregular. This task aligns with the dual-route view of Pinker and Prince (1988). However, this setup led to similar or slightly worse performance, leaving the cognitive validity of the discrete distinction between regular and irregular verbs open to further investigation. We emphasise the value of future research on using neural models to investigate cognitive processes such as morphological inflection.