Giulimondi, 7165838. Talking machines and linguistic cognition: a critical review of the use of large language models in linguistic theorizing
Summary
Increasingly, linguistic studies are employing LLMs to explain the underlying
mechanisms of human linguistic cognition by applying experimental methods
to LLMS that were previously adopted to test human participants (Hueb-
ner et al., 2021; Beguš et al., 2023; Piantadosi, 2023; Goldstein et al., 2020).
Computational cognitive scientists have argued that the assumptions under-
lying these research choices are incorrect(Guest and Martin, 2023; van Rooij
et al., 2023) and a growing body of linguists is taking a critical stance to-
wards LLMs (Martínez et al., 2023; Kodner et al., 2023; Katzir, 2023; Bender
and Koller, 2020; Bender et al., 2021). However, meta-theoretical linguistic
research is still scarce, and, so far, no systematic analysis of language stud-
ies using LLMs as experimental tools was conducted. This thesis aims to
understand how the use of LLMs in research is affecting theory building in
linguistics. More specifically, this analysis will focus on two research ques-
tions: 1) What is the theoretical relation of LLMs to human cognition, when
they are used for linguistic research? 2) How valid is the use of LLMs in
linguistic theory?
The thesis will review ten linguistic articles and argue that they share
the assumption that LLMs represent an artificial replication of human lin-
guistic cognition. Moreover, drawing from Guest and Martin (2023); Guest
(2024) and Sullivan (2022) theoretical framework, it will be discussed how
LLMs used to generate human-like linguistic behavior represent a theoretical
misuse.
It will be shown how this misuse of LLMs is motivated by an industry-
driven research mindset (Ahmed et al., 2023), which may be at the root of
the theoretical misconceptions hypothesized in the first inquiry of this study.
This analysis is relevant for the understanding of language technology by
language professionals and the possible systemic misinterpretations at play
in research in a human-machine era.
Collections
Related items
Showing items related by title, author, creator and subject.
-
Can linguistic features unmask fraudulent research? A study that builds an NLP classifier to distinguish retracted papers from non-retracted papers based on text and linguistic features.
Schmidt, Eveline (2022)Researchers experience a lot pressure to get published and cited, as their careers often depend on it. This pressure can result in various forms of misconduct. Fraud in academic research is an important problem that should ... -
Differentiating psychotic patients by linguistic features: clustering patients with psychotic disorder to explore the relationship between diagnostic and linguistic properties
Barkema, P.W. (2019)Psychotic disorder causes high social costs due to the impact it has on patients and the high prevalence, especially among adolescents. No reliable biological indicator exists for the di- agnosis of psychotic disorder, ...