ABSTRACT
This chapter addresses a series of interrelated questions about the origin of syntactic structures: How do language learners generalise from the linguistic stimulus with which they are presented? To what extent does linguistic cognition recruit domain-general (i.e., not language-specific) processes and representations? And to what extent are rules and generalisations about linguistic structure separate from rules and generalisations about linguistic meaning? We address these questions by asking what syntactic generalisations can be acquired by a domain-general learner from string input alone. The learning algorithm we deploy is a neural-network based Language Model (GPT-2; Radford et al., 2019), which has been trained to provide probability distributions over strings of text. We assess its linguistic capabilities by treating it like a human subject in a psycholinguistics experiment, and inspect behaviour in controlled, factorised tests that are designed to reveal the learning outcomes for one particular syntactic generalisation. The tests presented in this chapter focus on a variety of syntactic phenomena in two broad categories: rules about the structure of the sentence and rules about the relationships between smaller lexical units, including scope and binding. Results indicate that our target model has learned many subtle syntactic generalisations, yet it still falls short of humanlike grammatical competence in some areas, notably for cases of parasitic gaps (e.g., “I know what you burned __ after reading __ yesterday”). We discuss the implications of these results under three interpretive frameworks, which view the model as (a) a counter-argument against claims of linguistic innateness, (b) a positive example of syntactic emergentism, and (c) a fully articulated model of grammatical competence.
