Speaker
Description
The linguistic analysis and investigation of transformer language models has become an established line of research in recent work on LLMs. Typically, the aim of these analyses is to determine whether existing pre-trained LLMs learn human-like linguistic knowledge and generalise linguistic rules in a human-like manner. While this has been extremely fruitful in terms of revealing the linguistic abilities of existing LLMs, these studies are often inconclusive when it comes to handling apparent limitations and gaps in the linguistic knowledge of LLMs. In this talk, I will discuss how small language models, trained on controlled and plausible amounts of data, might offer new perspectives for linguistically oriented research on language models. I will present experiments on analysing the linguistic capabilities of small language models and show how they can be useful for deeper linguistic insight and hypothesis-driven modelling.
