Journal of Language Modelling is a free (for readers and authors alike) open-access peer-reviewed journal aiming to bridge the gap between theoretical linguistics and natural language processing. Although typical articles are concerned with linguistic generalisations – either with their application in natural language processing, or with their discovery in language corpora – possible topics range from linguistic analyses which are sufficiently precise to be implementable to mathematical models of aspects of language, and further to computational systems making non-trivial use of linguistic insights. We welcome articles on deeper linguistic aspects of large language models. However, articles focusing on the development of language models and their application to linguistic data without deeper linguistic insights are not in the scope of the Journal of Language Modelling. Papers appear as soon as they have been accepted – there are no delays typical of traditional paper journals. Accepted articles are then collected in half-yearly numbers and yearly volumes, with continuous page numbering, and are made available as hard copies via print on demand, at a nominal fee. On the other hand, Journal of Language Modelling has a fully traditional view of quality: all papers are carefully refereed by at least three reviewers (usually including at least one member of the Editorial Board) and they are only accepted if they adhere to the highest scientific, typographic and stylistic standards. Apart from full-length articles, the journal also accepts squibs and polemics with other papers. All journal content appears on the Creative Commons Attribution 4.0 International Licence. JLM is indexed by SCOPUS, ERIH PLUS, DBLP, DOAJ, EBSCO, Linguistics Abstracts Online, MLA Directory of Periodicals, and the Polish Ministry of Science and Education (list B). JLM is also a member of OASPA. In order to submit an article, you have to be registered. Further submission instructions for Authors are available here.
Institute of Computer Science of the Polish Academy of Sciences
All content is licensed under the Creative Commons Attribution 4.0 International Licence.
Large language models as proxies for theories of human linguistic cognition
We consider the possible role of current large language models (LLMs) in the study of human linguistic cognition. We focus on the use of such models as proxies for theories of cognition that are relatively linguistically-neutral in their representations and learning but differ from current LLMs in key ways. We illustrate this potential use of LLMs as proxies for theories of cognition in the context of two kinds of questions: (a) whether the target theory accounts for the acquisition of a given pattern from a given corpus; and (b) whether the target theory makes a given typologically-attested pattern easier to acquire than another, typologically-unattested pattern. For each of the two questions we show, building on recent literature, how current LLMs can potentially be of help, but we note that at present this help is quite limited.
Copyright (c) 2026 Imry Ziv, Nur Lan, Emmanuel Chemla, Roni Katzir https://creativecommons.org/licenses/by/4.0
LLM-based multi-agent poetry generation in non-cooperative environments
Despite substantial progress in large language models (LLMs) for automatic poetry generation, LLM-generated poetry often lacks diversity, and the training process differs greatly from human learning. Under the rationale that the poetry generation systems should learn more like humans and produce more diverse and novel outputs, we introduce a social learning-based framework that emphasizes non-cooperative interactions besides cooperative interactions to encourage diversity. Our experiments represent the first attempt at LLM-based multi-agent poetry generation in non-cooperative environments, employing both TRAINING-BASED agents (GPT-2) and PROMPT-BASED agents (GPT-3 and GPT-4). Evaluation on 96K generated poems demonstrates that our framework improves the performance of TRAINING-BASED agents, yielding a 3.0–3.7 percentage point (pp) increase in diversity and a 5.6–11.3 pp increase in novelty, as measured by distinct and novel n-grams. Poems generated by TRAINING-BASED agents also exhibit clear group divergence in lexicon, style, and semantics. PROMPT-BASED agents likewise benefit from non-cooperative environments. However, these agents show a decrease in lexical diversity over time and fail to demonstrate the intended group-based divergence within the social network. Our work argues for a paradigm shift in creative tasks such as automatic poetry generation to include social learning processes (via LLM-based agent modeling) similar to human interaction.
Copyright (c) 2026 Ran Zhang, Steffen Eger https://creativecommons.org/licenses/by/4.0
Ground promotion and complex particle constructions in Norwegian
This paper presents an HPSG analysis of Norwegian particle constructions with Ground promotion, as exemplified in (1), where av ‘off’ functions as a particle and the object bordet ‘the table’ is a Ground argument. (1) Jeg rydder AV bordet. I clear off table.DEF ‘I clear the table.’ It will be shown that the Ground promotion construction has similarities with both regular particle constructions and constructions with selected prepositions, and an analysis will be presented where Ground promotion is assumed to be a combination of these two constructions, viz. a complex particle construction. The analysis is implemented in a bidirectional typed feature structure grammar, and it will demonstrate that unification and type inheritance facilitates the merger of two constructions into one, while simultaneously allowing them to be kept apart.
Copyright (c) 2025 Petter Haugereid https://creativecommons.org/licenses/by/4.0
In search of semantic distance: metaphoric and non-metaphoric constructions in static and contextual representations
Phrases such as burning question, digital waste or invasion of technology are relatively ordinary expressions understood by any speaker of English. While diverse in structure and meaning, they demonstrate a semantic tension between the basic meanings of a metaphoric and a non-metaphoric constituent. Albeit frequent in discourse, they remain a challenge for automatic language processing systems, especially for smaller, less represented languages. In this work, we inspect a broad array of language models to embed noun phrases in Slovene and investigate the potential of word embeddings to identify metaphoric phrases via the semantic distance of its constituents as measured via cosine similarity. The study shows both static and contextual monolingual embeddings encode relevant semantic information while multilingual embeddings demonstrate no significant effect in this experimental setting. Moreover, the study unravels the most effective layers for basic meaning representation and highlights the influence of other, non-semantic factors on cosine similarity. By shedding light on these mechanisms, the study provides new insights for both metaphor processing and our understanding of the inner workings of language models.
Copyright (c) 2025 Mojca Brglez, Špela Vintar https://creativecommons.org/licenses/by/4.0
Computational approaches to morphological typology
Introduction to the Special Issue.
Copyright (c) 2024 Micha Elsner, Sacha Beniamine https://creativecommons.org/licenses/by/4.0
Tier-based strict locality and the typology of agreement
This paper presents a subregular analysis of syntactic agreement patterns modeled using command strings over Minimalist Grammar (MG) dependency trees (Graf and Shafiei 2019), incorporating a novel MG treatment of agreement. Phenomena of interest include relativized minimality and its exceptions, direction of feature transmission, and configurations involving chains of agreeing elements. Such patterns are shown to fall within the class of tier-based strictly 2-local (TSL-2) languages, which has previously been argued to subsume the majority of long-distance syntactic phenomena, as well as those in phonology and morphology (Graf 2022a). This characterization places a tight upper bound on the range of configurations that are predicted to occur while providing parameters for variation which closely match the observed typology.
Copyright (c) 2025 Kenneth Hanson https://creativecommons.org/licenses/by/4.0
Against successive cyclicity: A proof-theoretic account of extraction pathway marking
This paper proposes a novel analysis of extraction pathway marking in Type-Logical Grammar, taking advantage of proof-theoretic properties of logical proofs whose empirical application has so far been underexplored. The key idea is to allow certain linguistic expressions to be sensitive to the intermediate status of a syntactic proof. The relevant conditions can be stated concisely as constraints at the level of the proof term language, formally a special type of λ-calculus. The proposed analysis does not have any direct analog to either of the two familiar techniques for analyzing extraction pathway marking, namely, successive cyclic movement in derivational syntax and the SLASH feature percolation in HPSG. Moreover, the ‘meaning-centered’ perspective that naturally emerges from this new analysis is conceptually revealing: on this approach, extraction pathway marking essentially boils down to a strategy that certain languages employ to overtly flag the existence of a semantic variable inside a partially derived linguistic expression whose interpretation is dependent on a higher-order operator that is located in a larger structure.
Copyright (c) 2025 Yusuke Kubota, Robert Levine https://creativecommons.org/licenses/by/4.0
Corpus-based measures discriminate inflection and derivation cross-linguistically
In morphology, a distinction is commonly drawn between inflection and derivation. However, a precise definition of this distinction which reflects the way it manifests across languages remains elusive within linguistic theory, typically being based on subjective tests. In this study, we present 4 quantitative measures which use the statistics of a raw text corpus in a language to estimate to what extent a given morphological construction changes the form and distribution of lexemes. In particular, we measure both the average and the variance of this change across lexemes. Crucially, distributional information captures syntactic and semantic properties and can be operationalised by word embeddings. Based on a sample of 26 languages, we find that we can reconstruct 89±1% of the classification of constructions into inflection and derivation in UniMorph using our 4 measures, providing large-scale cross-linguistic evidence that the concepts of inflection and derivation are associated with measurable signatures in terms of form and distribution that behave consistently across a variety of languages. We also use our measures to identify in a quantitative way whether categories of inflection which have been considered noncanonical in the linguistic literature, such as inherent inflection or transpositions, appear so in terms of properties of their form and distribution. We find that while combining multiple measures reduces the amount of overlap between inflectional and derivational constructions, there are still many constructions near the model’s decision boundary between the two categories. This indicates a gradient, rather than categorical, distinction.
Copyright (c) 2024 Coleman Haley, Edoardo M. Ponti, Sharon Goldwater https://creativecommons.org/licenses/by/4.0
From speech signal to syntactic structure: A computational implementation
This paper presents a new computational implementation bridging several modules of grammar from phonetics to phonology to syntax. The system takes as input a speech signal annotated with syllables, interprets the phonetic data in phonological/prosodic terms, matches the data against a lexicon and makes the results available to a linguistically deep computational grammar. The system is showcased by means of syntactically ambiguous structures in German which can be disambiguated based on prosodic constituency information. A system evaluation with the German data showed good results for this new combination of automatic speech signal analysis and computational grammars, which takes a significant step towards a linguistically fine-grained computational analysis and hence towards real automatic speech understanding.
Copyright (c) 2025 Tina Bögel, Tianyi Zhao https://creativecommons.org/licenses/by/4.0
This article presents the structure of the ATLAs Alignment Module, a typological database designed to exhaustively capture languageinternal variation in argument marking (indexing and flagging). The flexible design of our database can be extended to cover further aspects of morphosyntactic alignment. We demonstrate with a small diversity sample how the database can be queried and the data aggregated at different levels of structure (e.g. for a language as a whole or for individual referential types in the form of alignment statements) for the purposes of cross-linguistic comparison. The database is made available in the Cross-Linguistic Data Formats (CLDF), and we provide code that generates an array of aggregations.
Copyright (c) 2024 David Inman, Alena Witzlack-Makarevich, Natalia Chousou-Polydouri, Melvin Steiger https://creativecommons.org/licenses/by/4.0
Constructional approaches in formal grammar
Copyright (c) 2023 Nurit Melnik, Manfred Sailer https://creativecommons.org/licenses/by/4.0
This study examines zero marking, i.e. the absence of an overt exponent, in adjectival, nominal, and verbal inflectional morphology across languages. The first part of the study provides an overview of the distribution of zero markers in inflection paradigms using the UniMorph dataset. The results show that there is a general preference against zero marking. The distribution of zero markers varies to a great extent across languages and lemmas, the only robust trend being that they are avoided in cells that express a high number of grammatical values. The second part of this study examines the association between marker frequencies and phonological length, using the Universal Dependencies treebanks. While token frequency is a good predictor for the length of overt markers, it does not account for the occurrence of zero markers. This is taken as evidence to support a differential non-development scenario of zero marking rather than a phonetic reduction scenario.
Copyright (c) 2024 Laura Becker https://creativecommons.org/licenses/by/4.0
QRGS – Question Responses Generation via crowdsourcing
QRGS stands for the Question Responses Generation System. It is an online game-like framework designed for gathering various types of question responses. A QRGS user is asked to read a simple story and impersonate its main character. As the story unfolds the user is confronted with four questions and (s)he is expected to answer these in the way the main character would. In this way, we obtain responses to questions of a desired type. The data gathered via QRGS is a useful supplement to the linguistic data already present in language corpora – especially for languages for which such resources are sparse. As such, it opens the possibility for better understanding of the use of questions in natural language dialogues and analysing the response space of such questions. In this paper, we present the main idea of QRGS and the results of five studies (in Polish and in English) that test the framework. Our discussion addresses issues concerning the efficiency and accuracy of the proposed approach. We also discuss the availability of the QRGS and its potential future improvements.
Copyright (c) 2024 Pawel Lupkowski, Jonathan Ginzbug, Ewelina Chmurska, Adrianna Płatosz, Aleksandra Kwiecień, Barbara Adamska, Magdalena Szkalej https://creativecommons.org/licenses/by/4.0
An analogical approach to the typology of inflectional complexity
This paper studies the inflectional complexity of nouns, verbs and adjectives in 137 datasets, across 71 languages. I follow Ackerman and Malouf (2013) in distinguishing between E(numerative) complexity and I(ntegrative) complexity. The first one encompasses aspects of inflection, like the number of principal parts, paradigm size, and number of exponents, while the second one captures the implicative relations between paradigm cells (how difficult it is to predict one cell of a paradigm knowing a different cell). I provide a formalism and computational implementation to estimate both I- and E-complexity expressed through Word and Paradigm morphology (Blevins 2006, 2016), which is flexible and powerful enough for typological research. The results show that, as suggested by Ackerman and Malouf (2013), I-complexity is relatively low across the languages in the sample, with only two clear exceptions (Navajo and Yaitepec-Chatino). The results also show that E-complexity can vary considerably crosslinguistically. Finally, I show there is a clear correlation between I- and E-complexity.
Copyright (c) 2024 Matías Guzmán Naranjo https://creativecommons.org/licenses/by/4.0
There is no consensus in the literature regarding the defining characteristics of copy raising (CR), or in other words what determines whether a CR-like expression is CR or not. As a result, existing analyses target different data sets. In this paper, I propose a different approach to these constructions, which takes a functional perspective. I propose to abandon the term copy raising, which is misleading in a number of ways. Instead, I distinguish between perceptual depiction reports and perceptual inference reports and show that the functions which they fulfill are not particular to CR-like constructions, but are in fact more general. Such an approach, I claim, resolves existing conundrums surrounding CR. The analysis is formalized in the framework of Head-driven Phrase Structure Grammar (HPSG) and is inspired by previous accounts of CR in related frameworks such as LFG and SBCG, as well as HPSG analyses. In the spirit of HPSG, the analysis employs type inheritance hierarchies to distinguish between what is shared by the two constructions and what is construction-specific in order to account for alternative realizations of a single lexeme and to ascribe constructional (or extra-lexical) meaning to linguistic elements.
Copyright (c) 2023 Nurit Melnik https://creativecommons.org/licenses/by/4.0
On German verb sense disambiguation: A three-part approach based on linking a sense inventory (GermaNet) to a corpus through annotation (TGVCorp) and using the corpus to train a VSD classifier (TTvSense)
We develop a three-part approach to Verb Sense Disambiguation (VSD) in German. After considering a set of lexical resources and corpora, we arrive at a statistically motivated selection of a subset of verbs and their senses from GermaNet. This sub-inventory is then used to disambiguate the occurrences of the corresponding verbs in a corpus resulting from the union of TüBa-D/Z, Salsa, and E-VALBU. The corpus annotated in this way is called TGVCorp. It is used in the third part of the paper for training a classifier for VSD and for its comparative evaluation with a state-of-the-art approach in this research area, namely EWISER. Our simple classifier outperforms the transformer-based approach on the same data in both accuracy and speed in German but not in English and we discuss possible reasons.
Copyright (c) 2024 Dominik Mattern, Wahed Hemati, Andy Lücking, Alexander Mehler https://creativecommons.org/licenses/by/4.0
Control, inner topicalisation, and focus fronting in Mandarin Chinese: modelling in parallel constraint-based grammatical architecture
This paper proposes a formal analysis of two displacement phenomena in Mandarin Chinese, namely inner topicalisation and focus fronting, capturing their correlational relationships with control and complementation. It examines a range of relevant data, including corpus examples, to derive empirical generalisations. Acceptability-judgment tasks, followed by mixed-effects statistical models, were conducted to provide additional evidence. This paper presents a constraint-based lexicalist proposal that is couched in the framework of Lexical-Functional Grammar (LFG). The lexicon plays an important role in regulating the behaviour of complementation verbs as they participate in the displacement phenomena. Unlike previous analyses that cast inner topicalisation and focus fronting as restructuring phenomena, this lexicalist proposal does not rely on hypothesised clause-size differences. It captures the empirical properties more accurately and accounts for a wider range of empirical patterns. Adopting the formally explicit framework of LFG, this proposal uses constraints that have mathematical precision. The constraints are computationally implemented using the grammar engineering tool Xerox Linguistic Environment, safeguarding their precision.
Copyright (c) 2024 Chit-Fung Lam https://creativecommons.org/licenses/by/4.0
The expected sum of edge lengths in planar linearizations of trees
Dependency trees have proven to be a very successful model to represent the syntactic structure of sentences of human languages. In these structures, vertices are words and edges connect syntacticallydependent words. The tendency of these dependencies to be short has been demonstrated using random baselines for the sum of the lengths of the edges or their variants. A ubiquitous baseline is the expected sum in projective orderings (wherein edges do not cross and the root word of the sentence is not covered by any edge), that can be computed in time O(n). Here we focus on a weaker formal constraint, namely planarity. In the theoretical domain, we present a characterization of planarity that, given a sentence, yields either the number of planar permutations or an efficient algorithm to generate uniformly random planar permutations of the words. We also show the relationship between the expected sum in planar arrangements and the expected sum in projective arrangements. In the domain of applications, we derive a O(n)-time algorithm to calculate the expected value of the sum of edge lengths. We also apply this research to a parallel corpus and find that the gap between actual dependency distance and the random baseline reduces as the strength of the formal constraint on dependency structures increases, suggesting that formal constraints absorb part of the dependency distance minimization effect. Our research paves the way for replicating past research on dependency distance minimization using random planar linearizations as random baseline.
syntactic dependency distance minimization
Copyright (c) 2024 Lluís Alemany-Puig, Ramon Ferrer-i-Cancho https://creativecommons.org/licenses/by/4.0
Detecting inflectional patterns for Croatian verb stems using class activation mappings
All verbal forms in the Croatian language can be derived from two basic forms: the infinitive and the present stems. In this paper, we present a neural computation model that takes a verb in an infinitive form and finds a mapping to a present form. The same model can be applied vice-versa, i.e. map a verb from its present form to its infinitive form. Knowing the present form of a given verb, one can deduce its inflections using grammatical rules. We experiment with our model on the Croatian language, which belongs to the Slavic group of languages. The model learns a classifier through these two classification tasks and uses class activation mapping to find characters in verbs contributing to classification. The model detects patterns that follow established grammatical rules for deriving the present stem form from the infinitive stem form and vice-versa. If mappings can be found between such slots, the rest of the slots can be deduced using a rule-based system.
Croatian infinitive and present verb stems
Copyright (c) 2024 Domagoj Ševerdija, Rebeka Čorić, Lucian Šošić, Marko Orešković https://creativecommons.org/licenses/by/4.0
Copyright (c) 2022 Elżbieta Hajnicz https://creativecommons.org/licenses/by/4.0
Lexical Functional Grammar as a Construction Grammar
Lexical Functional Grammar (LFG) is a lexicalist, constraint-based grammatical theory that shares a lot of the basic assumptions of Construction Grammar (CxG), such as a commitment to surface-oriented descriptions (no transformations), and the simultaneous representation of form, meaning, and other grammatical information (no derivations). Nevertheless, LFG is not standardly viewed as a kind of CxG, in particular since its adherence to the principle of Lexical Integrity means that it insists on a strict morphology-syntax distinction where CxG canonically rejects such a divide. However, such a distinction is in fact entirely compatible with CxG assumptions; the actual problem with viewing LFG as a CxG is the difficulty it has in describing the more substantive end of the schematic-substantive spectrum of constructions. I suggest that by replacing the limited context-free grammar base of LFG responsible for this shortcoming with a more expressive formalism (in this case a description-based tree-adjoining grammar), we can obtain a fully constructional LFG, suitable as a formal framework for CxG.
Copyright (c) 2023 Jamie Y. Findlay https://creativecommons.org/licenses/by/4.0
The Dutch Anaphoric Possessive Construction
The Dutch anaphoric possessive construction (APC), as exemplified by Tom zijn fiets ‘Tom his bike’, shows a peculiar mix of regularity and idiosyncracy. The article provides a theory-neutral description of its properties and quantitative information about its use in two treebanks, one of spoken Dutch (CGN) and one of written Dutch (Lassy Small). It argues that the APC has a right branching structure and models it in the framework of Constructional Head-driven Phrase Structure Grammar. The latter’s organization of constructions in terms of a finegrained hierarchy of phrase types is shown to provide the means to capture both what the APC has in common with other possessive constructions and what is idiosyncratic of it.
Constructional Head-driven Phrase Structure Grammar
Copyright (c) 2023 Frank Van Eynde https://creativecommons.org/licenses/by/4.0
This paper proposes a formal model of regular languages enriched with unbounded copying. We augment finite-state machinery with the ability to recognize copied strings by adding an unbounded memory buffer with a restricted form of first-in-first-out storage. The newly introduced computational device, finite-state buffered machines (FS-BMs), characterizes the class of regular languages and languages de-rived from them through a primitive copying operation. We name this language class regular copying languages (RCLs). We prove a pumping lemma and examine the closure properties of this language class. As suggested by previous literature (Gazdar and Pullum 1985, p. 278), regular copying languages should approach the correct characteriza-tion of natural language word sets.
Copyright (c) 2023 Yang Wang, Tim Hunter https://creativecommons.org/licenses/by/4.0
An algebraic approach to translating Japanese
We use Lambek’s pregroups and the framework of compositional distributional models of language (“DisCoCat”) to study translations from Japanese to English as pairs of functors. Adding decorations to pregroups, we show how to handle word order changes between languages.
Copyright (c) 2023 Valentin Boboc https://creativecommons.org/licenses/by/4.0
We thought the eyes of coreference were shut to multiword expressions and they mostly are
Multiword expressions are combinations of words that exhibit pecu-liar semantic properties, such as different degrees of non-compositio-nality, decomposability, transparency and figuration. Long-standing linguistic debates suggest that such semantic idiosyncrasy can con-dition the morpho-syntactic configurations in which a given multi-word expression can occur. Here, we extend this argumentation to a particular semantic and pragmatic phenomenon: nominal coreference. We hypothesise that the internal components of a multiword expres-sion are unlikely to occur in coreference chains. While previous work has identified the rareness of coreference-related phenomena in pres-ence of multiword expressions, this observation has never been quan-tified, to the best of our knowledge. We bridge this gap by performing an automated corpus-based study of the intersections between verbal multiword expressions and nominal coreference in French. The results largely corroborate our hypothesis but also display various tendencies depending on the type of multiword expression and the corpus genre. The analysis of the corpus examples highlights interesting properties of coreference, notably in speech.
Copyright (c) 2023 Agata Savary, Jianying Liu, Anaëlle Pierredon, Jean-Yves Antoine, Loïc Grobol https://creativecommons.org/licenses/by/4.0
Neural heuristics for scaling constructional language processing
Constructionist approaches to language make use of form-meaning pairings, called constructions, to capture all linguistic knowledge that is necessary for comprehending and producing natural language expressions. Language processing consists then in combining the constructions of a grammar in such a way that they solve a given language comprehension or production problem. Finding such an adequate sequence of constructions constitutes a search problem that is combinatorial in nature and becomes intractable as grammars increase in size. In this paper, we introduce a neural methodology for learning heuristics that substantially optimise the search processes involved in constructional language processing. We validate the methodology in a case study for the CLEVR benchmark dataset. We show that our novel methodology outperforms state-of-the-art techniques in terms of size of the search space and time of computation, most markedly in the production direction. The results reported on in this paper have the potential to overcome the major efficiency obstacle that hinders current efforts in learning large-scale construction grammars, thereby contributing to the development of scalable constructional language processing systems.
Copyright (c) 2022 Paul Van Eecke, Jens Nevens, Katrien Beuls https://creativecommons.org/licenses/by/4.0
Evaluating syntactic proposals using minimalist grammars and minimum description length
Many patterns found in natural language syntax have multiple pos-sible explanations or structural descriptions. Even within the cur-rently dominant Minimalist framework (Chomsky 1995, 2000), it is not uncommon to encounter multiple types of analyses for the same phenomenon proposed in the literature. A natural question, then, is whether one could evaluate and compare syntactic proposals from a quantitative point of view. In this paper, we show how an evaluation measure inspired by the minimum description length principle (Rissa-nen 1978) can be used to compare accounts of syntactic phenomena implemented as minimalist grammars (Stabler 1997), and how argu-ments for and against this kind of analysis translate into quantitative differences.
Copyright (c) 2023 Marina Ermolaeva https://creativecommons.org/licenses/by/4.0
Idiosyncratic frequency as a measure of derivation vs. inflection
There is ongoing discussion about how to conceptualize the nature of the distinction between inflection and derivation. A common approach relies on qualitative differences in the semantic relationship between inflectionally versus derivationally related words: inflection yields ways to discuss the same concept in different syntactic contexts, while derivation gives rise to words for related concepts. This differential can be expected to manifest in the predictability of word frequency between words that are related derivationally or inflectionally: predicting the token frequency of a word based on information about its base form or about related words should be easier when the two words are in an inflectional relationship, rather than a derivational one. We compare prediction error magnitude for statistical models of token frequency based on distributional and frequency information of inflectionally or derivationally related words in French. The results conform to expectations: it is easier to predict the frequency of a word from properties of an inflectionally related word than from those of a derivationally related word. Prediction error provides a quantitative, continuous method to explore differences between individual processes and differences yielded by employing different predicting information, which in turn can be used to draw conclusions about the nature and manifestation of the inflection–derivation distinction.
Copyright (c) 2022 Maria Copot, Timothee Mickus, Olivier Bonami https://creativecommons.org/licenses/by/4.0
Introduction to the special section on the interaction between formal and computational linguistics
Copyright (c) 2022 Timothée Bernard, Grégoire Winterstein https://creativecommons.org/licenses/by/4.0
Simplicity and learning to distinguish arguments from modifiers
We present a learnability analysis of the argument-modifier distinction, asking whether there is information in the distribution of English constituents that could allow learners to identify which constituents are arguments and which are modifiers. We first develop a general description of some of the ways in which arguments and modifiers differ in distribution. We then identify two models from the literature that can capture these differences, which we call the argument-only model and the argument-modifier model. We employ these models using a common learning framework based on two simplicity biases which tradeoff against one another. The first bias favors a small lexicon with highly reusable lexical items, and the second, opposing, bias favors simple derivations of individual forms – those using small numbers of lexical items. Our first empirical study shows that the argument-modifier model is able to recover the argument-modifier status of many individual constituents when evaluated against a gold standard. This provides evidence in favor of our general account of the distributional differences between arguments and modifiers. It also suggests a kind of lower bound on the amount of information that a suitably equipped learner could use to identify which phrases are arguments or modifiers. We then present a series of analyses investigating how and why the argument-modifier model is able to recover the argument-modifier status of some constituents. In particular, we show that the argumentmodifier model is able to provide a simpler description of the input corpus than the argument-only model, both in terms of lexicon size, and in terms of the complexity of individual derivations. Intuitively, the argument-modifier model is able to do this because it is able to ignore spurious modifier structure when learning the lexicon. These analyses further support our general account of the differences between arguments and modifiers, as well as our simplicity-based approach to learning.
Copyright (c) 2022 Journal of Language Modelling https://creativecommons.org/licenses/by/4.0