Speakers
Description
With easy access to APIs that query large language models (LLM), a good number of scientific disciplines explore their use for tasks for which they have previously used human resources or traditional technologies. LLM have also been explored in lexicography to support experts in constructing and maintaining dictionaries. There are some members of the field who even proclaim the death of lexicography because LLMs will soon be able to generate lexical entries and even entire dictionaries [1].
We report our work on testing this claim using a prominent representative of LLMs, ChatGPT, in the context of GermaNet, the largest lexical-semantic wordnet for German [2]. The latest version of GermaNet (18.0) features 215,000 lexical units (nouns, verbs, adjectives) that are attached to 167,163 synsets. It has 181,530 conceptual relations, 12,602 lexical relations (synonymy excluded), and a representation of 121,655 split compounds. GermaNet, hence, covers a large spectrum of the German language. But while all verbs in GermaNet come with at least one example sentence to illustrate the verb’s usage (for a given reading), GermaNet has only few example sentences for nouns and adjectives. It is this gap that we would like to close with the help of LLM.
Since LLM are built using enormous amounts of corpus data, we expect Chat-GPT to perform very well in this task. But while ChatGPT’s performance on generating example sentences for monosemous words is very good, it shows that the language competence of our human experts easily outperforms the language competence of ChatGPT when it comes to the generation of example sentences for polysemous words. In the poster, we show examples where ChatGPT uses incorrect or atypical word collocations, i.e., in verb-object and adjective-noun pairs. Often, ChatGPT displays an incorrect or insufficient understanding at the word (e.g., Erleben vs. Erlebnis, Wirken vs. Wirkung; Ensetzen vs. Entsetzung) and sentence level (e.g., Sie sah atemberaubend aus in ihrem samtigen Abendkleid, das bei jedem Schritt leise raschelte – velvet does not rustle). Sometimes, ChatGPT uses a numerus that is uncommon in a given context (Schuhband vs. Schuhbänder), gives an incorrect historical placing of words (e.g., Disco Roller were popular in the 90s not in the 80s), or makes use of subordinate clauses that fail to contribute to the meaning of sentences. Occasionally, ChatGPT generates orthographic errors, uses the wrong case, or hallucinates on words it does not know (e.g., Nebelkappe as synonym to Tarnkappe, which it explains as a kind of cap that one can wear during fog).
In sum, lexicographers must not fear that LLMs are taking over their entire work. Often however, AI generated content is of high quality and can be used with little, if any edits. As a result, our team embraces the new technology as an effective support for the development and maintenance of GermaNet.
References
[1] Gilles-Maurice de Schryver. Generative AI and Lexicography: The Current State of the Art Using ChatGPT. International Journal of Lexicography, 36(4):355–387, 10 2023.
[2] B. Hamp and H. Feldweg. GermaNet - a Lexical-Semantic Net for German. In Proceedings of the ACL workshop Automatic Information Extraction and Building of Lexical Semantic Resources for NLP Applications, 1997. Madrid, Spain.
