Semantic networks in humans and Large Language Models in Spanish: a semantic fluency approach
DOI:
https://doi.org/10.21814/diacritica.6867Keywords:
Semantic fluency, Semantic networks, Lexical availability, Large Language Models, SpanishAbstract
Semantic fluency tasks, like those traditionally employed in lexical availability research, provide a well-established method for analysing how lexical knowledge is retrieved and organised in humans. This study applies these tasks to Large Language Models (LLMs), comparing their lexical productions with those of human participants in Spanish for the category animals. One hundred university students and five LLMs (ChatGPT, Gemini, Claude, Grok, LLaMA) were tested, and their responses were analysed using both quantitative and qualitative data and network-based metrics. Results reveal a clear asymmetry between humans and LLMs. Human participants generated more diverse and heterogeneous lexical sets, producing networks characterised by lower connectivity but higher modularity. LLMs, by contrast, displayed convergent and cohesive outputs, forming denser networks with higher average degrees and clustering but reduced modularity and novelty. Disaggregated analysis showed systematic differences across the LLMs themselves. These findings confirm that LLMs approximate the underlying structure of semantic fluency but do not fully replicate the variability and contextual sensitivity characteristic of human lexical production. The study highlights both the strengths and limitations of artificial systems as models of lexical organization and points to new directions for cross-linguistic comparisons.
References
Abramski, K., Citraro, S., Lombardi, L., Rossetti, G., & Stella, M. (2023). Cognitive network science reveals bias in GPT-3, GPT-3.5 turbo, and GPT-4 mirroring math anxiety in high-school students. Big Data and Cognitive Computing, 7(3), 124. https://doi.org/10.3390/bdcc7030124
Agustín-Llach, M. P., & Rubio, J. (2024). Navigating the mental lexicon: network structures, lexical search and lexical retrieval. Journal of Psycholinguistic Research, 53, 21. https://doi.org/10.1007/s10936-024-10059-8
Aitchison, J. (2003). Words in the mind. Blackwell.
Bartol Hernández, J. A. (2007). El proyecto panhispánico de léxico disponible. In C. Hernández Alonso & L. Castañeda San Cirilo (Eds.), El español de América: Actas del VI Congreso Internacional de “El español de América” (Tordesillas, Valladolid, 25-29 de octubre de 2005) (pp. 305–318). Diputación Provincial de Valladolid.
Battig, W. F., & Montague, W. E. (1969). Category norms of verbal items in 56 categories: a replication and extension of the Connecticut category norms. Journal of Experimental Psychology, 80(3, Pt. 2), 1–46. https://doi.org/10.1037/h0027577
Beguš, G., Dabkowski, M., & Rhodes, R. (2025). Large linguistic models: investigating LLMs’ metalinguistic abilities. In IEEE Transactions on Artificial Intelligence (vol. 6, pp. 3453–3467). https://doi.org/10.1109/TAI.2025.3575745
Borge-Holthoefer, J., & Arenas, A. (2010). Semantic networks: structure and dynamics. Entropy, 12(5), 1264–1302. https://doi.org/10.3390/e12051264
Borodkin, K., Kenett, Y. N., Faust, M., & Mashal, N. (2016). When pumpkin is closer to onion than to squash: the structure of the second language lexicon. Cognition, 156, 60–70.
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., & Xie, X. (2024). A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3), 1–45. https://dl.acm.org/doi/10.1145/3641289
Chomsky, N., Roberts, I., & Watumull, J. (2024, March 16). Noam Chomsky: The false promise of ChatGPT. The New York Times. https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html
De Deyne, S., & Storms, G. (2008). Word associations: network and semantic properties. Behavior Research Methods, 40, 213–231. https://doi.org/10.3758/BRM.40.1.213
Dentella, V., Günther, F., Murphy, E., Marcus, G., & Leivada, E. (2024). Testing AI on language comprehension tasks reveals insensitivity to underlying meaning. Scientific Reports, 14, 28083. https://doi.org/10.1038/s41598-024-79531-8
Digutsch, J., & Kosinski, M. (2023). Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans. Scientific Reports, 13(1), 5035.
Gougenheim, G., Michéa, R., Rivenc, P., & Sauvageot, A. (1964). L’élaboration du Français Fondamental (I Degré). Étude sur l’établissement d’un vocabulaire et d’une grammaire de base. Didier.
Gudmundson, A. (2020). The mental lexicon of multilingual adult learners of Italian L3: a study of word association behavior and cross-lingual semantic priming. In C. Bardel & L. Sánchez (Eds.), Third language acquisition: age, proficiency and multilingualism (pp. 67–109). Language Science Press.
Hadi, M. U., Al Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., Shoman, M., Wu, J., Mirjalili, S., & Shah, M. (2025, February 10). Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. TechRxiv. https://doi.org/10.36227/techrxiv.23589741.v4
Hansen, H., & Hebart, M. N. (2022). Semantic features of object concepts generated with GPT-3. In Proceedings of the Annual Meeting of the Cognitive Science Society, 44. https://escholarship.org/uc/item/44s454ng
Harnad, S. (2005). Cognition is categorization. In H. Cohen & C. Lefebvre (Eds.), Handbook of categorization in cognitive science. Elsevier.
Hernández Muñoz, N. (2006). Hacia una teoría cognitiva integrada de la disponibilidad léxica: el léxico disponible de los estudiantes castellano-manchegos. Ediciones Universidad de Salamanca.
Hernández Muñoz, N., Tomé Cornejo, C., & López García, M. (2025a). Análisis de las herramientas automáticas para el léxico disponible: LexPro. In I. C. Santos Díaz & A. M. Ávila Muñoz (Eds.), Avances y desarrollo de los estudios sobre el léxico disponible. Una aproximación desde la variación léxica (pp. 71–91). Peter Lang.
Hernández Muñoz, N., Tomé Cornejo, C., & López, M. (2025b). Redefining linguistic categories through network theory. Language and Cognition, 17, e40. https://doi.org/10.1017/langcog.2025.10
Kleiber, G. (1990). La sémantique du prototype : catégories et sens lexical. PUF.
López-Chávez, J., & Strassburger, C. (2000). El diseño de una fórmula matemática para obtener un índice de disponibilidad léxica confiable. Anuario de Letras, 38, 227–251.
López Morales, H. (1999). Léxico disponible de Puerto Rico. Arco/Libros.
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024). Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011
Piantadosi, S. T. (2024). Modern language models refute Chomsky’s approach to language. In E. Gibson & M. Poliak (Eds.), From fieldwork to linguistic theory: a tribute to Dan Everett (pp. 353–414). Language Science Press. https://doi.org/10.5281/zenodo.12665933
Samper Padilla, J. A., Bellón Fernández, J. J., & Samper Hernández, M. (2003). El proyecto de estudio de la disponibilidad léxica en español. In R. Ávila, J. A. Samper, H. Ueda et al. (Eds.), Pautas y pistas en el análisis del léxico hispano(americano) (pp. 27–140). Iberoamericana Vervuert.
Sánchez-Saus Laserna, M. (2024). Lexical availability and foreign language teaching: Main contributions of a growing field. Theory and Practice of Second Language Acquisition, 10(1), 1–20. https://doi.org/10.31261/TAPSLA.14839
Sánchez-Saus Laserna, M. (2025). Evolución de los centros de interés en la investigación sobre disponibilidad léxica. Orígenes y nuevas perspectivas. In N. Hernández Muñoz & C. Tomé Cornejo (Eds.), Más allá de la palabra: Investigaciones en torno a la organización y recuperación del léxico (pp. 17–42). Visor.
Steyvers, M., & Tenenbaum, J. B. (2005). The large-scale structure of semantic networks: statistical analyses and a model of semantic growth. Cognitive Science, 29(1), 41–78.
Suresh, S., Mukherjee, K., Yu, X., Huang, W.-C., Padua, L., & Rogers, T. (2023). Conceptual structure coheres in human cognition but not in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 722–738). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.47
Tomé Cornejo, C. (2025). Exploring the structure of semantic networks in Spanish as a foreign language. In R. M. Jiménez Catalán (Ed.), Lexicon exploration in English and Spanish learners (Educational Linguistics, vol 67). Springer. https://doi.org/10.1007/978-3-031-99435-7_7
Tomé Cornejo, C., Hernández Muñoz, N., & López García, M. (2025). The network as a process: on the role of the speaker in the construction of semantic networks in Spanish L1 and L2. Caplletra. Revista Internacional de Filología, 78, 33–52. https://doi.org/10.7203/caplletra.78.30264
Wang, Y., Deng, Y., Wang, G., Li, T., Xiao, H., & Zhang, Y. (2025). The fluency-based semantic network of LLMs differs from humans. Computers in Human Behavior: Artificial Humans, 3, 100103. https://doi.org/10.1016/j.chbah.2024.100103
Xiao, B., Duan, X., Haslett, D. A., & Cai, Z. G. (2025). Human-likeness of LLMs in the mental lexicon [Preprint]. In Proceedings of the 29th Conference on Computational Natural Language Learning (pp. 586–601). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.conll-1.38
Yamin, K., Ghosal, G., & Wilder, B. (2025). LLMs struggle to perform counterfactual reasoning with parametric knowledge. arXiv. https://doi.org/10.48550/arXiv.2506.15732
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Marta Sánchez-Saus Laserna, Mario Crespo Miguel, Luis Escoriza Morera, Vanesa Álvarez Torres
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
