Sunghye Cho and Mark Liberman are authors on a new paper with Galit Agmon, Sameer Pradhan, Sharon Ash, Naomi Nevler, and Murray Grossman:
Agmon, G., Pradhan, S., Ash, S., Nevler, N., Liberman, M., Grossman, M., and Cho, S. (2024) Automated measures of syntactic complexity in natural speech production: Older and younger adults as a case study. Journal of Speech, Language, and Hearing Research. https://doi.org/10.1044/2023_JSLHR-23-00009
Abstract
Method: We used natural speech samples produced in a picture description task by younger (n= 76, ages 18–22 years) and older (n= 36, ages 53–89 years) healthy participants, manually transcribed and segmented into sentences. We manually verified that older participants produced fewer complex structures. We developed a metric of syntactic complexity using automatically extracted syntactic structures as features in a multidimensional metric. We compared our metric to seven other metrics: Yngve score, Frazier score, Frazier–Roark score, developmental level, syntactic frequency, mean dependency distance, and sentence length. We examined the success of each metric in identifying the age group using logistic regression models. We repeated the analysis with automatic transcription and segmentation using an automatic speech recognition (ASR) system.
Results: Our multidimensional metric was successful in predicting age group (area under the curve [AUC] = 0.87), and it performed better than the other metrics. High AUCs were also achieved by the Yngve score (0.84) and sentence length (0.84). However, in a fully automated pipeline with ASR, the performance of these two metrics dropped (to 0.73 and 0.46, respectively), while the performance of the multidimensional metric remained relatively high (0.81).
Conclusions: Syntactic complexity in spontaneous speech can be quantified by directly assessing syntactic structures and considering them in a multivariable manner. It can be derived automatically, saving considerable time and effort compared to manually analyzing large-scale corpora, while maintaining high face validity and robustness.