Keywords
Cross-cultural
Dimensions of meaning
Multidimensional analysis
Métricas
How to Cite
Abstract
We present a meta-analysis of results from experimental studies on attitude reception in seven languages (Brazilian Portuguese, Japanese, French, German, Cantonese, American English, Hindi). The studies involved free-labeling of perceived attitudes in audio-visual stimuli. The productions of 88 speakers from the seven languages were obtained using the same elicitation methodology, allowing to record sixteen audiovisual attitudes. These performances, rated in preceding works using a free-labeling paradigm, were grouped and analyzed to compare how the attitudinal performances spread along the main dimensions of the shared cognitive representation. A hierarchical clustering then regrouped attitudinal expressions as a function of their cognitive proximity. A large cluster solution showed the main dimension that organizes these expressions may be interpreted as the “Unpredictability” dimension proposed by Fontaine et al. (2007) for emotions, followed by the Evaluation-Pleasantness one; Activation-Arousal and Potency-Control arrived later but played a determinant role in the organization of attitudes. A fine-grained 13-cluster solution showed most attitudes were singled out by the listeners despite the variations in speakers and elicitation contexts. The analysis of each of these clusters brings insight into the cultural similarities and differences in the reception of these different attitudinal expressions. A notable result is the variation in valence attributed to the expression of Irony that underlines the potential communication problems that may be linked to interaction routines. On the other hand, Surprise was clearly identified. The existence and importance of the Unpredictability dimension and its relation to the illocutionary opposition between assertive and interrogative acts underlines the pertinence of Mello & Raso’s (2011) analysis.
References
1. Aubergé V, Grepillat T, Rilliard A. Can we perceive attitudes before the end of sentences? the gating paradigm for prosodic contours. In: 5th European Conference on Speech Communication and Technology (Eurospeech 1997). Rhodes, Greece: ISCA; 1997. p. 871–4.
2. Bécue-Bertaut M, Pagès J. Multiple factor analysis and clustering of a mixture of quantitative, categorical and frequency data. Comput Stat Data Anal. 2008 Feb;52(6):3255–68.
3. Bossaglia G, Mello H, Raso T. Chapter 7. Illocution as a unit of reference for spontaneous speech: An account of insubordinated adverbial clauses in Brazilian Portuguese. In: Izre’el S, Mello H, Panunzi A, Raso T, editors. Studies in Corpus Linguistics [Internet]. Amsterdam: John Benjamins Publishing Company; 2020. p. 221–56. Available from: https://benjamins.com/catalog/scl.94.07bos
4. Brown P, Levinson SC. Politeness: some universals in language usage. Cambridge [Cambridgeshire] ; New York: Cambridge University Press; 1987. (Studies in interactional sociolinguistics).
5. Chang NCT. Tones and Intonation in the Chengtu Dialect (Szechuan, China). Phonetica. 1958 Jan 1;2(1–2):59–85.
6. Culpeper J, Bousfield D, Wichmann A. Impoliteness revisited: with special reference to dynamic and prosodic aspects. J Pragmat. 2003 Oct;35(10–11):1545–79.
7. Erickson D, Rilliard A, de Moraes J, Shochi T. Personality Judgments Based on Speaker’s Social Affective Expressions. In: Fang Q, Dang J, Perrier P, Wei J, Wang L, Yan N, editors. Studies on Speech Production [Internet]. Cham: Springer International Publishing; 2018. p. 3–13. (Lecture Notes in Computer Science; vol. 10733). Available from: http://link.springer.com/10.1007/978-3-030-00126-1_1
8. Erickson D, Rilliard A, Thurgood E, Moraes J, Shochi T. Acoustic and perceptual profiles of American English social affective expressions: a case study. J Speech Sci. 2024 Dec 18;13:e024004.
9. Escofier B, Pagès J. Multiple factor analysis (AFMULT package). Comput Stat Data Anal. 1994 Aug;18(1):121–40.
10. Fónagy I. Vocal Expression of Emotions and Attitudes. Quad Studi Semiot. 1987;47/48:65–85.
11. Fónagy I, Bérard E. «Il est huit heures»: contribution à l’analyse sémantique de la vive voix. Phonetica. 1972;26(3):157–92.
12. Fónagy I, Bérard E, Fónagy J. Clichés mélodiques. Folia Linguist. 1983;17(1–4):153–86.
13. Fontaine JRJ, Scherer KR, Roesch EB, Ellsworth PC. The World of Emotions is not Two-Dimensional. Psychol Sci. 2007 Dec;18(12):1050–7.
14. Fujisaki H, Hirose K. Analysis and perception of intonation expressing paralinguistic information in spoken Japanese. In: ESCA Workshop on Prosody. Lund, Sweden; 1993. p. 254–7.
15. Goudbeek M, Scherer K. Beyond arousal: Valence and potency/control cues in the vocal expression of emotion. J Acoust Soc Am. 2010;128(3):1322–36.
16. Gu W, Zhang T, Fujisaki H. Prosodic analysis and perception of Mandarin utterances conveying attitudes. In: Interspeech 2011 [Internet]. ISCA; 2011. p. 1069–72. Available from: https://www.isca-archive.org/interspeech_2011/gu11_interspeech.html
17. Guerry M, Rilliard A, Erickson D, Shochi T. Perception of prosodic social affects in Japanese: A free-labeling study. In: Speech Prosody 2016 [Internet]. ISCA; 2016a. p. 811–5. Available from: https://www.isca-speech.org/archive/speechprosody_2016/guerry16_speechprosody.html
18. Guerry M, Shochi T, Rilliard A, Erickson D. Perception of prosodic social affects in French: a free-labeling study. In: 18th International Congress of Phonetic Sciences [Internet]. Glasgow, UK; 2015. p. 4p. Available from: https://www.internationalphoneticassociation.org/icphs-proceedings/ICPhS2015/proceedings.html
19. Guerry M, Shochi T, Rilliard A, Erickson D. Perception of Prosodic Social Affects in Japanese by American Learners of Japanese. In: International Symposium on Applied Phonetics (ISAPh 2016) [Internet]. ISCA; 2016b. p. 84–8. Available from: https://www.isca-speech.org/archive/isaph_2016/guerry16_isaph.html
20. Hill B, Ide S, Ikuta S, Kawasaki A, Ogino T. Universals of linguistic politeness: Quantitative evidence from Japanese and American English. J Pragmat. 1986 Jun;10(3):347–71.
21. Hirst D, Di Cristo A. A survey of intonation systems. In: Hirst D, Di Cristo A, editors. Intonation systems: a survey of twenty languages. Cambridge, U.K.: Cambridge University Press; 1998. p. 1–44.
22. Hirst DJ. Form and function in the representation of speech prosody. Speech Commun. 2005;46(3/4):334–47.
23. Hönemann A, Mixdorff H, Rilliard A. Social Attitudes - Recordings and Evaluation of an audio-visual Corpus in German. In: 7th Forum Acusticum 2014, Krakow. 2014.
24. Husson F, Josse J, Le S, Mazet J. FactoMineR: Multivariate Exploratory Data Analysis and Data Mining with R [Internet]. 2013. Available from: http://CRAN.R-project.org/package=FactoMineR
25. Husson F, Lê S, Pagès J. Exploratory multivariate analysis by example using R. Second edition. Boca Raton: CRC Press; 2017.
26. Ide S. The Speaker’s Viewpoint and Indexicality in a High Context Culture. In: Kataoka K, Ide S, editors. Bunka, Intaakushon, Gengo [Culture, Interaction, and Language]. Tokyo: Hitsuji Shobo; 2002. p. 3–20.
27. Kerbrat-Orecchioni C. Politeness in France: How to buy bread politely. In: Hickey L, Stewart M, editors. Politeness in Europe. Clevedon, UK: Multilingual Matters; 2005. p. 29–44.
28. Kerbrat-Orecchioni C. A multilevel approach in the study of talk-in-interaction. Pragmat Q Publ Int Pragmat Assoc IPrA. 2022 Jul 6;7(1):1–20.
29. Khatri P, Mixdorff H, Rao P, Rilliard A. Recognition of audio-visual attitudes. In: Grawunder S, editor. Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2025 [Internet]. TUDpress, Dresden; 2025. p. 19–26. Available from: https://www.essv.de/pdf/2025_19_26.pdf
30. Lê S, Josse J, Husson F. FactoMineR : An R Package for Multivariate Analysis. J Stat Softw [Internet]. 2008;25(1). Available from: http://www.jstatsoft.org/v25/i01/
31. Lee T, Ma KHM, Rilliard A, Mixdorff H, Hönemann A. Free Labeling of Audio-visual Attitudinal Expressions in Cantonese. In: Speech Prosody 2018 [Internet]. ISCA; 2018. p. 483–7. Available from: https://www.isca-speech.org/archive/speechprosody_2018/lee18_speechprosody.html
32. Levinson SC. Turn-taking in Human Communication – Origins and Implications for Language Processing. Trends Cogn Sci. 2016 Jan;20(1):6–14.
33. Lu Y, Aubergé V, Rilliard A. Do you hear my attitude? prosodic perception of social affects in Mandarin. In: Speech Prosody 2012. ISCA; 2012. p. 685–8.
34. Mac DK, Aubergé V, Rilliard A, Castelli E. Audio-visual prosody of social attitudes in vietnamese: building and evaluating a tones balanced corpus. In: Interspeech 2009. ISCA; 2009. p. 2263–6.
35. Martins-Baltar M. De l’énoncé à l’énonciation: une approche des fonctions intonatives. Paris: Didier; 1977.
36. Mello H, Raso T. Illocution, modality, attitude: different names for different categories. In: Mello H, Panunzi A, Raso T, editors. Pragmatics and Prosody: Illocution, Modality, Attitude, Information Patterning and Speech Annotation [Internet]. 1st ed. Firenze: Firenze University Press; 2011. p. 1–18. (Strumenti per la didattica e la ricerca; vol. 120). Available from: https://books.fupress.com/isbn/9788866550846
37. Miranda L, Swerts M, de Moraes JA, Rilliard A. The Role of the Auditory and Visual Modalities in the Perceptual Identification of Brazilian Portuguese Statements and Echo Questions. Lang Speech. 2021 Mar;64(1):3–23.
38. Mixdorff H, Ghosh D, Rilliard A, Hönemann A. Perception of Audio-visual Expressions in German and Cantonese by Native Speakers of Hindi. In: Speech Prosody 2020 [Internet]. ISCA; 2020. p. 31–5. Available from: https://www.isca-speech.org/archive/speechprosody_2020/mixdorff20_speechprosody.html
39. Mixdorff H, Hönemann A, Rilliard A, Lee T, Ma MKH. Audio-visual expressions of attitude: How many different attitudes can perceivers decode? Speech Commun. 2017 Dec;95:114–26.
40. Mixdorff H, Nayan N, Rilliard A, Rao P, Ghosh D. Developing a corpus of audio-visual attitudinal expressions in Hindi. In: Proceedings of the 20th ICPhS [Internet]. Prague; 2023. p. 4150–4. Available from: https://guarant.cz/icphs2023/631.pdf
41. Mixdorff H, Rilliard A, Nayan N. Perceptual Evaluation of Attitudinal Expressions. In: 5th International Symposium on Applied Phonetics (ISAPh 2024) [Internet]. ISCA; 2024. p. 60–4. Available from: https://www.isca-archive.org/isaph_2024/mixdorff24_isaph.html
42. Moore CC, Romney AK, Hsia TL, Rusch CD. The Universality of the Semantic Structure of Emotion Terms: Methods for the Study of Inter- and Intra-Cultural Variability. Am Anthropol. 1999 Sep;101(3):529–46.
43. de Moraes JA. The pitch accents in Brazilian portuguese: analysis by synthesis. In: Speech Prosody 2008. ISCA; 2008. p. 389–97.
44. de Moraes JA. From a prosodic point of view: remarks on attitudinal meaning. In: Mello H, Panunzi A, Raso T, editors. Pragmatics and Prosody: Illocution, Modality, Attitude, Information Patterning and Speech Annotation [Internet]. 1st ed. Firenze: Firenze University Press; 2011. p. 19–37. (Strumenti per la didattica e la ricerca; vol. 120). Available from: https://books.fupress.com/isbn/9788866550846
45. de Moraes JA, Rilliard A. Illocution, attitudes and prosody: A multimodal analysis. In: Raso T, Mello H, editors. Studies in Corpus Linguistics [Internet]. Amsterdam: John Benjamins Publishing Company; 2014. p. 233–70. Available from: https://benjamins.com/catalog/scl.61.09mor
46. de Moraes JA, Rilliard A. Prosody and Emotion in Brazilian Portuguese. In: Armstrong ME, Henriksen N, del Mar Vanrell M, editors. Issues in Hispanic and Lusophone Linguistics [Internet]. Amsterdam: John Benjamins Publishing Company; 2016. p. 135–52. Available from: https://benjamins.com/catalog/ihll.6.07mor
47. de Moraes JA, Rilliard A, Mota BADO, Shochi T. Multimodal perception and production of attitudinal meaning in Brazilian Portuguese. In: Speech Prosody 2010. Chicago, IL, USA: ISCA; 2010. p. 1–4.
48. Ohala JJ. The frequency code underlies the sound-symbolic use of voice pitch. In: Hinton L, Nichols J, Ohala JJ, editors. Sound Symbolism [Internet]. 1st ed. Cambridge University Press; 1994. p. 325–47. Available from: https://www.cambridge.org/core/product/identifier/CBO9780511751806A036/type/book_part
49. Osgood CE, May WH, Miron MS. Cross-cultural universals of affective meaning. University of Illinois Press; 1975.
50. Osgood CE, Suci GJ, Tannenbaum PH. The measurement of meaning. Urbana: University of Illinois Press; 1957.
51. Pierrehumbert J, Hirschberg J. The Meaning of Intonational Contours in the Interpretation of Discourse. In: Cohen PR, Morgan J, editors. Intentions in communication. Cambridge, Mass.: MIT Press; 1990. p. 271–311. (System development foundation benchmark series).
52. Post B, d’Imperio M, Gussenhoven C. Fine phonetic detail and intonational meaning. In Saarbruecken, Germany; 2007. p. 191–6.
53. R Core Team. R: A Language and Environment for Statistical Computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2023. Available from: https://www.R-project.org/
54. Rilliard A, Erickson D, de Moraes JA, Shochi T. Cross-cultural perception of some Japanese politeness and impoliteness expressions. In: Baider F, Cislaru G, editors. Linguistic Approaches to Emotions in Context [Internet]. Amsterdam: John Benjamins Publishing Company; 2014. p. 251–76. (Pragmatics & Beyond New Series; vol. 241). Available from: https://benjamins.com/catalog/pbns.241.15ril
55. Rilliard A, Erickson D, de Moraes JA, Shochi T. Perception of expressive prosodic speech acts performed in USA English by L1 and L2 speakers. J Speech Sci. 2017 Nov 1;6(1):27–45.
56. Rilliard A, Erickson D, Shochi T, de Moraes JA. Social face to face communication — American English attitudinal prosody. In: Interspeech 2013 [Internet]. ISCA; 2013. p. 1648–52. Available from: https://www.isca-speech.org/archive/interspeech_2013/rilliard13_interspeech.html
57. Rilliard A, de Moraes JA. Social affective variations in Brazilian Portuguese: a perceptual and acoustic analysis. Rev Estud Ling. 2017 Jun 13;25(3):1043–74.
58. Romney AK, Boyd JP, Moore CC, Batchelder WH, Brazill TJ. Culture as shared cognitive representations. Proc Natl Acad Sci. 1996 May 14;93(10):4699–705.
59. Romney AK, Moore CC, Rusch CD. Cultural universals: Measuring the semantic structure of emotion terms in English and Japanese. Proc Natl Acad Sci. 1997 May 13;94(10):5489–94.
60. Russell JA. Culture and the categorization of emotions. Psychol Bull. 1991;110(3):426.
61. Sadanobu T. A natural history of Japanese pressed voice. J Phon Soc Jpn. 2004;8(1):29–44.
62. Sadanobu T. An Unofficial Guide for Japanese Characters [Internet]. 2012. Available from: http://dictionary.sanseido-publ.co.jp/wp/author/sadanobu-e/
63. Sadanobu T. “Characters” in Japanese Communication and Language: An Overview. Acta Linguist Asiat. 2015 Dec 29;5(2):9–28.
64. Scherer KR. Emotions are emergent processes: they require a dynamic computational architecture. Philos Trans R Soc B Biol Sci. 2009a Dec 12;364(1535):3459–74.
65. Scherer KR. The dynamic architecture of emotion: Evidence for the component process model. Cogn Emot. 2009b Nov;23(7):1307–51.
66. Shochi T, Brousse A, Guerry M, Erickson D, Rilliard A. Learning effect of social affective prosody in Japanese by French learners. In: Speech Prosody 2016 [Internet]. ISCA; 2016. p. 970–3. Available from: https://www.isca-speech.org/archive/speechprosody_2016/shochi16_speechprosody.html
67. Shochi T, Gagnié G, Rilliard A, Erickson D, Aubergé V. Perception of prosodic French social affects for Japanese learners of French language. Trans Tech Comm Psychol Physiol Acoust Acoust Soc Jpn. 2009a;39.
68. Shochi T, Gagnié G, Rilliard A, Erickson D, Aubergé V. Learning effect of prosodic social affects for Japanese learners of French language. In: Speech Prosody 2010. ISCA; 2010. p. 1–4.
69. Shochi T, Rilliard A, Aubergé V, Erickson D. Intercultural perception of English, French and Japanese social affective prosody. In: Hancil S, editor. The role of prosody in affective speech. Bern, Switzerland: Peter Lang AG; 2009b. p. 31–60. (Linguistic Insights).
70. Shochi T, Rilliard A, Erickson D. Chapter 8. Perceptual changes between adults and children for multimodal im/politeness in Japanese. In: Jucker AH, Hübscher I, Brown L, editors. Multimodal Im/politeness: Signed, spoken, written [Internet]. Amsterdam: John Benjamins Publishing Company; 2023. p. 213–49. (Pragmatics & Beyond New Series; vol. 333). Available from: https://benjamins.com/catalog/pbns.333.08sho
71. Spencer-Oatey H. Reconsidering power and distance. J Pragmat. 1996 Jul;26(1):1–24.
72. Swerts M, Krahmer E. Audiovisual prosody and feeling of knowing. J Mem Lang. 2005 Jul;53(1):81–94.
73. Uldall E. Attitudinal meanings conveyed by intonation contours. Lang Speech. 1960;3(4):223–34.
74. Wichmann A. The attitudinal effects of prosody, and how they relate to emotion. In: ISCA Tutorial and Research Workshop (ITRW) on Speech and Emotion. 2000. p. 143–8.
75. Wierzbicka A. Human emotions: universal or culture-specific? Am Anthropol. 1986;88(3):584–94.
76. Wierzbicka A. Defining emotion concepts. Cogn Sci. 1992;16(4):539–81.
77. Wierzbicka A. Japanese cultural scripts: Cultural psychology and “cultural grammar.” Ethos. 1996;24(3):527–55.
78. Wierzbicka A. “Happiness” in cross-linguistic & cross-cultural perspective. Dædalus. 2004;133(2):34–43.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2025 Albert Rilliard, João Antônio de Moraes, Donna Erickson, Marine Guerry, Angelika Hönemann, Tan Lee, Matthew King-Hang Ma, Hansjörg Mixdorff, Preeti Rao, Takaaki Shochi