Development of an automatic attitude recognition system: a multimodal analysis of video blogs

Communicative content in human communication involves expressivity of socio-affectivestates. Research in Linguistics, Social Signal Processing and Affective Computing in particular, highlights the importance of affect, emotion and attitudes as sources of information forcommunicative content. Attitud...

Full description

Saved in:

Bibliographic Details
Main Author:	Noor Alhusna Madzlan
Format:	thesis
Language:	eng
Published:	2017
Subjects:	TK Electrical engineering Electronics Nuclear engineering
Online Access:	https://ir.upsi.edu.my/detailsg.php?det=5364
Tags:	Add Tag No Tags, Be the first to tag this record!

id	oai:ir.upsi.edu.my:5364
record_format	uketd_dc
institution	Universiti Pendidikan Sultan Idris
collection	UPSI Digital Repository
language	eng
topic	TK Electrical engineering Electronics Nuclear engineering
spellingShingle	TK Electrical engineering Electronics Nuclear engineering Noor Alhusna Madzlan Development of an automatic attitude recognition system: a multimodal analysis of video blogs
description	Communicative content in human communication involves expressivity of socio-affectivestates. Research in Linguistics, Social Signal Processing and Affective Computing in particular, highlights the importance of affect, emotion and attitudes as sources of information forcommunicative content. Attitudes, considered as socio-affective states of speakers, areconveyed through a multitude of signals during communication. Understanding the expres sion ofattitudes of speakers is essential for establishing successful communication. Taking theempirical approach to studying attitude expressions, the main objective of this research isto contribute to the development of an automatic attitude classification system through afusion of multimodal signals expressed by speakers in video biogs. The present study describes a new communicative genre of self-expression through social media: video blogging, whichprovides opportunities for interlocutors to disseminate information through a myriad of multimodal characteristics. This study describes main features of this novel communica tion medium andfocuses attention to its possible exploitation as a rich source of information for humancommunication. The dissertation describes manual annotation of attitude expres sions from the vlogcorpus, multimodal feature analysis and processes for development of an automatic attitudeannotation system. An ontology of attitude annotation scheme for speech in video biogs iselaborated and five attitude labels are derived. Prosodic and visual fea tureextraction procedures are explained in detail. Discussion on processes of developing an automaticattitude classification model includes analysis of automatic prediction of attitude labelsusing prosodic and visual features through machine-learning methods. This study also elaboratesdetailed analysis of individual feature contributions and their predictive power tothe classification task
format	thesis
qualification_name
qualification_level	Doctorate
author	Noor Alhusna Madzlan
author_facet	Noor Alhusna Madzlan
author_sort	Noor Alhusna Madzlan
title	Development of an automatic attitude recognition system: a multimodal analysis of video blogs
title_short	Development of an automatic attitude recognition system: a multimodal analysis of video blogs
title_full	Development of an automatic attitude recognition system: a multimodal analysis of video blogs
title_fullStr	Development of an automatic attitude recognition system: a multimodal analysis of video blogs
title_full_unstemmed	Development of an automatic attitude recognition system: a multimodal analysis of video blogs
title_sort	development of an automatic attitude recognition system: a multimodal analysis of video blogs
granting_institution	Universiti Pendidikan Sultan Idris
granting_department	Fakulti Bahasa dan Komunikasi
publishDate	2017
url	https://ir.upsi.edu.my/detailsg.php?det=5364
_version_	1747833183961874432
spelling	oai:ir.upsi.edu.my:53642020-11-17 Development of an automatic attitude recognition system: a multimodal analysis of video blogs 2017 Noor Alhusna Madzlan TK Electrical engineering. Electronics Nuclear engineering Communicative content in human communication involves expressivity of socio-affectivestates. Research in Linguistics, Social Signal Processing and Affective Computing in particular, highlights the importance of affect, emotion and attitudes as sources of information forcommunicative content. Attitudes, considered as socio-affective states of speakers, areconveyed through a multitude of signals during communication. Understanding the expres sion ofattitudes of speakers is essential for establishing successful communication. Taking theempirical approach to studying attitude expressions, the main objective of this research isto contribute to the development of an automatic attitude classification system through afusion of multimodal signals expressed by speakers in video biogs. The present study describes a new communicative genre of self-expression through social media: video blogging, whichprovides opportunities for interlocutors to disseminate information through a myriad of multimodal characteristics. This study describes main features of this novel communica tion medium andfocuses attention to its possible exploitation as a rich source of information for humancommunication. The dissertation describes manual annotation of attitude expres sions from the vlogcorpus, multimodal feature analysis and processes for development of an automatic attitudeannotation system. An ontology of attitude annotation scheme for speech in video biogs iselaborated and five attitude labels are derived. Prosodic and visual fea tureextraction procedures are explained in detail. Discussion on processes of developing an automaticattitude classification model includes analysis of automatic prediction of attitude labelsusing prosodic and visual features through machine-learning methods. This study also elaboratesdetailed analysis of individual feature contributions and their predictive power tothe classification task 2017 thesis https://ir.upsi.edu.my/detailsg.php?det=5364 https://ir.upsi.edu.my/detailsg.php?det=5364 text eng closedAccess Doctoral Universiti Pendidikan Sultan Idris Fakulti Bahasa dan Komunikasi [l] Jens Allwood. A framework for studying human multimodal communication. Coverbal Synchrony in Human-Machine Interaction, page 17, 2013.[2] Antonio Damasio and Raymond J Dolan. The feeling of what happens. Nature,401(6756):847-847, 1999.[3] Antonio Damasio. Descartes' error: Emotion, reason and the human brain. Random House,2008.[4] James A Russell and Doreen Ridgeway. Dimensions underlying children's emotion concepts.Developmental Psychology, 19(6):795, 1983.[5] M. Wetherell. Affect and Emotion: A New Social Science Understanding. SAGE Publications,2012.[6] Mark P Zanna and John K Rempel. Attitudes: A new look at an old concept. 1988.[7] Eric Shouse. Feeling, emotion, affect. Mic journal, 8(6):26, 2005.[8] Stuart Oskamp and P Wesley Schultz. Attitudes and opinions. Psychology Press, 1977.[9] Martin Fishbein and leek Ajzen. Attitudes and opinions. Annual review ofpsychology,23(1):487-544, I 972.[ IO] Veronique Auberge. A gestalt morphology of prosody directed by functions: the exampleofa step by step model developed at icp. In Speech Prosody 2002, International Conference, 2002.[11) Jens Allwood. Multimodal corpora. Corpus linguistics: An international handbook, l:207-224, 2008[12) Alessandro Vinciarelli and Gelareh Mohammadi. Towards a technology of nonverbalcom munication: Vocal behavior in social and affective phenomena. Technical report.igi-global,2010[13] Mark Knapp, Judith Hall, and Terrence Horgan. Nonverbal communication in human interaction. Cengage Learning, 2013.[14] Virginia P Richmond, James C McCroskey, and Steven K Payne. N0/1\'erbaI beIiav1or 1??111ter- personal relations. Prentice Hall Englewood Cliffs, NJ, 1991.[15] Paul Ekman. Facial expression and emotion. American psychologist, 48(4):384, 1993.[16] Alessandro Vinciarelli and Fabio Valente. Social signal processing: Understandingnonverbal communication in social interactions. In Proceedings of Measuring Behavior 2010,Eindhoven (The Netherlands), number EPFL-CONF-163182, 20 I 0.[17] Louis-Philippe Morency, Rada Mihalcea, and Paya) Doshi. Towards multimodalsentiment analysis: Harvesting opinions from the web. In Proceedings of the 13th internationalconfer ence on multimodal inte1faces, pages 169-176. ACM, 2011.[18] Veronica Rosas, Rada Mihalcea, and L Morency. Multimodal sentiment analysis of spanishonline videos. 2013.[19] Christer Gobi, Ailbhe N1, et al. The role of voice quality in communicating emotion,mood and attitude. Speech communication, 40(1):189-212, 2003.[20] Ginevra Castellano, Santiago D Villalba, and Antonio Camurri. Recognising human emotionsfrom body movement and gesture dynamics. In Affective computing and intelligent interaction, pages71-82. Springer, 2007.[21] Nadia Bianchi-Berthouze, Paul Cairns, Anna Cox, Charlene Jennett, and Whan Woong Kim.On posture as a modality for expressing and recognizing emotions. In Emotion and HCIworkshop at BCS HCI London, 2006.[22] Dang-Khoa Mac, Veronique Auberge, Albert Rilliard, and Eric Castelli. Cross-culturalper ception of vietnamese audio-visual prosodic attitudes. In Speech Prosody, 2010.[23] Joao Antonio de Moraes, Albert Rilliard, Bruno Alberto de Oliveira Mota, and Takaaki Shochi.Multimodal perception and production of attitudinal meaning in brazilian portuguese. In Proc. ofSpeech Pmsody, 20 I 0.[24] Yann Morlee, Grard Bailly, and Vronique Auberg. Generating prosodic attitudes infrench: Data. model and evaluation. Speech Communication. 33(4):357-371. 2001.[25] Bonn ie A Nardi, Diane J Schia no' and Michelle Gumbrecht. B l oggmg as social act1 v1ty. or. would you let 900 million people read your ct ? I p 1rnry. n rocee<mgs of the 2004 ACM co1if

Development of an automatic attitude recognition system: a multimodal analysis of video blogs

Similar Items