Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

Please always quote using this URN: urn:nbn:de:bvb:20-opus-114548
  • In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice's emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broaderIn everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice's emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.show moreshow less

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author: Antje B. M. Gerdes, Matthias J. Wieser, Georg W. Alpers
URN:urn:nbn:de:bvb:20-opus-114548
Document Type:Journal article
Faculties:Fakultät für Humanwissenschaften (Philos., Psycho., Erziehungs- u. Gesell.-Wissensch.) / Institut für Psychologie
Language:English
Parent Title (English):Frontiers in Psychology
ISSN:1664-1078
Year of Completion:2014
Volume:5
Pagenumber:1351
Source:Frontiers Psychology 5:1351. doi:10.3389/fpsyg.2014.01351
DOI:https://doi.org/10.3389/fpsyg.2014.01351
Pubmed Id:https://pubmed.ncbi.nlm.nih.gov/25520679
Dewey Decimal Classification:1 Philosophie und Psychologie / 15 Psychologie / 150 Psychologie
Tag:attention capture; audiovisual integration; audiovisual interactions; auditory stimuli; emotional pictures; emotional scene stimuli; emotional sounds; facial expressions; human amygdala; human brain; multimodal emotion processing; multisensory integration; relevance detection; social anxiety; visual stimuli
Release Date:2015/06/30
Licence (German):License LogoCC BY: Creative-Commons-Lizenz: Namensnennung