Interactions between Auditory Elevation, Auditory Pitch and Visual Elevation during Multisensory Perception

Yaseen Jamal, Simon Lacey, Lynne Nygaard, Krishnankutty Sathian

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Cross-modal correspondences refer to associations between apparently unrelated stimulus features in different senses. For example, high and low auditory pitches are associated with high and low visual elevations, respectively. Here we examined how this crossmodal correspondence between visual elevation and auditory pitch relates to auditory elevation. We used audiovisual combinations of high- or low-frequency bursts of white noise and a visual stimulus comprising a white circle. Auditory and visual stimuli could each occur at high or low elevations. These multisensory stimuli could be congruent or incongruent for three correspondence types: cross-modal featural (auditory pitch/visual elevation), within-modal featural (auditory pitch/auditory elevation) and cross-modal spatial (auditory and visual elevation). Participants performed a 2AFC speeded classification (high or low) task while attending to auditory pitch, auditory elevation, or visual elevation. We tested for modulatory interactions between the three correspondence types. Modulatory interactions were absent when discriminating visual elevation. However, the within-modal featural correspondence affected the cross-modal featural correspondence during discrimination of auditory elevation and pitch, while the reverse modulation was observed only during discrimination of auditory pitch. The cross-modal spatial correspondence modulated the other two correspondences only when auditory elevation was being attended, was modulated by the cross-modal featural correspondence only during attention to auditory pitch, and was modulated by the within-modal featural correspondence while performing discrimination of either auditory elevation or pitch. We conclude that the cross-modal correspondence between auditory pitch and visual elevation interacts strongly with auditory elevation.

Original languageEnglish (US)
Pages (from-to)287-306
Number of pages20
JournalMultisensory Research
Volume30
Issue number3-5
DOIs
StatePublished - Jan 1 2017

Fingerprint

White noise
Pitch Discrimination
Modulation
Discrimination (Psychology)

All Science Journal Classification (ASJC) codes

  • Experimental and Cognitive Psychology
  • Ophthalmology
  • Sensory Systems
  • Computer Vision and Pattern Recognition
  • Cognitive Neuroscience

Cite this

@article{dc9f2353c0644c1798070637f86d29d1,
title = "Interactions between Auditory Elevation, Auditory Pitch and Visual Elevation during Multisensory Perception",
abstract = "Cross-modal correspondences refer to associations between apparently unrelated stimulus features in different senses. For example, high and low auditory pitches are associated with high and low visual elevations, respectively. Here we examined how this crossmodal correspondence between visual elevation and auditory pitch relates to auditory elevation. We used audiovisual combinations of high- or low-frequency bursts of white noise and a visual stimulus comprising a white circle. Auditory and visual stimuli could each occur at high or low elevations. These multisensory stimuli could be congruent or incongruent for three correspondence types: cross-modal featural (auditory pitch/visual elevation), within-modal featural (auditory pitch/auditory elevation) and cross-modal spatial (auditory and visual elevation). Participants performed a 2AFC speeded classification (high or low) task while attending to auditory pitch, auditory elevation, or visual elevation. We tested for modulatory interactions between the three correspondence types. Modulatory interactions were absent when discriminating visual elevation. However, the within-modal featural correspondence affected the cross-modal featural correspondence during discrimination of auditory elevation and pitch, while the reverse modulation was observed only during discrimination of auditory pitch. The cross-modal spatial correspondence modulated the other two correspondences only when auditory elevation was being attended, was modulated by the cross-modal featural correspondence only during attention to auditory pitch, and was modulated by the within-modal featural correspondence while performing discrimination of either auditory elevation or pitch. We conclude that the cross-modal correspondence between auditory pitch and visual elevation interacts strongly with auditory elevation.",
author = "Yaseen Jamal and Simon Lacey and Lynne Nygaard and Krishnankutty Sathian",
year = "2017",
month = "1",
day = "1",
doi = "10.1163/22134808-00002553",
language = "English (US)",
volume = "30",
pages = "287--306",
journal = "Multisensory research",
issn = "2213-4794",
publisher = "Brill",
number = "3-5",

}

Interactions between Auditory Elevation, Auditory Pitch and Visual Elevation during Multisensory Perception. / Jamal, Yaseen; Lacey, Simon; Nygaard, Lynne; Sathian, Krishnankutty.

In: Multisensory Research, Vol. 30, No. 3-5, 01.01.2017, p. 287-306.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Interactions between Auditory Elevation, Auditory Pitch and Visual Elevation during Multisensory Perception

AU - Jamal, Yaseen

AU - Lacey, Simon

AU - Nygaard, Lynne

AU - Sathian, Krishnankutty

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Cross-modal correspondences refer to associations between apparently unrelated stimulus features in different senses. For example, high and low auditory pitches are associated with high and low visual elevations, respectively. Here we examined how this crossmodal correspondence between visual elevation and auditory pitch relates to auditory elevation. We used audiovisual combinations of high- or low-frequency bursts of white noise and a visual stimulus comprising a white circle. Auditory and visual stimuli could each occur at high or low elevations. These multisensory stimuli could be congruent or incongruent for three correspondence types: cross-modal featural (auditory pitch/visual elevation), within-modal featural (auditory pitch/auditory elevation) and cross-modal spatial (auditory and visual elevation). Participants performed a 2AFC speeded classification (high or low) task while attending to auditory pitch, auditory elevation, or visual elevation. We tested for modulatory interactions between the three correspondence types. Modulatory interactions were absent when discriminating visual elevation. However, the within-modal featural correspondence affected the cross-modal featural correspondence during discrimination of auditory elevation and pitch, while the reverse modulation was observed only during discrimination of auditory pitch. The cross-modal spatial correspondence modulated the other two correspondences only when auditory elevation was being attended, was modulated by the cross-modal featural correspondence only during attention to auditory pitch, and was modulated by the within-modal featural correspondence while performing discrimination of either auditory elevation or pitch. We conclude that the cross-modal correspondence between auditory pitch and visual elevation interacts strongly with auditory elevation.

AB - Cross-modal correspondences refer to associations between apparently unrelated stimulus features in different senses. For example, high and low auditory pitches are associated with high and low visual elevations, respectively. Here we examined how this crossmodal correspondence between visual elevation and auditory pitch relates to auditory elevation. We used audiovisual combinations of high- or low-frequency bursts of white noise and a visual stimulus comprising a white circle. Auditory and visual stimuli could each occur at high or low elevations. These multisensory stimuli could be congruent or incongruent for three correspondence types: cross-modal featural (auditory pitch/visual elevation), within-modal featural (auditory pitch/auditory elevation) and cross-modal spatial (auditory and visual elevation). Participants performed a 2AFC speeded classification (high or low) task while attending to auditory pitch, auditory elevation, or visual elevation. We tested for modulatory interactions between the three correspondence types. Modulatory interactions were absent when discriminating visual elevation. However, the within-modal featural correspondence affected the cross-modal featural correspondence during discrimination of auditory elevation and pitch, while the reverse modulation was observed only during discrimination of auditory pitch. The cross-modal spatial correspondence modulated the other two correspondences only when auditory elevation was being attended, was modulated by the cross-modal featural correspondence only during attention to auditory pitch, and was modulated by the within-modal featural correspondence while performing discrimination of either auditory elevation or pitch. We conclude that the cross-modal correspondence between auditory pitch and visual elevation interacts strongly with auditory elevation.

UR - http://www.scopus.com/inward/record.url?scp=85020099207&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020099207&partnerID=8YFLogxK

U2 - 10.1163/22134808-00002553

DO - 10.1163/22134808-00002553

M3 - Article

C2 - 31287081

AN - SCOPUS:85020099207

VL - 30

SP - 287

EP - 306

JO - Multisensory research

JF - Multisensory research

SN - 2213-4794

IS - 3-5

ER -