All published articles of this journal are available on ScienceDirect.
Activations of Human Auditory Cortex During Visual and Auditory Selective Attention Tasks with Varying Difficulty
Abstract
The present study was designed to directly test the hypothesis that suppression of activations to task-irrelevant sounds contributes to the attention-related modulations of auditory cortex (AC) activations observed in previous fMRI studies. Subjects selectively attended to auditory (broadband noise bursts with pitch) or visual (Gabor gratings) asynchronous fast-rate stimulus streams concurrently presented to left-ear, right-ear, above-fixation, or below-fixation. Auditory and visual task difficulty was parametrically manipulated in three levels. Behavioral data obtained during fMRI indicated that subjects achieved acceptable performance levels in all tasks and that the task-difficulty manipulation was effective. Consistent with previous studies, AC activations strongly depended on the direction of attention. AC activations to sounds were higher during auditory than during visual tasks and AC activations were higher in the hemisphere contralateral to the attended ear. However, the effects of task difficulty on AC activations were weak or non-existent. In particular, increasing task difficulty was not associated with a systematic decrease of AC activations in areas that were modulated by attention. These results suggest that suppression of AC activations to task-irrelevant sounds is likely to be small or negligible as compared with the strong activation enhancements observed in fMRI during active auditory tasks.
INTRODUCTION
Previous functional magnetic resonance imaging (fMRI) studies in humans have shown enhanced auditory cortex (AC) activations to attend sounds as compared with a condition when the same sounds are presented during a visual task [1-6]. Some studies also report that the distribution of AC activations to sounds during auditory and visual attention tasks differ so that the most pronounced auditory attention effects are seen in non-primary parts of AC in the lateral superior temporal gyrus (STG) while AC activations to sounds during visual attention are focused around Heschl’s gyrus (HG) where primary AC is located [2, 3, 6]. Further, AC activations depend on the stimuli and characteristics of the auditory attention tasks [3, 7-11]. For example, AC activations to sounds are enhanced in the hemisphere contralateral to the attended ear as compared with activations in the same hemisphere during ipsilateral attention [12-15]. The functional significance of these modulations is not well understood but it is commonly assumed that the increase of AC activations during auditory tasks is due to activation of some additional processes required by active listening tasks, enhanced representation of task-relevant auditory information, or both. AC activations to task-irrelevant sounds may also be suppressed when attention is directed away from the sounds during, for example, a visual task [5, 13, 16, 17]. However, this idea has not been systematically investigated in fMRI studies. The present study was designed to directly test the hypothesis that suppression of activations to task-irrelevant sounds contributes to the attention-related AC modulations observed in previous fMRI studies.
fMRI data analysis is typically comprised of two-condition comparisons. However, based on a two-condition comparison alone, it is difficult to resolve whether the observed differences are due to increased activations during one condition or decreased activations during the other condition. Ideally, to study AC suppression effects, one would like to measure AC activations to sounds during auditory and, for example, visual attention tasks in relation to a neutral baseline condition when attention is not engaged to sounds or pictures. However, in practice, such neutral baseline is difficult to obtain. When there is no specific task (e.g., ‘rest’ or ‘passive’ condition), it is likely that subjects involuntarily anticipate the next task (beginning in a few seconds) or covertly attend to experimental stimuli or to complex sounds produced by the scanner and its cooling system. Instructing subjects to ignore sounds or to remain passive does not guarantee that no attentional resources are involuntary allocated [18]. If AC activations during a baseline condition are contaminated by uncontrolled auditory attention effects, then it is possible that an illusory suppression is observed when AC activations during visual attention (activations to sounds when they are to be ignored) are compared with AC activations in the baseline condition (enhanced AC activations due to uncontrolled covert auditory attention). Thus, testing the suppression hypothesis is challenging as it is not easy to obtain a neutral strictly-controlled baseline when AC activations are not enhanced by attention to sounds (potentially leading to illusory suppression effects) or suppressed (blotting out suppression-related differences) because attention is directed away from the sounds.
Visual fMRI studies have shown that visual cortex activations to task-irrelevant pictures decrease as the difficulty of task at fixation is increased [18, 19] suggesting that the AC suppression hypothesis could be tested by systematically varying task difficulty in an attention-engaging task. In the present study, task difficulty was parametrically varied in three levels during visual and auditory selective-attention tasks. Subjects were presented with four asynchronous fast-rate stimulus streams throughout the experiment (Fig. 1): Left-ear and right-ear auditory streams consisted of broadband noise bursts. Temporal regularity of the noise bursts was manipulated so that there was a distinct pitch difference between the left and right ear sounds. In the auditory tasks (Auditory Left and Auditory Right), the subjects were required to selectively attend to sounds at the designated ear and to detect frequent (about once per second) pitch increases or decreases (small, medium or large pitch difference depending on the difficulty level) among the attended sounds and to indicate, by pressing one of the two buttons, the direction of the pitch change. The two visual streams, presented below or above fixation, consisted of Gabor gratings with varying orientations. In the visual tasks, the subjects were required to selectively attend to either visual stream, to detect orientation changes (small, medium or large orientation difference depending on the difficulty level) in that stream, and to indicate the direction (clockwise or counterclockwise) of the change. Based on previous studies, it was expected that AC activations to sounds would be higher during auditory attention than during visual attention and that AC activations would be higher in the hemisphere contralateral to the attended ear. Although the effect of task difficulty on AC activations has not been systematically studied in previous fMRI studies, it was assumed that increasing behavioral demands would increase AC activations to sounds presented in the attended ear. To test the suppression hypothesis, AC activations during visual attention and during ipsilateral auditory attention were examined. If increasing task difficulty during these conditions decreases AC activations, then the suppression hypothesis would be supported.
In 15-s blocks (alternating with 15-s baseline with no stimuli), the subjects were presented with broadband noise bursts (gray circles; duration 100 ms) with varying pitch and Gabor gratings with varying orientation in four (left and right ear, above and below fixation) concurrent, independent and asynchronous streams (within stream onset-to-onset interval 200–400 ms). In the auditory tasks, the subjects selectively attended to sounds presented to the left ear and ignored the input to the right ear or vice versa. In the visual tasks, the subjects selectively attended to Gabor gratings presented above fixation and ignored the pictures below fixation or vice versa. Throughout the experiment, subjects focused on a fixation mark which also indicated the current task. For example, an arrowhead pointing up instructed the subjects to attend to Gabor gratings presented above fixation.
MATERIAL AND METHODOLOGY
Subjects
Subjects (N=9, 5 women, all right-handed) were 23–30 years (mean 26 years) of age. All subjects had normal hearing, normal or corrected-to-normal vision, and no history of psychiatric or neurological illnesses. An informed written consent was obtained from each subject before the experiment. The study protocol was approved by the Ethical Committee of the Hospital District of Helsinki and Uusimaa, Finland.
Stimuli and Tasks
Auditory stimuli consisted of bursts of iterated rippled noise (IRN, generated by iteratively adding delayed broadband noise, 3 iterations, duration 100 ms including a 5 ms linear onset and offset ramps). Asynchronous left- and right-ear sequences of IRN sounds were presented throughout all task blocks in the experiment. Within a sequence, sound onset-to-onset intervals varied randomly between 200 and 400 ms. Each sound stream contained either lower (pitch range approximately corresponding to 97–136 Hz, divided in 14 steps), intermediate (208–299 Hz, 14 steps), or higher pitch (466–687 Hz, 14 steps) IRN sounds. In each task block, sounds from one of these pitch groups were delivered to the left ear and sounds from another group to the right ear. In each sound stream, one pitch was repeated 3–6 times after which the pitch increased or decreased (Fig. 1).
Visual streams were constructed in an analogous manner using Gabor gratings (duration 100 ms, onset-to-onset interval 200–400 ms) presented below- and above-fixation (Fig. 1). The orientation of Gabor gratings varied from -60 to 60 degrees (in 14 steps). The onsets of visual and auditory stimuli were asynchronous.
During auditory tasks, subjects were required to selectively attend to sounds at the designated ear (Auditory Left and Auditory Right tasks) in order to detect pitch increases or decreases (1, 2 or 5–8 steps depending on the difficulty level) among the attended sounds and to indicate, by pressing one of the two buttons, the direction of the pitch change. During visual tasks, subjects were required to selectively attend to either visual stream, to detect orientation changes (1, 2 or 5–8 steps depending on the difficulty level) in that stream, and to indicate the direction (clockwise or counterclockwise) of the change.
The sounds and pictures were presented in 15-s blocks (30 Attend Left, 30 Attend Right and 30 Attend Visual blocks for each subject) alternating with 15-s breaks. During the breaks, the subjects focused on a fixation mark (‘x’) presented in the middle of a screen (viewed through a mirror fixed to the head coil) and waited for the next task. The fixation mark was replaced by an arrowhead 2 s before the onset of the next block. An arrowhead pointing to the left, right, up or down instructed the subjects to focus their attention to left or right ear sounds sounds or Gabor gratings presented above or below fixation, respectively.
The auditory stimuli were delivered with an UNIDES ADU2a audio system (Unides Design, Helsinki, Finland) via plastic tubes through a porous EAR-tip (ER3, Etymotic Research) acting as an earphone. The noise of the scanner was attenuated by the earplugs, circumaural ear protectors, and viscoelastic mattress inside and around the headcoil and under the subject. The experiment was controlled using Presentation software (Neurobehavioral Systems, Albany, CA).
The hardest difficulty levels were made intentionally highly demanding in order to reveal differences in brain activity between easy and hard tasks. Before the fMRI session, subjects were carefully trained to perform the demanding tasks (2(25 min of training 1-10 days before fMRI).
Analysis of the Behavioral Data
Mean hit rates (HRs), reaction times (RTs) and relative number of false alarms (FaRs) were calculated separately for each task and difficulty level. Responses (with both buttons) occurring between 200 and 1400 ms from target onset were accepted as hits. HR was defined as the number of hits divided by the number of targets. FaR was defined as the number of false alarms divided by the number of responses. Behavioral results were analyzed using two-way repeated measures ANOVAs with factors Direction of Attention (above/left, below/right) and Task Difficulty (3 levels).
fMRI Data Acquisition and Analysis
fMRI data were acquired with a 3.0 T GE Signa system retrofitted with an Advanced NMR operating console and a quadrature birdcage coil. Functional images were acquired using a T2*-weighted gradient-echo echo-planar (GE-EPI) sequence (TR = 2000 ms, TE = 32 ms, flip angle 90°, voxel matrix 96 x 96, FOV = 20 cm, slice thickness 2.1 mm with no gap, in-plane resolution 2.1 mm x 2.1 mm, number of slices 24). Based on an anatomical scout image (sagittal slices, slice thickness 3 mm, in-plane resolution 0.94 mm x 0.94 mm), the middle EPI slices were aligned along Sylvian fissures to cover the superior temporal lobe, insula and most of the inferior parietal lobe in both hemispheres. The functional scanning was divided in two 23 min runs resulting in approximately 2 × 690 images. After the first run, there was a short break during which subjects remained in the scanner and were instructed not to move their head or speak. After the functional scans, a fluid-attenuated inversion recovery image using the same imaging slices but with denser in-plane resolution was acquired (FLAIR; TR 10000 ms, TE 120 ms, voxel matrix 320 × 192, FOV 20, slice thickness 2.1 mm, inplane resolution 0.39 x 0.39). Finally, at the end of the session, high-resolution anatomical images were acquired (voxel matrix 156 × 256 × 256, resolution 1 mm × 0.98 mm × 0.98 mm).
Global voxel-wise analysis was performed using the tools developed by the Analysis Group at the Oxford Centre for Functional MRI of the Brain (FMRIB) and implemented within FMRIB’s software library (FSL, release 4.1, www.fmrib.ox.ac.uk/fsl,). First, data from the the two runs were combined into one file for motion correction. The motion-corrected data was again splitted into two separate files, high-pass filtered (cutoff 100 s), and spatially smoothed (Gaussian kernel of 7 mm full-width half-maximum). First-level statistical analysis was carried out using FMRIB’s improved linear model. Based on the timing information recorded during the experiment, each functional image was labeled as either attend left (3 difficulty levels), attend right (3 levels), or attend visual (3 levels), or baseline (15-s breaks with no sound stimuli). The hemodynamic response function was modeled with a gamma function (mean lag 6 s, SD 3 s) and its temporal derivative. Contrasts were specified to create Z-statistic images testing for task and difficulty effects. A second-level statistical analysis using fixed-effects combined the data from the two runs.
For analysis across participants (third level analysis), the data were anatomically normalized in the following steps: First, cortical surfaces were extracted from high-resolution anatomical images, transformed to spherical standard space, and anatomically normalized on the basis of the cortical gyral and sulcal patterns using FreeSurfer (v4.0.5, http://surfer.nmr.mgh.harvard.edu). Next, the three dimensional (3D) spherical cortical surfaces were rotated and projected to a two dimensional (2D) space separately for each hemisphere using equal area Mollweide projection (Python libraries matplotlib and basemap, http://matplotlib.sourceforge.net). This procedure produced 3D-to-2D anatomical transformation matrices for each subject that were then applied separately for each subject to transform the results of the 3D second-level statistical analysis to 2D.
Finally, the group analysis (FMRIB’s local analysis of mixed effects using automatic outlier de-weighting, N = 9) was run on these flattened data. Z-statistic images were thresholded using clusters determined by Z > 2.3 and a (corrected) cluster significance threshold of P < 0.05 (using Gaussian random field theory).
RESULTS
Behavior
Subjects achieved acceptable performance levels in all tasks (Fig. 2; mean RT 819 and 943 ms in the visual and auditory tasks, respectively; mean HR 78 and 66%; mean FaR 6 and 5 %). RTs increased (ANOVA, Direction of Attention (above/left, below/right) x Task Difficulty (3 levels), linear contrast (Task Difficulty), visual task: F(1,8)=37, P<0.001; auditory task: F(1,8)=47, P<0.001) and HRs decreased (visual task: F(1,8)=48, P<0.001; auditory task: F(1,8)=67, P<0.0001) with increasing task difficulty.
Performance in the visual and auditory tasks as a function of task difficulty level (L1–L3).
fMRI
AC activations to sounds in the absence of auditory attention were isolated by contrasting activations during the visual tasks with activations during the 15-s breaks with no stimuli. In both hemispheres, HG, areas posterior to HG (planum temporale), anterior insula and, in addition, areas in the right IPL were activated by presentation of sounds during visual tasks (Fig. 3b, blue and yellow; for anatomical labels, see Fig. 3g). General effects of auditory tasks were isolated by contrasting AC activations during auditory and visual tasks. AC activations were enhanced during auditory as compared with visual attention task in wide AC regions extending from the anterior to posterior STG and including HG (Fig. 3a, red and yellow). Auditory Left (Fig. 3b, blue and yellow) and Auditory Right (red and yellow) tasks activated (vs. visual tasks) similar areas (for individual subject data see Supplementary Fig. F1). However, left AC activations were higher during Auditory Right than during Auditory Left task (Fig. 3c, red). Correspondingly, right AC activations were enhanced during contralateral attention (blue).
(a, f) Activations shown on flattened mean 2D cortical surface (N = 9, threshold Z > 2.3, cluster-corrected P < 0.05 unless otherwise specified). (a) Activations to sounds in the absence of auditory attention (blue) were isolated by contrasting activations during visual tasks with activations during the 15-s breaks with no sounds. General effects of auditory tasks were isolated by contrasting all auditory tasks with visual tasks (red). Areas showing significant activations in both contrasts are shown in yellow. (b) Activations specific to Auditory Left (blue) and Auditory Right (red) tasks were extracted by comparing each auditory task (all difficulty levels) with activations during the visual task. For individual subject data, see Supplementary Fig F1. (c) Areas where activations were higher during the Auditory Left than Right (blue) and during the Auditory Right than Left (red). (d) Results of linear inverse contrast revealing areas where activations decreased with increasing visual task difficulty (threshold Z > 1.64 corresponding to uncorrected P = 0.05). For individual subject data, see Supplementary Fig F1. (e) Effects of task difficulty (linear contrasts) during Auditory Left (blue) and Auditory Right (red; threshold Z > 1.64 corresponding to uncorrected P = 0.05). (f) Areas where activations decreased (inverse linear contrasts) with increasing task difficulty during Auditory Left (blue) and Auditory Right (red). (g) Anatomical labels. STG superior temporal gyrus, HG Heschl’s gyrus, IPL inferior parietal lobule (consisting of angular gyrus and supramarginal gyrus).
Visual task difficulty did not significantly modulate AC activations. Only with a more lenient threshold (Z > 1.64, corresponding to uncorrected P< 0.05), clusters where activations decreased with increasing visual task demands were detected (Fig. 3d, blue; for individual subject data see Supplementary Fig. F1). These clusters were dominantly in the left hemisphere.
Increasing auditory task-difficulty did not significantly increase AC activations. Even with a more lenient threshold (Z > 1.64, corresponding to uncorrected P < 0.05), no substantial activation clusters were detected (Fig. 3e). However, during Auditory Right task (Fig. 3f, red and yellow) activations decreased with increasing task demands in areas of the anterior and posterior STG and during Auditory Left task (Fig. 3f, blue and yellow) in posterior STG.
DISCUSSION
The present study was designed to test whether AC activations to task-irrelevant sounds are suppressed during intermodal (auditory vs. visual tasks) and intramodal (auditory left vs. right) selective attention tasks in such a way that this suppression significantly contributes to attention-related modulations observed with fMRI. While most previous fMRI studies have shown enhanced AC activations to sounds during auditory tasks, some studies have also reported decreased AC activations when the sounds are to be ignored. However, it is not known whether such suppression of AC activations is a major or minor component of the attention-related modulations. The present study aimed first to demonstrate reliable intermodal and intramodal attention effects and, then, to examine whether suppression of AC activations during visual (intermodal attention) and ipsilateral (intramodal attention) auditory tasks is substantial enough to significantly contribute to these effects. As suppression effects and activation increases are difficult to disentangle from each other using two-condition comparisons, the present study used an experimental design in which task difficulty was parametrically varied: it was assumed that if suppression significantly contributes to attention-related modulations, then AC activations to task-irrelevant sounds should decrease when task difficulty is increased.
Behavioral data obtained during fMRI verified that the task and task-difficulty manipulations were effective: In both visual and auditory tasks, RTs increased and HRs decreased with increasing task demands. As expected, AC activations to sounds were higher during auditory than during visual tasks in areas extending from the anterior to posterior STG including HG in both hemispheres (Fig. 3a red and yellow; Fig. 3b and Supplementary Fig. F1). During intramodal auditory selective attention, AC activations were higher during contralateral than ipsilateral auditory attention in areas adjacent to HG (Fig. 3c). In general, the effects of task difficulty on AC activations were weak or non-existent (Fig. 3d-3f). In some AC areas activations tended to decrease with increasing visual task difficulty (Fig. 3d). However, these effects, most notable in the left anterior insula and left STG, do not match well with the observed robust AC attention effects (Fig. 3a red and yellow, see also Supplementary Fig. F1). During intramodal auditory selective attention, activations in some STG areas decreased with increasing task demands (Fig. 3f). However, this decrease did not systematically occur ipsilaterally to the attended ear and, therefore it cannot explain the enhanced AC activations observed during contralateral attention (Fig. 3c). Thus, although distinct intermodal and intramodal attention effects were detected, there was no systematic evidence for the hypothesis that suppression of AC activations to task-irrelevant sounds contributes to the attention-related modulations observed in the present and previous fMRI studies. Note that the present study was carefully designed to yield strong and reliable attention-related modulations. AC activation enhancements were reliably detected during both intermodal and intramodal attention conditions although intramodal (contralateral) attention effects are often more challenging to observe. Thus, it could be argued that the present study would have been able to detect suppression of AC activations to task-irrelevant sounds, if such suppression constitutes a major component of attention-related modulations. However, the present results suggest that suppression effects are small as compared with the strong attention-related modulations. Thus, it appears that the attention effects observed in the present and previous studies are mainly due to task-dependent activation enhancements and not due to suppression.
The present results seem in contrast with previous fMRI studies showing that visual-cortex activations to task-irrelevant visual stimuli are suppressed and decrease further with increasing difficulty of the relevant task at fixation [18, 19]. The retinotopic organization of the visual cortex makes it relatively easy to investigate activation differences between cortical areas processing stimuli in different locations of the visual field while in AC such topographic differences are less distinct. Previous fMRI studies have been able to reveal the tonotopic organization of different AC areas [6, 20, 21] but it appears more difficult to detect attention-related modulations of frequency-specific activations [6]. Thus, although neurophysiologic studies in animals have shown that attention to one frequency enhances activation in tonotopically organized AC neurons preferring that frequency and suppresses activations in the adjacent neurons processing slightly different frequencies [22], the effect of such local suppression in topographically organized AC areas is likely to be negligible as compared with the strong and wide-spread attention-related AC modulations that are easily observed in fMRI.
Auditory and visual attention systems may also differ in their functional roles in behavior. For example, unlike vision, the auditory system receives and processes sensory information from all directions (including behind the head) and operates also in darkness and during sleep. It could be argued that, among other functions, the auditory attention system serves as a primary warning system detecting potentially significant events in the unattended environment. This idea is supported, for example, by the vast literature on the mismatch negativity (MMN) [23], an event-related potential (ERP) mainly generated in the bilateral AC, showing that even during highly demanding visual attention tasks [24-26] and sleep [27, 28] task-irrelevant auditory information is processed at least to the level where changes in the auditory environment can be detected. Thus, the contrast between previous fMRI studies showing suppression of visual-cortex activations to task-irrelevant stimuli and the present study finding no such effects in AC could be due to the more distinct topographical organization of the visual cortex and functional differences between visual and auditory attention systems.
Our recent study compared activations during two different auditory tasks, pitch memory and pitch discrimination, that were varied in difficulty [3]. In general, we found that activations in the anterior STG increased during pitch discrimination but not during pitch memory tasks while IPL activations were increased during pitch memory tasks but not during pitch discrimination. Interestingly, we also found that as the difficulty of the pitch memory task increased activations in anterior AC areas decreased. This effect was not observed in the pitch discrimination task. We suggested that the decrease of anterior AC activations during the pitch memory tasks requiring only rudimentary pitch information was due to interruption of detailed pitch analysis that occurs as default for each incoming sound. Perhaps in the present study such AC suppression effects were small because the processing of task-relevant sounds during the present pitch discrimination task did not considerably differ from the default-mode processing of task-irrelevant sounds.
CONCLUSION
The present results suggest that attention-related modulations of AC activations observed in the present and previous fMRI studies are mainly due to increase of activations during the active listening task. Typically, suppression of AC activations to task-irrelevant sounds is likely to be small or negligible as compared with activation enhancements during active auditory tasks. However, strong AC deactivations may be observed in certain cases during demanding auditory tasks with conflicting processing requirements [3].
SUPPLEMENTARY MATERIAL
Supplementary material can be viewed at Download Supplementary Material
ACKNOWLEDGMENTS
This work was supported by the Academy of Finland (grants #210587 and #135900) and Research Funds of the University of Helsinki. Special thanks to Ms. Suvi Talja for her help during data acquisition and in programming data analysis tools.
