All published articles of this journal are available on ScienceDirect.
Repeated Contextual Search Cues Lead to Reduced BOLD-Onset Times in Early Visual and Left Inferior Frontal Cortex
Abstract
Repetition of context can facilitate search for targets in distractor-filled displays. This contextual cueing goes along with enhanced event-related brain potentials in visual cortex, as previously demonstrated with depth electrodes in the human brain. However, modulation of the BOLD-response in striate and peristriate cortices has, to our knowledge, not yet been reported as a consequence of contextual cueing. Here, we report a selective reduction of the BOLD onset latency for repeated distractor configurations in these areas. In addition, the same onset latency reduction was observed in posterior inferior frontal cortex, a potential source area for feedback signals to early visual areas.
INTRODUCTION
Attending to a particular part of the environment can be guided by regularities in the environment, even if they are not consciously perceived. To exploit such regularities is clearly adaptive, because it enables us to find stimuli more efficiently than by a random spatial search. Repetition of the spatial layout of a scene can facilitate finding the target location. This has consistently been reported in investigations of the contextual cueing effect (first reported by Chun & Jiang [1]). The core finding is that target detection is facilitated if the spatial arrangement of distractor stimuli is repeated across experimental blocks compared to search for the same target in a new distractor arrangement. This effect occurs although participants are unable to recognize the distractor repetition. Thus, implicit memory for spatial locations, gained by incidental learning, increases the speed of target detection in visual search. The cognitive mechanisms and stimulus characteristics leading to contextual cueing have been intensively investigated. To name a few: contextual cueing was found for form and dynamic cueing [2], both absolute position and relative item location can contribute to the effect, which survives also shifts and size changes [3]. The adaptive value of contextual cueing is supported by its long duration; facilitation of search performance by contextual cueing has been observed over periods as long as one week after training [4].
The neural foundations of contextual cueing are still less well investigated than its behavioural aspects. Chun & Phelps [5] reported reduced contextual cueing in a patient with a medial temporal lesion. This was initially unexpected, because implicit learning is typically spared in amnesics with medial temporal lobe pathology. Contextual cueing, as a form of implicit learning, therefore might have been expected not to rely on the medial temporal lobe. Thus, the question of medial temporal and specifically hippocampal involvement dominates the literature on the neural basis of contextual cueing until today [6-8].
However, repetition of search configurations also leads to changes in early visual areas. This was demonstrated in a study using cortical surface electrodes in patients undergoing medical diagnostic procedures for epilepsy treatment [9]. Repeated configurations led to enhanced event-related potentials over anatomically early stations of the visual pathway, including striate cortex (V1). Interestingly, these anatomically early potential changes occurred relatively late, in a time window of 200-400ms after stimulus onset, or 100-300ms after initial activation of V1. Olson and colleagues therefore interpreted their findings as feedback effects, possibly originating from higher visual areas or from the medial temporal lobe. They did not find potential changes related to display repetition from depth electrodes in the hippocampus in the same study, but argue that it may have been difficult to record such changes in their setup.
In the current study, we investigated changes in blood oxygenation level (BOLD) associated with the repetition of distractor configurations in visual search. Using fMRI, we expected to see changes in blood oxygenation in early visual areas, down to V1, in agreement with the electrophysiological findings of Olson et al. In addition, the whole brain coverage afforded by fMRI put us in a position to search for activation changes outside of visual cortex which may indicate brain areas which harbour the source of the top-down signals for the modulation of early visual activation.
A related question is whether repetition of distractor configurations will lead to changes of BOLD amplitude or latency. To our knowledge, only BOLD amplitude changes have so far been investigated in relation to contextual cueing. However, changes in visual search processes may also be accompanied by BOLD latency changes [10, 11]. In particular, it might be assumed that context repetition may lead to speeded processing, mirrored by reduced onset of the BOLD-response.
MATERIALS AND METHODOLOGY
Participants
Sixteen healthy volunteers (21-34 years old, average 25.94, 8 males and 8 females, all right handed) were paid to participate in this study. All had normal or corrected-to-normal vision and were naïve about the purpose of the experiment. They provided informed consent and the protocol was approved by the institution’s ethics committee.
Two participants were excluded from the analysis, due to severe artifacts in the fMRI data. One further subject was excluded from the analysis because she responded to less than 45% of trials.
Stimuli and Apparatus
We used the contextual cueing paradigm introduced by Chun & Jiang ([1], Exp. 1) with some modifications (Fig. 1). Each display contained one target (90° or 270° rotated “T”) and 15 distractors (0°, 90°, 180°, 270° rotated “L”). The orientation of the target was randomly chosen in each trial, so that its stem pointed either to the right or to the left. The color of both target and distractors randomly varied across trials among yellow, red, blue and green. The background was always gray (RGB = 128, 128, 128). Each item subtended 2.3 x 2.3 degrees and displays were generated by randomly placing items on an imaginary 10 x 10 grid that subtended approximately 37.2 x 28.3 degrees of visual angle. The center position of each item was randomly jittered in steps of 0.2 degrees (within a range of ± 0.8 degrees in visual angle along the vertical and horizontal axes) in order to prevent collinearities with other stimuli.
Stimuli and trial structure. In 'old' displays, all distractor locations are repeated, whereas distractor locations are random in 'new' displays. Both old and new displays have a set of target locations, so that target locations are repeated equally in both conditions.
Stimuli were displayed by a DLP-projector on a back-projection screen mounted in the bore of the magnet behind the subject’s head. Participants viewed the screen by means of a mirror positioned on top of the head coil.
Experimental Procedure
The experiment consisted of a single session including training trials, main experiment and a recognition test, with a total duration of approximately 30 minutes. The whole experiment was carried out inside the fMRI scanner, but no images were acquired during the recognition test. In the training session, 32 randomly generated configurations were presented for each subject. The experimental session was framed into 12 blocks of 24 trials, for a total of 288 trials. For each subject, 16 target locations were randomly chosen at the beginning of the experiment. Eight of them, balanced between the four quarters of the screen, were assigned to 8 randomly generated spatial distractor configurations that were preserved during the experiment (“repeated configurations”), while the distractor configurations of the other 8 targets, equally balanced between the four quarters of the screen, were newly generated in each block (“novel configurations”). Additionally, 8 null events were presented in each block, in which the fixation point was presented for the whole duration of the trial.
Each trial lasted 3000 ms (Fig. 1). After 500 ms of fixation (of a white cross in the middle of the screen), the search display was presented for 2500 ms. During this time, the subject had to find and identify the target, by pressing either the left or the right button of the mouse, in accordance with the pointing direction of the stem of the “T”. Participants were told to respond as fast as possible, without sacrificing accuracy, but no feedback was given after the response. At the end of the experiment, participants conducted a recognition test. For this test, the 8 repeated configurations of the learning part of the experiment were sequenced randomly with 8 not previously shown displays, and participants were asked for each display to indicate by an alternative forced choice button press whether or not they had seen the display in the experiment.
fMRI Methods
Data were recorded with a Siemens Trio scanner at a field strength of 3Tesla. An echo-planar imaging sequence was used to acquire functional data, with a repetition time (TR) of 2000 ms, echo time (TE) of 30 ms and flip angle of α = 90 degrees.
The experiment consisted of a single run with 670 volumes, each volume consisting of 32 contiguous transverse slices (field of view (FOV) = 224mmx224mm, matrix = 64x64 pixels, slice thickness = 3.5 mm, gap = 0 mm) which were acquired in interleaved order.
Data were preprocessed and analyzed using BrainVoyager QX (Brain Innovation, Maastricht, The Netherlands). Preprocessing of the functional data included slice time correction using sinc interpolation, head motion correction by spatial alignment of all volumes of each subject to the first volume by rigid body transformation, high pass filtering with frequency cut off of 0.03 Hz and spatial smoothing using a Gaussian filter with FWHM of 6 mm. For each subject, a three-dimensional high-resolution T1-weighted image of the whole brain was also acquired in the same session (192 transverse slices, matrix = 256x256, voxel resolution 1x1x1 mm).
The anatomical data were preprocessed in order to reduce intensity inhomogeneity [12] and then transformed into the Talairach standard space [13]. In order to perform a cortex-based data analysis, a cortex segmentation of the anatomical images was performed [14]. Finally, functional data were coregistered to the corresponding anatomical volume, in order to be transformed into Talairach space.
Regional activations were detected using multi-subject general linear model (GLM) analysis with random effects model. Predictor functions were constructed for each configuration and for each epoch using a classical two-gamma hemodynamic response function. Statistical maps were thresholded at a voxel-level threshold of p=0.005 and a cluster-level threshold of p=0.05 [15].
BEHAVIORAL RESULTS
Response Accuracy
Participants’ accuracy (Table 1) was analyzed using a two-way repeated measures ANOVA, with configuration (novel and repeated) and epoch (1, 3) as factors. Each epoch consisted of 4 blocks. The main effect of configurations (F(1,12) = 3.552, p = 0.084) was not significant but showed a trend towards more accurate responses for repeated displays. The main effect of epoch was significant (F(2,24) = 6.831, p = 0.004), reflecting a general improvement in participants’ performance over time. The interaction between configurations and epochs (F(2,24) = 0.146, p = 0.865) was not significant. When we repeated the ANOVA only on epochs 2 and 3, where a benefit of context repetition would be expected, the main effect of configuration became marginally significant (F(1,12) = 4.721, p = 0.051).
Response Accuracy for Repeated and Novel Configurations as a Function of Epochs
| Epoch1 | Epoch2 | Epoch3 | |
|---|---|---|---|
| Repeated | 80.77% | 85.34% | 88.70% |
| Novel | 78.37% | 79.57% | 85.58% |
Response Time
Fig. (2) shows the mean response times. Error trials were excluded from all response time analyses. Furthermore, response times outside the range of mean±2*standard deviations were removed as outliers. An ANOVA on configuration (novel, repeated) and epoch (1, 3) showed a significant main effect of epoch (F(1,12) = 28.367, p < 0.001) but neither a significant main effect of configuration (p = 0.831) nor a significant interaction (p = 0.577).
Mean response time for repeated and novel configurations as a function of epochs. Error bars indicate standard error of means.
As for accuracy, we carried out an additional ANOVA only on epochs 2 and 3. Neither main effects (configuration: F(1,12) = 0.518, p = 0.485; epoch: F(1,12) = 0.694, p = 0.421) nor the interaction (F(1,12) = 0.001, p = 0.979) were significant.
A paired sample t-test on new-old response times in epoch 3 yielded no significant difference (t(-0.472) = 0.645).
Recognition Test
Binomial tests on recognition judgments (yes and no) and on configurations (novel and repeated) pointed out no significant difference (all p > 0.212). Thus, there was no indication of explicit recognition of repeated displays.
FMRI RESULTS
As for the behavioral data, we calculated an ANOVA on the fMRI data with configuration (old, new) and epoch (1, 3) as within subject factors. The main effect of configuration led to a significant activation in left lateral orbital gyrus, left inferior frontal gyrus, right fusiform gyrus and left cuneus (Table 2). The main effect of epoch was significant in right inferior frontal gyrus and right lingual gyrus. However, the configuration x epoch interaction was of central interest, because contextual cueing predicts changes over time which depend on distractor repetition. The interaction was significant in left inferior precentral gyrus and bilaterally in striate and ventral peristriate (lingual) cortices. The signal amplitude estimates indicated a cross-over interaction in all three areas (Fig. 3), with old displays yielding higher activation than new displays in epoch 1 and the reverse pattern in epoch 3. However, in order to investigate the nature of this interaction further, we extracted the BOLD-signal time courses from the activated areas (Fig. 4). It is immediately apparent, that in all areas, the BOLD signal for old displays in epoch 3 differed from all other conditions by its earlier onset and associated reduced initial undershoot, which was the main reason for its reduced amplitude, rather than a diminished peak amplitude relative to new displays in epoch 3. This impression was confirmed by statistical analysis. Onset was computed using the BOLD latency mapping tool available in BrainVoyager. For each pre-selected region of interest, the BOLD response of each trial of each voxel is fitted using a piece-wise linear (PWL) pseudo-trapezoid fitting, with a time window of 7 seconds (optimized to center the onset). Before the fitting, data are interpolated with a factor of 4, in order to increase the fitting precision.
Table of Activation Clusters. x-z Indicate Talairach Coordinates
| x | y | z | Volume (mm3) | Structure |
|---|---|---|---|---|
| Configuration main effect | ||||
| -25 | 24 | -2 | 214 | orbital gyrus |
| -35 | 11 | -11 | 135 | inferior frontal gyrus |
| 35 | -39 | -14 | 155 | fusiform gyrus |
| 0 | -94 | 8 | 393 | cuneus |
| Epoch main effect | ||||
| 61 | 9 | 22 | 229 | inferior frontal gyrus |
| 5 | -61 | 2 | 308 | lingual gyrus |
| Configuration x epoch interaction | ||||
| -58 | -1 | 25 | 136 | inferior frontal gyrus |
| 6 | -75 | -14 | 361 | lingual gyrus |
| -4 | -78 | -10 | 424 | striate cortex |
Brain areas with a significant configuration x epoch interaction: a) striate cortex, b) lingual gyrus, c) inferior frontal gyrus. Areas are shown on the left with the associated estimated signal amplitudes on the right. The y-axis indicates β-values. Old: repeated distractors, new: random distractors. Left hemisphere is shown on the left.
BOLD-time courses obtained in the three areas shown in Fig. (3). a) striate cortex, b) lingual gyrus, c) inferior frontal gyrus. BOLD time courses are averaged across participants. Time = 0s indicates display onset. Abbreviations: O: old, N: new, 1, 3: epochs.
Paired t-tests on onset latency yielded significant differences between old (epoch 3) – new (epoch 3) for all three areas (IFG: t(12)=-3.994, p=0.002; LG: t(12)=-3.294, 0.006; V1: t(12)=-3.112, p=0.009). In contrast, onset differences for old (epoch 1) – new (epoch 1) were not significantly different in any of the three areas (all p>0.138). Likewise, additional analyses on time to peak and peak amplitude yielded no significant differences which survived the groupwise (3 ROI) corrected significance criterion of α =0.05/3=0.016. All p>0.091, except for a marginally significant difference for old (epoch1) – new (epoch 1) in lingual gyrus (t(12)=2.75, p=0.018). The BOLD-signal differences do not simply reflect search time differences. As reported above, search times were not different between conditions, neither as a whole nor specifically in epoch 3.
DISCUSSION
We found a neural correlate of context repetition in visual search in a reduced onset latency of striate and peristriate cortices as well as left posterior inferior frontal cortex.
These speeded responses were observed in the absence of explicit recognition of repeated targets. It thus cannot be a volitionally controlled response towards stimulus repetition. The fast onset further occurred in the absence of a significant search time difference for repeated and novel displays. It thus does not simply reflect faster search times for repeated displays.
Previous eye-movement studies have shown that contextual cueing leads to an improvement of search which is visible in several gaze parameters. The number of fixations is reduced, the scan pattern becomes more efficient and a final monotonic approach phase starts earlier in repeated than in novel displays [16-18]. The earlier onset for repeated displays appears to be another parameter that signifies some kind of implicit response to the repetition, even though in the present study, this did not (yet) result in faster search times as the endpoint of search.
The absence of a response time advantage for repeated displays may have been due to the short time window our participants had to report target presence. Furthermore, the sample size may have been somewhat small to yield sufficient power to obtain a contextual cueing effect on response times under the taxing conditions of an fMRI experiment. Nevertheless, the advantage gained by context repetition was visible in a marginally significant higher accuracy in epochs 2 and 3.
Modulation of striate and peristriate cortex activity by contextual cueing has, to our knowledge, not been reported before in fMRI studies. However, in a study using cortical surface electrodes in patients undergoing medical diagnostic procedures for epilepsy treatment, enhanced event-related potentials were observed for repeated contexts [9]. These potentials were recorded over anatomically early stations of the visual pathway, including striate cortex. On the other hand, enhancement was found in late potentials, in a time window of 200-400ms after stimulus onset, or 100-300ms after initial activation of V1. Olson and colleagues therefore interpreted their findings as feedback effects, possibly originating from higher visual areas or from the medial temporal lobe. Our selective onset reductions in striate/peristriate cortex for repeated configurations fits well with the localization of the enhanced electrical brain potentials reported by Olson and colleagues.
Most previous work on the neural foundations of contextual cueing was devoted to the role of medial temporal lobe structures [6-8]. The medial temporal has been discussed as a potential source for feedback signals to visual cortex by Olson, Chun and Ellison [9]. These authors, however, did not find ERP-changes for repeated displays in depth electrodes in the medial temporal lobe. Similarly, we did not find an indication of medial temporal structures as possible originating structures for feedback signals to visual cortex. Since negative findings are difficult to interpret, this does not rule out that medial temporal lobe structures are involved in such feedback.
In the present study, the only structure outside the occipital cortex which showed a configuration x epoch-interaction was the posterior inferior frontal gyrus. As in striate and peristriate cortex, repetition led to reduced BOLD-signal onset times. Thus, in principle, this area could be a source of feedback signals to visual areas. Many imaging studies showed left posterior inferior frontal cortex involvement in priming, however in the form of repetition suppression (see reviews by Henson [19] and Schacter and Buckner [20]). The posterior left inferior frontal gyrus was the target of repetitive TMS in a study of repetition priming [21]. It was found that TMS disrupted a categorical living / non-living object classification task. Frontal rTMS did not, however, eliminate repetition suppression in ventral occipitotemporal cortex. The authors concluded that this pattern of results supports a distinction between semantic priming, subserved by inferior frontal cortex and sensory priming, based in posterior visual areas.
These studies differ from ours in many respects, such as awareness of stimulus repetition and semantic processing. The overlap of activation found in priming and the current experiment does not allow the reverse inference that the same mechanisms are involved in contextual cueing and priming [22]. However, future experiments may investigate the mechanisms that lead to repetition suppression versus BOLD-onset reduction in left posterior inferior frontal cortex, thereby elucidating the commonalities or differences between repetition priming and contextual cueing.
ACKNOWLEDGMENT
This study was supported by a grant from the German Research Foundation (DFG, PO 548/6-2).
