Sound symbolism refers to non‐arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as “maluma” and “takete” with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity (...) analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics‐to‐noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound‐to‐shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross‐modal correspondence as a basis for language users' sensitivity to this type of sound symbolism. (shrink)

The auditory system is able to recognize auditory objects and is thought to form predictive models of them even though the acoustic information arriving at our ears is often imperfect, intermixed, or distorted. We investigated implicit regularity extraction for acoustically intact versus disrupted six-tone sound patterns via event-related potentials. In an exact-repetition condition, identical patterns were repeated; in two distorted-repetition conditions, one randomly chosen segment in each sound pattern was replaced either by white noise or by a wrong (...) pitch. In a roving-standard paradigm, sound patterns were repeated 1–12 times in a row before a new pattern occurred. The participants were not informed about the roving rule and had to detect rarely occurring loudness changes. Behavioral detectability of pattern changes was assessed in a subsequent behavioral task. Pattern changes elicited mismatch negativity and P3a, and were behaviorally detected above the chance level in all conditions, suggesting that the auditory system extracts regularities despite distortions in the acoustic input. However, MMN and P3a amplitude were decreased by distortions. At the level of MMN, both types of distortions caused similar impairments, suggesting that auditory regularity extraction is largely determined by the stimulus statistics of matching information. At the level of P3a, wrong-pitch distortions caused larger decreases than white-noise distortions. Wrong-pitch distortions likely prevented the engagement of restoration mechanisms and the segregation of disrupted from true pattern segments, causing stronger informational interference with the relevant pattern information. (shrink)

Asymmetric distribution of function between the cerebral hemispheres has been widely investigated in the auditory modality. The current approach borrows heavily from visual local–global research in an attempt to determine whether, as in vision, local–global auditory processing is lateralised. In vision, lateralised local–global processing likely relies on spatial frequency information. Drawing analogies between visual spatial frequency and auditory dimensions, two sets of auditory stimuli were developed. In the high–low stimulus set we manipulate frequency information, and (...) in the fast–slow stimulus set we manipulate temporal information. The fast–slow stimuli additionally mimic visual hierarchical stimulus structure, in which the arrangement of local patterns determines the global pattern. Unlike previous auditory stimuli, the current stimulus sets contain the experimental flexibility of visual local–global hierarchical stimuli allowing independent manipulation of structural levels. Previous findings of frequency and temporal range priming were replicated. Additionally, by presenting stimuli monaurally, we found that priming of frequency ranges (but not temporal ranges) was found to vary by ear, supporting the contention that the hemispheres asymmetrically retain traces of prior frequency processing. These results contribute to the extensive literature revealing cerebral asymmetries for the processing of frequency information, and extend those results to the realm of priming. (shrink)

Background. Attending to goal-relevant information can leave us metaphorically ‘blind’ or ‘deaf’ to the next relevant information while searching among distracters. This temporal cost lasting for about a half a second on the human selective attention has been long explored using the attentional blink paradigm. Although there is evidence that certain visual stimuli relating to one’s area of expertise can be less susceptible to attentional blink effects, it remains unexplored whether the dynamics of temporal selective attention vary with expertise and (...) objects types in the auditory modality. Methods. Using the auditory version of the attentional blink paradigm, the present study investigates whether certain auditory objects relating to musical and perceptual expertise could have an impact on the transient costs of selective attention. In this study, expert cellists and novice participants were asked to first identify a target sound, and then to detect instrumental timbres of cello or organ, or human voice as a second target in a rapid auditory stream. Results. The results showed moderate evidence against the attentional blink effect for voices independent of participants’ musical expertise. Experts outperformed novices in their overall accuracy levels of target identification and detection, reflecting a clear benefit of musical expertise. Importanly, the musicianship advantage disappeared when the human voices served as the second target in the stream. Discussion. The results are discussed in terms of stimulus salience, the advantage of voice processing, as well as perceptual and musical expertise in relation to attention and working memory performances. (shrink)

Priming is a useful tool for ascertaining the circumstances under which previous experiences influence behavior. Previously, using hierarchical stimuli, we demonstrated (Justus & List, 2005) that selectively attending to one temporal scale of an auditory stimulus improved subsequent attention to a repeated (vs. changed) temporal scale; that is, we demonstrated intertrial auditory temporal level priming. Here, we have extended those results to address whether level priming relied on absolute or relative temporal information. Both relative and absolute temporal (...) information are important in auditory perception: Speech and music can be recognized over various temporal scales but become uninterpretable to a listener when presented too quickly or slowly. We first confirmed that temporal level priming generalized over new temporal scales. Second, in the context of multiple temporal scales, we found that temporal level priming operates predominantly on the basis of relative, rather than absolute, temporal information. These findings are discussed in the context of expectancies and relational invariance in audition. (shrink)

Attention is often conceived as a gateway to consciousness . Although endogenous spatial attention may be independent of conscious perception , exogenous spatial orienting seems instead to be an important modulator of CP . Here, we investigate the role of auditory alerting in CP in normal observers. We used a behavioral task in which phasic alerting tones were presented either at unpredictable or at predictable time intervals prior to the occurrence of a near-threshold visual target. We find, for the (...) first time in neurologically intact observers, that phasic alertness increases CP, both objectively and subjectively. This result is consistent with evidence showing that phasic alerting can ameliorate the spatial bias exhibited by visual neglect patients . The alerting network may increase the activity of fronto-parietal networks involved in top-down amplification required to bring a stimulus into consciousness. (shrink)

People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and speech stimuli. Moreover, participants were more tolerant of AVOA of speech (...) stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of “synchronous” responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception. (shrink)

The Temporal-Order Judgment (TOJ) paradigm has been widely investigated in previous studies as an accurate measure of temporal resolution and sequencing abilities in the millisecond time range. Two auditory TOJ tasks are often used: (1) a spatial TOJ task, in which two identical stimuli are presented in rapid succession monaurally and the task is to indicate which ear received the first stimulus and which ear received the second one ( left-right or right-left ), and (2) a spectral TOJ (...) task, in which two tones of different frequencies are presented asynchronously to both ears binaurally and the task is to report the sequence of these tones ( low-high or high-low ). The previous literature studies conducted on young volunteers indicated that the measured temporal acuity on these two tasks depended on the procedure used. As considerable data are now available about age-related decline in temporal resolution ability, the aim of the present study was to compare in elderly subjects the pattern of performance on these two tasks. A total of 40 normal healthy volunteers aged from 62 to 78 years performed two TOJ tasks. The measurement was repeated in two consecutive sessions. Temporal resolution was indexed by the Auditory Temporal-Order Threshold (ATOT), i.e., the minimum time gap between successive stimuli necessary for a participant to report a before-after relation with 75% correctness. The main finding of the present study was the indication of differences in the elderly in performance on two tasks. In the spatial task, the distribution of obtained ATOT values did not deviate from the Gaussian distribution. In contrast, the distribution of data in the spectral task deviated significantly from the Gaussian and was spread more to the right. Although lower ATOT values were usually observed in Session 2 than in Session 1, such difference was significant only in the spectral task. We conclude that although temporal acuity and sequencing abilities in the millisecond time range are probably based in neuronal oscillatory activity, the measured ATOTs in the elderly seem to be stimulus-dependent, procedure-related, and influenced by the perceptual strategies used by participants. (shrink)

Previous reports have demonstrated that the comprehension of sentences describing motion in a particular direction (toward, away, up, or down) is affected by concurrently viewing a stimulus that depicts motion in the same or opposite direction. We report 3 experiments that extend our understanding of the relation between perception and language processing in 2 ways. First, whereas most previous studies of the relation between perception and language processing have focused on visual perception, our data show that sentence processing can (...) be affected by the concurrent processing of auditory stimuli. Second, it is shown that the relation between the processing of auditory stimuli and the processing of sentences depends on whether the sentences are presented in the auditory or visual modality. (shrink)

Mismatch negativity is the electroencephalographic waveform obtained by subtracting event-related potential responses evoked by unexpected deviant stimuli from responses evoked by expected standard stimuli. While the MMN is thought to reflect an unexpected change in an ongoing, predictable stimulus, it is unknown whether MMN responses evoked by changes in different stimulus features have different magnitudes, latencies, and topographies. The present study aimed to investigate whether MMN responses differ depending on whether sudden stimulus change occur in pitch, duration, (...) location or vowel identity, respectively. To calculate ERPs to standard and deviant stimuli, EEG signals were recorded in normal-hearing participants who listened to roving oddball sequences of artificial syllables. In the roving paradigm, any given stimulus is repeated several times to form a standard, and then suddenly replaced with a deviant stimulus which differs from the standard. Here, deviants differed from preceding standards along one of four features. The feature levels were individually chosen to match behavioral discrimination performance. We identified neural activity evoked by unexpected violations along all four acoustic dimensions. Evoked responses to deviant stimuli increased in amplitude relative to the responses to standard stimuli. A univariate analysis yielded no significant differences between MMN responses following violations of different features. However, in a multivariate analysis, acoustic features could be decoded from the topography of mismatch responses, although at later latencies than those typical for MMN. These results support the notion that deviant feature detection may be subserved by a different process than general mismatch detection. (shrink)

Though dividing one’s attention between two input streams typically impairs performance, detecting a behaviorally relevant stimulus can sometimes enhance the encoding of unrelated information presented at the same time. Previous research has shown that selection of this kind boosts visual cortical activity and memory for concurrent items. An important unanswered question is whether such effects are reflected in processing quality and functional connectivity in visual regions and in the hippocampus. In this fMRI study, participants were asked to memorize a (...) stream of naturalistic images and press a button only when they heard a predefined target tone. Images could be presented with a target tone, with a distractor tone, or without a tone. Auditory target detection increased activity throughout the ventral visual cortex but lowered it in the hippocampus. Enhancements in functional connectivity between the ventral visual cortex and the hippocampus were also observed following auditory targets. Multi-voxel pattern classification of image category was more accurate on target tone trials than on distractor and no tone trials in the fusiform gyrus and parahippocampal gyrus. This effect was stronger in visual cortical clusters whose activity was more correlated with the hippocampus on target tone than on distractor tone trials. In agreement with accounts suggesting that subcortical noradrenergic influences play a role in the attentional boost effect, auditory target detection also caused an increase in locus coeruleus activity and phasic pupil responses. These findings outline a network of cortical and subcortical regions that are involved in the selection and processing of information presented at behaviorally relevant moments. (shrink)

Many situations require the simultaneous processing of auditory and visual information, however, stimuli presented to one sensory modality can sometimes interfere with processing in a second sensory modality (i.e., modality dominance). The current study further investigated modality dominance by examining how task demands and bimodal presentation affect speeded auditory and visual discriminations. Participants in the current study had to quickly determine if two words, two pictures, or two word-picture pairings were the same or different, and we manipulated task (...) demands across three different conditions. In an immediate recognition task, there was only one second between the two stimuli/stimulus pairs and auditory dominance was found. Compared to the respective unimodal baselines, pairing pictures and words together slowed down visual responses and sped up auditory responses. Increasing the interstimulus interval to four seconds and blocking verbal rehearsal weakened auditory dominance effects, however, conflicting and redundant visual cues sped up auditory discriminations. Thus, simultaneously presenting pictures and words had different effects on auditory and visual processing, with bimodal presentation slowing down visual processing and speeding up auditory processing. These findings are consistent with a proposed mechanism underlying auditory dominance, which posits that auditory stimuli automatically grab attention and attenuate/delay visual processing. (shrink)

The quality of the animal-human relationship and, consequently, the welfare of animals can be improved by gentle interactions such as stroking and talking. The perception of different stimuli during these interactions likely plays a key role in their emotional experience, but studies are scarce. During experiments, the standardization of verbal stimuli could be increased by using a recording. However, the use of a playback might influence the perception differently than ‘live’ talking, which is closer to on-farm practice. Thus, we compared (...) heifers’ (n = 28) reactions to stroking while an experimenter was talking soothingly (‘live’) or while a recording of the experimenter talking soothingly was played (‘playback’). Each animal was tested three times per condition and each trial comprised three phases: pre-stimulus, stimulus (stroking and talking) and post-stimulus. In both conditions, similar phrases with positive content were spoken calmly, using long low-pitched vowels. All tests were video recorded and analyzed for behaviors associated with different affective states. Effects on the heifers’ cardiac parameters were assessed using analysis of heart rate variability. Independently of the auditory stimuli, longer durations of neck stretching occurred during stroking, supporting our hypothesis of a positive perception of stroking. Observation of ear positions revealed longer durations of the back up position and less ear flicking and changes of ear positions during stroking. The predicted decrease in HR during stroking was not confirmed; instead we found a slightly increased mean HR during stroking with a subsequent decrease in HR, which was stronger after stroking with live talking. In combination with differences in HRV parameters, our findings suggest that live talking might have been more pleasurable to the animals and had a stronger relaxing effect than ‘playback’. The results regarding the effects of the degree of standardization of the stimulus on the variability of the data were inconclusive. We thus conclude that the use of recorded auditory stimuli to promote positive affective states during human-animal interactions in experimental settings is possible, but not necessarily preferable. (shrink)

Spatial region concepts such as “front,” “back,” “left,” and “right” reflect our typical interaction with space, and the corresponding surrounding regions have different statuses in memory. We examined the representation of spatial directions in the auditory space, specifically in how far natural response actions, such as orientation movements toward a sound source, would affect the categorization of egocentric auditory space. While standing in the middle of a circle with 16 loudspeakers, participants were presented acoustic stimuli coming from the (...) loudspeakers in randomized order, and verbally described their directions by using the concept labels “front,” “back,” “left,” “right,” “front-right,” “front-left,” “back-right,” and “back-left.” Response actions varied in three blocked conditions: (1) facing front, (2) turning the head and upper body to face the stimulus, and (3) turning the head and upper body plus pointing with the hand and outstretched arm toward the stimulus. In addition to a protocol of the verbal utterances, motion capture and video recording generated a detailed corpus for subsequent analysis of the participants’ behavior. Chi-square tests revealed an effect of response condition for directions within the left and right sides. We conclude that movement-based response actions influence the representation of auditory space, especially within the sides’ regions. Moreover, the representation of auditory space favors the front and the back regions in terms of resolution, which is possibly related to the physiological characteristics of the human auditory system, as well as to the ecological requirements of action control in the different regions. (shrink)

Numerous studies have shown that stimulus-response-compatibility (SRC) effects in the go-nogo version of the Simon task can be elicited as a result of performing the task together with another human or non-human agent (e.g., a Japanese-waving-cat, a working-clock, or a ticking-metronome). A parsimonious explanation for both social and non-social SRC effects is that highlighting the spatial significance of alternative (non-/social) action events makes action selection more difficult. This holds even when action events are task-irrelevant. Recent findings, however, suggest that (...) this explanation holds only for cases of a modality correspondence between the Simon task as such (i.e., auditory or visual) and the alternative (non-/social) action event that needs to be discriminated. However, based on the fact that perception and action are represented by the same kind of codes, an event that makes the go-nogo decision more challenging should impact go-nogo Simon task performance. To tackle this issue, the present study tested if alternative stimulus events that come from a different sensory modality do impact SRC effects in the go-nogo version of the Simon task. This was tested in the presence and absence of alternative action events of a human co-actor. In a multimodal (auditory–visual) go-nogo Simon paradigm, participants responded to their assigned stimulus – e.g., a single auditory stimulus while ignoring the alternative visual stimulus or vice versa – in the presence or absence of a human co-actor (i.e., joint and single go-nogo condition). Results showed reliable SRCs in both, single and joint go-nogo Simon task conditions independent of the modality participants had to respond to. Although a correspondence between stimulus material and attention-grabbing event might be an efficient condition for SRCs to emerge, the driving force underlying the emergence of SRCs rather appears to be whether the attentional focus prevents or facilitates alternative events to be integrated. Thus, under task conditions in which the attentional focus is sufficiently broad to enable the integration and thus cognitive representation of alternative events, go-nogo decisions become more difficult, resulting in reliable SRCs in single and joint go-nogo Simon tasks. (shrink)

Pitch labeling in absolute pitch (AP), the ability to recognize the pitch class of a sound without an external reference, is effortless, fast, and presumably automatic. Previous studies have shown that pitch labeling in AP can interfere with task demands. In the current study, we used a cued auditory Go/Nogo task requiring same/different decisions to investigate both behavioral and electrophysiological correlates of increased inhibitory demands related to automatic pitch labeling. The task comprised two Nogo conditions: a Nogo condition with (...) pitch differences larger than one semitone, and a second Nogo condition with pitch differences of only a quarter semitone. The first Nogo condition tested if auditory-related inhibition processes are generally altered in AP musicians. The second Nogo condition tested the suppressibility of the pitch labeling using a Stroop-like effect: the two tones belonged to the same pitch class but were not identical in terms of tone frequency. If pitch labeling cannot be suppressed, the conflicting information would be expected to increase the inhibitory load in AP musicians. Our data provided no evidence for an increased difficulty to inhibit a prepotent response or to suppress conflicting pitch-labeling information in AP: AP musicians showed similar commission error rates as non-AP musicians in both Nogo conditions. N2d and P3d amplitudes of AP musicians were also comparable to those of non-AP musicians. The event-related potentials (ERPs) were, however, modulated by the Nogo condition, probably indicating an effect of stimulus similarity. It is possible that, depending on the context, pitch labeling in AP musicians is not entirely automatic and can be suppressed. (shrink)

Voluntary stream segregation was investigated in cochlear implant users and normal-hearing listeners using a segregation-promoting objective approach which evaluated the role of spectral and amplitude-modulation rate separations on stream segregation and its build-up. Sequences of 9 or 3 pairs of A and B narrowband noise bursts were presented which differed in either center frequency of the noise band, the AM-rate, or both. In some sequences, the last B burst was delayed by 35 ms from their otherwise-steady temporal position. In the (...) other sequences, the last B bursts were temporally advanced from 0 to 10 ms. A single interval yes/no procedure was utilized to measure participants’ sensitivity in identifying delayed vs. no-delay sequences. A higher d′ value showed the higher ability to segregate the A and B subsequences. For NH listeners, performance improved with each spectral separation. However, for CI users, performance was only significantly better for the condition with the largest spectral separation. Additionally, performance was significantly poorer for the largest AM-rate separation than for the condition with no AM-rate separation for both groups. The significant effect of sequence duration in both groups indicated that listeners made more improvement with lengthening the duration of stimulus sequences, supporting the build-up effect. The results of this study suggest that CI users are less able than NH listeners to segregate NBN bursts into different auditory streams when they are moderately separated in the spectral domain. Contrary to our hypothesis, our results indicate that AM-rate separation may interfere with the segregation of streams of NBN. Additionally, our results add evidence to the literature that CI users build up stream segregation at a rate comparable to NH listeners, when the inter-stream spectral separations are adequately large. (shrink)