Experiencing a stimulus in one sensory modality is often associated with an experience in another sensory modality. For instance, seeing a lemon might produce a sensation of sourness. This might indicate some kind of cross-modal correspondence between vision and gustation. The aim of the current study was to provide explore whether such cross-modal correspondences influence cross-modal integration during perceptual learning. To that end, we conducted 2 experiments. Using a speeded classification task, Experiment 1 (...) established a cross-modal correspondence between visual lightness and the frequency of an auditory tone. Using a short-term priming procedure, Experiment 2 showed that manipulation of such cross-modal correspondences led to the creation of a crossmodal unit regardless of the nature of the correspondence (i.e., congruent, Experiment 2a or incongruent, Experiment 2b). However, a comparison of priming-effects sizes suggested that cross-modal correspondences modulate cross-modal integration during learning and thus leading to new learned units that have different stability over time. We discuss the implications of our results for the relation between cross-modal correspondence and perceptual learning in the context of a Bayesian explanation of cross-modal correspondences. (shrink)

Recent studies have provided evidence of associations between neurochemistry and reading (dis)ability (Pugh et al., 2014). Based on a long history of studies indicating that fluent reading entails the automatic convergence of the written and spoken forms of language and our recently proposed Neural Noise Hypothesis (Hancock et al., 2017), we hypothesized that individual differences in cross-modal integration would mediate, at least partially, the relationship between neurochemical concentrations and reading. Cross-modal integration was measured in (...) 231 children using a two-alternative forced choice cross-modal matching task with three language conditions (letters, words, and pseudowords) and two levels of difficulty within each language condition. Neurometabolite concentrations of Choline (Cho), Glutamate (Glu), gamma-Aminobutyric (GABA), and N- acetyl-aspartate (NAA) were then measured in a subset of this sample (n = 70) with Magnetic Resonance Spectroscopy (MRS). A structural equation mediation model revealed that the effect of cross-modal word matching mediated the relationship between increased Glu (which has been proposed to be an index of neural noise) and poorer reading ability. In addition, the effect of cross-modal word matching fully mediated a relationship between increased Cho and poorer reading ability. Multilevel mixed effects models confirmed that lower Cho predicted faster cross-modal matching reaction time, specifically in the hard word condition. These Cho... (shrink)

The thesis of this paper is that the capacity to think of one’s perceptions as cross-modally integrated is incompatible with a reductionist account of the self. In §2 I distinguish three versions of the argument from cross-modality. According to the ‘unification’ version of the argument, what needs to be explained is one’s capacity to identify an object touched as the same as an object simultaneously seen. According to the ‘recognition’ version, what needs to be explained is one’s capacity, (...) having once seen an object, to reidentify that same object by touch alone. According to the ‘objectivity’ version, what needs to be explained is one’s capacity to think of one’s perceptions in different modalities as perceptions of one and the same object. The third version seems to establish that one must conceive of oneself substantially, as the numerically identical owner of one’s experiences, a conclusion in agreement with recent work in developmental psychology claiming to show that an infant’s cross-modal capacities are essentially implicated in their development of a sense of self. There is further work to be done if this is to be tumed into an argument against reductionism: there is no swift route from the epistemology of self-consciousness to the metaphysics of the self. In the §3, I will claim that there is, nevertheless, an argument linking the two. What I propose is an argument derived, not from the token-reflexive rule for the first-person, but resting on its anaphoric behaviour spanning intensional operators. (shrink)

In the present study, cross-modal (i.e., across sensory modalities such as smell and sound) iconicity (i.e., resemblance) and indexicality (i.e., contiguity) in lexical sensory and emotional signs in Finnish Sign Language will be considered from an articulatory perspective (i.e., the production of signs). Such cross-modal iconicity has not been extensively studied previously, so here, with the help of cognitive semiotics, I aim to carefully describe the cross-modal patterns observed across 118 signs, including 60 sensory (...) signs and 58 emotional signs. The analysis is framed within the theoretical model of Semiotic Hierarchy, which entails a non-reductionist view of meaning. In addition, a pheno-methodological triangulation will be applied: phenomenology (first-person method), literature of phenomenological and semiotic descriptions (second-person perspective) and experimental findings (third-person perspective). The results of this analysis show that (a) 71 of the 118 sensory and emotional signs are cross-modally indexical, (b) only 10 of the 71 signs can be regarded as cross-modally iconic, (c) cross-modal iconicity is highly diagrammatic, (d) iconicity and indexicality are highly integrated, and (e) articulatory feedback matters in the formation of semiotic patterns. This study contributes to our understanding of cross-modal iconicity in signed languages, as well as studies in semiotic systems more generally. (shrink)

Three priming experiments were conducted to determine how information about the self from different sensory modalities/cognitive domains affects self-face recognition. Being exposed to your body odor, seeing your name, and hearing your name all facilitated self-face recognition in a reaction time task. No similar cross-modal facilitation was found among stimuli from familiar or novel individuals. The finding of a left-hand advantage for self-face recognition was replicated when no primes were presented. These data, along with other recent results suggest (...) the brain processes/represents information about the self in highly integrated ways. (shrink)

We investigate the dynamics of sensory integration for perceiving musical performance, a complex natural behavior. Thirty musically trained participants saw, heard, or both saw and heard, performances by two clarinetists. All participants used a sliding potentiometer to make continuous judgments of tension (a measure correlated with emotional response) and continuous judgments of phrasing (a measure correlated with perceived musical structure) as performances were presented. The data analysis sought to reveal relations between the sensory modalities (vision and audition) and to (...) quantify the effect of seeing the performances on participants' overall subjective experience of the music. In addition to traditional statistics, functional data analysis techniques were employed to analyze time-varying aspects of the data. The auditory and visual channels were found to convey similar experiences of phrasing but different experiences of tension through much of the performances. We found that visual information served both to augment and to reduce the experience of tension at different points in the musical piece (as revealed by functional linear modeling and functional significance testing). In addition, the musicians' movements served to extend the sense of phrasing, to cue the beginning of new phrases, to indicate musical interpretation, and to anticipate changes in emotional content. Evidence for an interaction effect suggests that there may exist an emergent quality when musical performances are both seen and heard. The investigation augments knowledge of human communicative processes spanning language and music, and involving multiple modalities of emotion and information transfer. (shrink)

For more than a century now, researchers have acknowledged the existence of seemingly arbitrary crossmodal congruency effects between dimensions of sensory stimuli in the general population. Such phenomena, known by a variety of terms including 'crossmodal correspondences', involve individual stimulus properties, rely on a crossmodal mapping of unisensory features, and appear to be shared by the majority of individuals. In other words, members of the general population share underlying preferences for specific pairings across the senses. Crossmodal correspondences between complementary sensory (...) cues have often been referred to as synesthetic correspondences but, we would argue, differ from full-blown synesthetic experiences in a number of important ways, including the fact that there are no idiosyncratic concurrent sensations. Recent psychophysical evidence suggests that such crossmodal correspondences can modulate multisensory integration by helping to resolve the crossmodal binding problem. Here, we propose a model to account for the effects of crossmodal correspondences between complementary auditory and visual cues and critically review their relation to full-blown synesthesia. (shrink)

In a new integration, we show that the visual-spatial structuring of time converges with auditory-spatial left–right judgments for time-related words. In Experiment 1, participants placed past and future-related words respectively to the left and right of the midpoint on a horizontal line, reproducing earlier findings. In Experiment 2, neutral and time-related words were presented over headphones. Participants were asked to indicate whether words were louder on the left or right channel. On critical experimental trials, words were presented equally loud (...) binaurally. As predicted, participants judged future words to be louder on the right channel more often than past-related words. Furthermore, there was a significant cross-modal overlap between the visual-spatial ordering and the auditory judgments , which were continuously related. These findings provide support for the assumption that space and time have certain invariant properties that share a common structure across modalities. (shrink)

This study explored how congruency between facial mimicry and observed expressions affects the stability of conscious facial expression representations. Focusing on the congruency effect between proprioceptive/sensorimotor signals and visual stimuli for happy expressions, participants underwent a binocular rivalry task displaying neutral and happy faces. Mimicry was either facilitated with a chopstick or left unrestricted. Key metrics included Initial Percept (bias indicator), Onset Resolution Time (time from onset to Initial Percept), and Cumulative Time (content stabilization measure). Results indicated that mimicry manipulation (...) significantly impacted Cumulative Time for happy faces, highlighting the importance of congruent mimicry in stabilizing conscious awareness of facial expressions. This supports embodied cognition models, showing the integration of proprioceptive information significantly biases conscious visual perception of facial expressions. (shrink)

Visuo-motor adaptation to optical prisms displacing the visual scene (prism adaptation, PA) is a method used for investigating visuomotor plasticity in healthy individuals and, in clinical settings, for the rehabilitation of unilateral spatial neglect. In the standard paradigm, the adaptation phase involves repeated pointings to visual targets, while wearing optical prisms displacing the visual scene laterally. Here we explored differences in PA, and its aftereffects (AEs), as related to the sensory modality of the target. Visual, auditory, and multisensory - audio-visual (...) - targets in the adaptation phase were used, while participants wore prisms displacing the visual field rightwards by 10◦. Proprioceptive, visual, visual-proprioceptive, auditory-proprioceptive straight-ahead shifts were measured. Pointing to auditory and to audio-visual targets in the adaptation phase produces proprioceptive, visual-proprioceptive, and auditory-proprioceptive AEs, as the typical visual targets did. This finding reveals that cross-modal plasticity effects involve both the auditory and the visual modality, and their interactions (Experiment 1). Even a shortened PA phase, requiring only 24 pointings to visual and audio-visual targets (Experiment 2), is sufficient to bring about AEs, as compared to the standard 92-pointings procedure. Finally, pointings to auditory targets cause AEs, although PA with a reduced number of pointings (24) to auditory targets brings about smaller AEs, as compared to the 92-pointings procedure (Experiment 3). Together, results from the three experiments extend to the auditory modality the sensorimotor plasticity underlying the typical AEs produced by PA to visual targets. Importantly, PA to auditory targets appears characterized by less accurate pointings and error correction, suggesting that the auditory component of the PA process may be less central to the building up of the AEs, than the sensorimotor pointing activity per se. These findings highlight both the effectiveness of a reduced number of pointings for bringing about AEs, and the possibility of inducing PA with auditory targets, which may be used as a compensatory route in patients with visual deficits. (shrink)

Barsalou has recently argued against the strategy of identifying amodal neural representations by using their cross-modal responses (i.e., their responses to stimuli from different modalities). I agree that there are indeed modal structures that satisfy this “cross-modal response” criterion (CM), such as distributed and conjunctive modal representations. However, I argue that we can distinguish between modal and amodal structures by looking into differences in their cross-modal responses. A component of a distributed (...) cell assembly can be considered unimodal because its responses to stimuli from a given modality are stable, whereas its responses to stimuli from any other modality are not (i.e., these are lost within a short time, plausibly as a result of cell assembly dynamics). In turn, conjunctive modal representations, such as superior colliculus cells in charge of sensory integration, are multimodal because they have a stable response to stimuli from different modalities. Finally, some prefrontal cells constitute amodal representations because they exhibit what has been called ‘adaptive coding’. This implies that their responses to stimuli from any given modality can be lost when the context and task conditions are modified. We cannot assign them a modality because they have no stable relation with any input type. (shrink)

As most work on flower foraging focuses on bees, studying Lepidoptera can offer fresh perspectives on how sensory capabilities shape the interaction between flowers and insects. Through a combination of innate preferences and learning, many Lepidoptera persistently visit particular flower species. Butterflies tend to rely on their highly developed sense of colour to locate rewarding flowers, while moths have evolved sophisticated olfactory systems towards the same end. However, these modalities can interact in complex ways; for instance, butterflies’ colour preference can (...) shift depending on olfactory context. The mechanisms by which such cross‐modal interaction occurs are poorly understood, but the mushroom bodies appear to play a central role. Because of the diversity seen within Lepidoptera in terms of their sensory capabilities and the nature of their relationships with flowers, they represent a fruitful avenue for comparative studies to shed light on the co‐evolution of flowers and flower‐visiting insects. (shrink)

For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. The main examples of this are sign languages and gestures. Sign languages, the natural languages of Deaf communities, use systematic and conventionalized movements of the hands, face, and body for linguistic expression. Co-speech gestures, though non-linguistic, are produced in tight semantic and temporal integration with speech and constitute an integral part of language together with speech. The (...) articles in this issue explore and document how gestures and sign languages are similar or different and how communicative expression in the visual modality can change from being gestural to grammatical in nature through processes of conventionalization. As such, this issue contributes to our understanding of how the visual modality shapes language and the emergence of linguistic structure in newly developing systems. Studying the relationship between signs and gestures provides a new window onto the human ability to recruit multiple levels of representation in the service of using or creating conventionalized communicative systems. (shrink)

Type 2 diabetes mellitus is related to a variety of cognitive impairments that may even progress to dementia. Studies have found the angular gyrus is a cross-modal integration hub that is involved in a variety of cognitive processes. However, few studies have focused on the patterns of resting-state functional connections of the AG in patients with T2DM. This study explored the functional connection between the AG and the whole brain and the relationship between the FC and clinical/cognitive (...) variables in patients with T2DM. 44 patients with T2DM and 43 sex-, age-, and education-matched healthy controls underwent resting-state fMRI and received neuropsychological assessments. Compared with the control group, the T2DM group showed abnormal rsFCs between the AG and multiple brain regions. The FC between the left AG and the left medial temporal lobe in the T2DM group was positively correlated with scores on the Montreal Cognitive Assessment, after a Bonferroni correction. Collectively, patients with T2DM have abnormal FCs between the AG and extensive brain regions that may be related to various cognitive processes. (shrink)

The study of perception and the role of the senses have recently risen to prominence in philosophy and are now a major area of study and research. However, the philosophical history of the senses remains a relatively neglected subject. Moving beyond the current philosophical canon, this outstanding collection offers a wide-ranging and diverse philosophical exploration of the senses, from the classical period to the present day. Written by a team of international contributors, it is divided into six parts: -/- Perception (...) from Non-Western Perspectives Perception in the Ancient Period Perception in the Medieval Latin/Arabic Period Perception in the Early Modern Period Perception in the Post-Kantian Period Perception in the Contemporary Period. The volume challenges conventional philosophical study of perception by covering a wide range of significant, as well as hitherto overlooked, topics, such as perceptual judgment, temporal and motion illusions, mirror and picture perception, animal senses and cross-modal integration. By investigating the history of the senses in thinkers such as Plotinus, Auriol, Berkeley and Cavendish; and considering the history of the senses in diverse philosophical traditions, including Chinese, Indian, Byzantine, Greek and Latin it brings a fresh approach to studying the history of philosophy itself. -/- Including a thorough introduction as well as introductions to each section by the editors, The Senses and the History of Philosophy is essential reading for students and researchers in the history of philosophy, perception, philosophy of mind, philosophical psychology, aesthetics and eastern and non-western philosophy. It will also be extremely useful for those in related disciplines such as psychology, religion, sociology, intellectual history and cognitive sciences. (shrink)

Music can act as a mnemonic device that can elicit multiple memories. How musical and non-musical information integrate into complex cross-modal memory representations has however rarely been investigated. Here, we studied the ability of human subjects to associate visual objects with melodies. Musical laypersons and professional musicians performed an associative inference task that tested the ability to form and memorize paired associations between objects and melodies and to integrate these pairs into more complex representations where melodies are linked (...) with two objects across trials. We further investigated whether and how musical expertise modulates these two processes. We analyzed accuracy and reaction times of direct and indirect trials in both groups. We reasoned that the musical and cross-modal memory demands of musicianship might modulate performance in the task and might thus reveal mechanisms that underlie the association and integration of visual information with musical information. Although musicians showed a higher overall memory accuracy, non-musicians’ accuracy was well above chance level in both trial types, thus indicating a significant ability to associate and integrate musical with visual information even in musically untrained subjects. However, non-musicians showed shorter RTs in indirect compared to direct trials, whereas the reverse pattern was found in musicians. Moreover, accuracy of direct and indirect trials correlated significantly in musicians but not in non-musicians. Consistent with previous accounts of visual associative memory, we interpret these findings as suggestive of at least two complimentary mechanisms that contribute to visual-melodic memory integration. A default mechanism that mainly operates at encoding of complex visual-melodic associations and that works with surprising efficacy even in musically untrained subjects. A retrieval-based mechanism that critically depends on an expert ability to maintain and discriminate visual-melodic associations across extended memory delays. Future studies may investigate how these mechanisms contribute to the everyday experience of music-evoked memories. (shrink)

This volume explores the cross-linguistic diversity, and possibly inconsistency, of the span of linguistic means that signal reported speech and thought. The integration of broad linguistic (viewpoint in conversation and narrative) and cognitive (theory of mind and understanding the inner life and thought of others) strategies for handling mixed points of view will be considered.

Comparative Neurospychology is a collection of state-of-the-art essays by some of the world's leading neuropsychologists. Taking as their starting point the assumption that the human brain shares many of its most important functional systems with its primate relatives, the authors take acomparative evolutionary approach to understanding human cognition and brain function. A wide range of subject areas are covered, including memory, visual and somatosensory perception, motor control, attention, cross-modality integration, interhemispheric transmission, and behaviouralintelligence.

Our sense of smell pervasively influences our most common behaviors and daily experience, yet little is known about olfactory consciousness. Over the past decade and a half research in both the fields of Consciousness Studies and Olfaction has blossomed, however, olfactory consciousness has received little to no attention. The olfactory systems unique anatomy, functional organization, sensory processes, and perceptual experiences offers a fecund area for exploring all aspects of consciousness, as well as a external perspective for re-examining the assumptions of (...) contemporary theories of consciousness. It has even been suggested that the olfactory system may represent the minimal neuroanatomy that is required for conscious processing. Here, we seek contributions that should serve as the foundation for future research concerning consciousness using olfaction as a model. -/- Given the variegated nature of research on consciousness, we seek original papers including review articles, original research, and theoretical contributions concerning the nature of olfactory consciousness. The scope of the special edition widely incorporates olfaction as it relates to Consciousness, Awareness, Attention, Phenomenal- or Access-Consciousness, and Qualia. Research concerning olfaction and cross-modal integration as it relates to conscious experience is also welcome. -/- As the initial foray into this uncharted area of research, we welcome contributions from across all disciplines contributing to cognitive neuroscience, including neurobiology, neurology, psychology, philosophy, linguistics, and computer sciences. It is our hope that this special edition will serve as the impetus for future interdisciplinary research on olfaction and consciousness. (shrink)

Despite the recent surge in research on, and interest in, synesthesia, the mechanism underlying this condition is still unknown. Feedforward mechanisms involving overlapping receptive fields of sensory neurons as well as feedback mechanisms involving a lack of signal disinhibition have been proposed. Here I show that a broad range of studies of developmental synesthesia indicate that the mechanism underlying the phenomenon may involve reinstatement of brain activity in different sensory or cognitive streams in a way that is similar to what (...) happens during memory retrieval of semantically associated items. In the paper’s final sections I look at the relevance of synesthesia research, given the memory model, to our understanding of multisensory perception and common mapping patterns. (shrink)

The auditory gaze cueing effect (auditory-GCE) is a faster response to auditory targets at an eye-gaze cue location than at a non-cue location. Previous research has found that auditory-GCE can be influenced by the integration of both gaze direction and emotion conveyed through facial expressions. However, it is unclear whether the emotional information of auditory targets can be cross-modally integrated with gaze direction to affect auditory-GCE. Here, we set neutral faces with different gaze directions as cues and three (...) emotional sounds (fearful, happy, and neutral) as targets to investigate how the emotion of sound target modulates the auditory-GCE. Moreover, we conducted a controlled experiment using arrow cues. The results show that the emotional content of sound targets influences the auditory-GCE but only for those induced by facial cues. Specifically, fearful sounds elicit a significantly larger auditory-GCE compared to happy and neutral sounds, indicating that the emotional content of auditory targets plays a modulating role in the auditory-GCE. Furthermore, this modulation appears to occur only at a higher level of social meaning, involving the integration of emotional information from a sound with social gaze direction, rather than at a lower level, which involves the integration of direction and auditory emotion. (shrink)

How do humans manage to keep track of a gradually changing object or person as the same persisting individual despite the fact that the extraction of information about this individual must often rely on heterogeneous information sources and heterogeneous tracking methods? The article introduces the Empirical Tracking of Individuals theory to address this problem. This theory proposes an analysis of the concept of integrated tracking, which refers to the capacity to acquire, store, and update information about the identity and location (...) of individuals in our environment. It hypothesizes that certain functions of attention are a key to explaining how the cognitive flexibility of the human mind overcomes the heterogeneity of sources and methods in integrated tracking. At least two premises lend support to this hypothesis. First, heterogeneity of tracking sources is overcome by the combination of information from multiple perceptual modalities and a phenomenon of multisensory 'transparency'. Second, heterogeneity of tracking sources and methods may also be overcome by inferences that combine information across domains to acquire reasons to believe propositions about the target's location and identity. (shrink)

Molyneux’s problem asks whether a person blind from birth, upon gaining sight, could immediately recognize and distinguish objects by sight alone that were previously known only by touch. Historical and contemporary empirical studies have explored this question with inconclusive results due to empirical limitations. More recently, Held and colleagues (2011) found that treated congenitally blind individuals cannot immediately recognize objects previously familiar through touch. Piller and colleagues (2023) further reported the absence of visual illusions in blind and recently visually-restored individuals. (...) Nevertheless, cross-modal mappings gradually develop post-sight restoration. These findings suggest a reluctance of the mind to make cross-modal inferences, aligning with the predictive processing (PP) framework. PP posits that the mind generates top-down predictions about sensory stimuli, updating internal models through prediction errors when expectations are not met. With no prior visual experience, generative models in congenital blind individuals fail to produce accurate predictions. PP’s representational claims have been challenged by 4E cognitivists, who emphasize embodied, embedded, extended, and enactive aspects of cognition. This paper proposes a Situated Predictive Processing (SPP) framework that integrates PP with 4E cognition through the concept of situated mental representations, offering a new perspective on the Molyneux’s problem and emphasizing the role of experience and situatedness in the gradual development of visual-tactile mappings post-sight restoration. (shrink)

One understanding of naturalism about perception allows that results in the sciences bearing on the senses may have an impact upon philosophical theorising on perception. Its opponents reject or, at least, are much more wary about this possibility. I consider two cases: the implications of prediction error theories for naïve realism and the latest empirical research on cross modal illusions, and taste, for the traditional division of the senses into five. Although in neither case are the implications straightforward, (...) I argue that there is a tension in the first case, relating to naïve realism, given the structure of the debate against representationalism, what naïve realists say about hallucination, and assertion of the significant differences between perception and cognition. In the second, the conflict is avoided by recognising distinctive features of how senses are integrated and realised, and a new personal level of explanation. (shrink)

I argue that a class of recently-discovered cross-modal illusions gives reason to posit a dimension of content shared across perceptual modalities and to abandon the traditional view according to which perceptual content is exclusively constituted by discrete modality-specific contents.

Philosophers and scientists have studied sensory perception and, in particular, vision for many years. Increasingly, however, they have become interested in the nonvisual senses in greater detail and the problem of individuating the senses in a more general way. The Aristotelian view is that there are only five external senses—smell, taste, hearing, touch, and vision. This has, by many counts, been extended to include internal senses, such as balance, proprioception, and kinesthesis; pain; and potentially other human and nonhuman senses. This (...) “multisensory turn” has been driven partly by developments in contemporary psychology and neuroscience, which have revealed a host of complex interrelations and interactions between sensory modalities previously thought to be distinct. Contrasts between modalities and other crossmodal phenomena, including multisensory integration, synesthesia, and sensory substitution, have also begun to receive more attention in a burgeoning scientific and philosophical literature on multisensory perception and other crossmodal effects. This article focuses on recent empirically informed contributions to the philosophy of perception, as well as key scientific works that provide important background information and insights into the nature of the senses and sensory perception. Indeed, one of the lessons of the multisensory turn, and of contemporary philosophy of mind more generally, is that philosophers ignore this body of empirical research at their peril because many human and animal senses turn out to be richer and more complex than philosophers and scientists had previously imagined, making this a fruitful area for interdisciplinary interaction and research. (shrink)

This paper provides a categorization of cross-modal experiences. There are myriad forms. Doing so allows us to think clearly about the nature of different cross-modal experiences and allows us to clearly formulate competing hypotheses about the kind of experiences involved in different cross-modal phenomena.

This review covers experimental approaches to sound-symbolism—from infants to adults, and from Sapir’s foundational studies to twenty-first century product naming. It synthesizes recent behavioral, developmental, and neuroimaging work into a systematic overview of the cross-modal correspondences that underpin iconic links between form and meaning. It also identifies open questions and opportunities, showing how the future course of experimental iconicity research can benefit from an integrated interdisciplinary perspective. Combining insights from psychology and neuroscience with evidence from natural languages provides (...) us with opportunities for the experimental investigation of the role of sound-symbolism in language learning, language processing, and communication. The review finishes by describing how hypothesis-testing and model-building will help contribute to a cumulative science of sound-symbolism in human language. (shrink)

Cross-modal perceptual illusions occur when a stimulus to one modality impacts perceptual experience associated with another modality. Unlike synaesthesia, cross-modal illusions are intelligible as results of perceptual strategies for dealing with sensory stimulation to multiple modalities, rather than as mere quirks. I argue that understanding cross-modal illusions reveals an important flaw in a widespread conception of the senses, and of their role in perceptual experience, according to which understanding perception and perceptual experience is a (...) matter of assembling independently viable stories about vision, audition, olfaction, and the rest. (shrink)

In this paper we offer a theory of cross-modal objects. To begin, we discuss two kinds of linkages between vision and audition. The first is a duality. The the visual system detects and identifies surfaces ; the auditory system detects and identifies sources . Surfaces are illuminated by sources of light; sound is reflected off surfaces. However, the visual system discounts sources and the auditory system discounts surfaces. These and similar considerations lead to the Theory of Indispensable Attributes (...) that states the conditions for the formation of gestalts in the two modalities. The second linkage involves the formation of audiovisual objects, integrated cross-modal experiences. We describe research that reveals the role of cross-modal causality in the formation of such objects. These experiments use the canonical example of a causal link between vision and audition: a visible impact that causes a percussive sound. (shrink)

Chris Daly défend « l'explicationisme », la position selon laquelle l'inférence a la meilleure explication constitue une façon acceptable de justifier une théorie. Il la défend en tentant de justifier la position explicationiste par ses propres ressources, c'est-a-dire par elle-même. Je soutiens que dans le contexte de la métaphysique, cette défense échoue. L'explicationiste échoue à se justifier par ses propres ressources et l'une de ses premisses centrales ne peut pas être justifiée uniquement de façon externaliste.

This paper discusses cross-border projects and their respective social representations to local groups. We present as a case study the boatmen who pilot their boats between Brazil and France on the construction of the Binational Bridge. Catraieiros are currently responsible for much of the logistics between Amapá and French Guiana. Our theoretical framework is based on the literature on Regional Integration and Social Representation. We applied interviews to one third of the catraieiros. We concluded that in the Subject-Representation-Object (...) relationship, the group that understood itself as responsible for the dialogue between two cities built a negative and threatening representation on the Bridge. (shrink)

Visual information can be provided to blind users through sensory substitution devices that convert images into sound. Through extensive use to develop expertise, some blind users have reported visual experiences when using such a device. These blind expert users have also reported visual phenomenology to other sounds even when not using the device. The blind users acquired synthetic synaesthesia, with visual experience evoked by sounds only after gaining such expertise. Sensorimotor learning may facilitate and perhaps even be required to develop (...) expertise in the use of multimodal information. Furthermore, other areas where expertise is acquired in dividing attention amongst cross-modal information or integrating such information might also give rise to synthetic synaesthesia. (shrink)

We can perceive shapes visually and tactilely, and the information we gain about shapes through both sensory modalities is integrated smoothly into and functions in the same way in our behavior independently of whether we gain it by sight or touch. There seems to be no reason in principle we couldn't perceive shapes through other sensory modalities as well, although as a matter of fact we do not. While we can identify shapes through other sensory modalities—e.g., I may know by (...) smell (the scent of mango) that the object causing my sensory experience is round—this is not perceiving an object as shaped, but rather inferring from the character of one's sensory experience and collateral information that an object of a certain shape caused it. That it is possible to perceive shape by other modalities, however, is suggested by the case of bats and aquatic mammals like dolphins which navigate through their environment by a form of sonar. It is plausible that they have some form of auditory representation of space, and so of shape. These facts about shape perception raise important questions about the relation between those features of perceptual experience which are intrinsic to different sensory modalities and the nature of our perceptual representation of shapes, and, more generally, of the space within which we perceive shaped objects to be located. John Campbell's paper, "Molyneux's Problem" (see above), raises a number of interesting and important questions about the nature of our perception of shape properties, particularly the cross-modal nature of shape perception, and ties them to more general questions about the nature both of perceptual.. (shrink)

Objective: Evidence suggests people experience an oral size illusion and commonly perceive oral size inaccurately; however, the nature of the illusion remains unclear. The objectives of the present study were to confirm the presence of an oral size illusion, determine the magnitude (amount) and direction (underestimation or overestimation) of the illusion, and determine whether immediately prior crossmodal perceptual experiences affected the magnitude and direction. Design: Participants (N = 27) orally assessed 9 sizes of stainless steel spheres (1/16 in to 1/2 (...) in) categorized as small, medium, or big, and matched them with digital and visual reference sets. Each participant completed 20 matching tasks in 3 assessments. For control assessments, 6 oral spheres were matched with reference sets of same-sized spheres. For primer-control assessments, similar to control, 6 matching tasks were preceded by cross-modal experiences of the same-sized sphere. For experimental assessments, 8 matching tasks were preceded by a cross-modal experience of a differently sized sphere. Results: For control assessments, small and medium spheres were consistently underestimated, and big spheres were consistently overestimated. For experimental assessments, magnitude and direction of the oral size illusion varied according to the size of the sphere used in the cross-modal experience. Conclusion: Results seemed to confirm an oral size illusion, but direction of the illusion depended on the size of the object. Immediately prior cross-modal experiences influenced magnitude and direction of the illusion, suggesting that aspects of oral perceptual experience are dependent upon factors outside of oral perceptual anatomy and the properties of the oral stimulus. (shrink)

The article explores cross-modal iconic relations in nine diverse Western-music songs ranging from 1600 to 2015, all of them thematizing dysphoric weeping. Initial input comes from five recurrent features observed in ancient Greek texts associated with performative events, including the prominence of sound, interjections and strong self-referentiality, repetitions and refrains, the motif of endlessness, and tears associated with streams of water, dew, and libation liquids. The analysis adopts Peirce’s conceptual distinction between image, diagram, and metaphor iconicity, although the (...) continuum reading proposed by several scholars is ultimately favored. The sample turns out to offer plenty of evidence of iconic relations. Interjections, falsetto, verbal repetitions, musical repetitions, musical rendering of sighs and endlessness, ostinato patterns, downward notes, and water images (rain and rivers), all of this is shown to substantiate cross-modal iconicity encompassing weeping, lyrics, music (including musical notation), and images, in different combinations. Quantitative investigations confirming/disconfirming and adding iconicity patterns, and comparative analyses linking Western and non-Western lament traditions constitute desiderata for future work. (shrink)