This article discusses the importance of social interaction for the development of the representations for symbolic communication. We suggest that there is no need to distinguish between different representational systems emerging at different stages of development. Instead, we propose that representations are rich right from the beginning of a child’s life, and that they are driven mainly by acting and interacting in the physical and social world. The more variety in a child’s interactional experience (i.e., synchrony, sequentiality, and prediction), the (...) more enriched and abstracted the representations become. We review literature providing evidence for the ways in which infants’ development toward symbolic communication benefits from repeated social (inter)action and consider some implications for computational approaches. (shrink)

It has been proposed that the design of robots might benefit from interactions that are similar to caregiver–child interactions, which is tailored to children’s respective capacities to a high degree. However, so far little is known about how people adapt their tutoring behaviour to robots and whether robots can evoke input that is similar to child-directed interaction. The paper presents detailed analyses of speakers’ linguistic behaviour and non-linguistic behaviour, such as action demonstration, in two comparable situations: In one experiment, parents (...) described and explained to their nonverbal infants the use of certain everyday objects; in the other experiment, participants tutored a simulated robot on the same objects. The results, which show considerable differences between the two situations on almost all measures, are discussed in the light of the computer-as-social-actor paradigm and the register hypothesis. Keywords: child-directed speech ; motherese; robotese; motionese; register theory; social communication; human–robot interaction ; computers-as-social-actors; mindless transfer. (shrink)

Previous studies have found that narrative input conveyed through different media influences the structure and content of children’s narrative retellings. Visual, televised narratives appear to elicit richer and more detailed narratives than traditional, orally transmitted storybook media. To extend this prior work and drawing from research on narrative elaboration, the current study’s main goal was to identify the core plot component differences (the who, what, where, when, why, and how of a story) between children’s retellings of televised versus traditional storybook (...) narratives. However, because children also differ individually in their IQ, we further incorporated this variable into our analysis of children’s narrative retellings. For our purpose, a novel coding schema was developed, following and extending the existing narrative elaboration approaches. Participants were 46 typically developing children aged 4–5 years from Germany. The current study incorporated two narrative input conditions to which children were randomly assigned: in the video condition, children watched a non-verbal, visually conveyed, televised story from a DVD; and in the book condition, children read the story with an adult and experienced an orally conveyed version in the form of a book with minimal accompanying pictures. In both conditions, the same story was conveyed. After including IQ as a covariate in our analyses, results show that the children from the video condition gave significantly more elaborated retellings, particularly across the who, what, and where (sub-)components. Differences between the conditions in the component when, how and why did not reach statistical significance. Our findings indicate that different media types entail differential cognitive processing demands of a story, resulting in type-specific memories and narratives. The effect of different medial conditions was significant and persisted when individual differences in cognitive development were considered. Consequences for children’s development, education, and interaction with and within today’s digital world are discussed. (shrink)

This article presents results from a multidisciplinary research project on the integration and transfer of language knowledge into robots as an empirical paradigm for the study of language development in both humans and humanoid robots. Within the framework of human linguistic and cognitive development, we focus on how three central types of learning interact and co-develop: individual learning about one's own embodiment and the environment, social learning (learning from others), and learning of linguistic capability. Our primary concern is how these (...) capabilities can scaffold each other's development in a continuous feedback cycle as their interactions yield increasingly sophisticated competencies in the agent's capacity to interact with others and manipulate its world. Experimental results are summarized in relation to milestones in human linguistic and cognitive development and show that the mutual scaffolding of social learning, individual learning, and linguistic capabilities creates the context, conditions, and requisites for learning in each domain. Challenges and insights identified as a result of this research program are discussed with regard to possible and actual contributions to cognitive science and language ontogeny. In conclusion, directions for future work are suggested that continue to develop this approach toward an integrated framework for understanding these mutually scaffolding processes as a basis for language development in humans and robots. (shrink)

In a longitudinal naturalistic study, we observed German mothers interacting with their infants when they were 3 and 6 months old. Pursuing the idea that infants’ attention is socialized in everyday interactions, we explored whether eye contact is reinforced selectively by behavioral modification in the input provided to infants. Applying a microanalytical approach focusing on the sequential organization of interaction, we explored how the mother draws the infant’s attention to herself and how she tries to maintain attention when the infant (...) is looking at her. Results showed that eye contact is reinforced by specific infant-directed practices: interrogatives and conversational openings, multimodal stimulation, repetition, and imitation. In addition, these practices are contingent on the infant’s own behavior. By comparing the two data points, we showed how the education of attention evolves hand-in-hand with the developing capacities of the infant. Keywords: multimodal input; eye contact; interactional adaptation; ecology of attention; social learning. (shrink)

When young children learn to use language, they start to use their hands in co‐verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) analysis (...) to characterize combinations of speech and gesture meaning and related them to the child's visual‐spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON‐R test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities. (shrink)

The gesture-as-simulated-action framework explains the occurrence of iconic gestures. Accordingly, simulated visual imagery gives rise to observer-viewpoint, whereas simulated motor imagery gives rise to character-viewpoint gestures. Because little is known about whether this relationship is either the product of becoming a competent speaker in different communicative tasks or exists from an early age, we investigated 4-year-olds. In the first session, 55 children performed three different communicative tasks. In the second session, we administered a SON-R non-verbal intelligence test to assess children’s (...) cognitive abilities. Results revealed a strong association between the frequency with which children used observer-viewpoints gestures and their scores in visual processing abilities. Because the association between gestures and visuospatial cognitive abilities was found across three communicative tasks, we can propose that it is a general ability rather than a phenomenon of task-specific discourse competence. We discuss these findings in terms of gestural threshold and cognitive offloading. (shrink)

In order for artificial intelligent systems to interact naturally with human users, they need to be able to learn from human instructions when actions should be imitated. Human tutoring will typically consist of action demonstrations accompanied by speech. In the following, the characteristics of human tutoring during action demonstration will be examined. A special focus will be put on the distinction between two kinds of motion events: path-oriented actions and manner-oriented actions. Such a distinction is inspired by the literature pertaining (...) to cognitive linguistics, which indicates that the human conceptual system can distinguish these two distinct types of motion. These two kinds of actions are described in language by more path-oriented or more manner-oriented utterances. In path-oriented utterances, the source, trajectory, or goal is emphasized, whereas in manner-oriented utterances the medium, velocity, or means of motion are highlighted. We examined a video corpus of adult–child interactions comprised of three age groups of children—pre-lexical, early lexical, and lexical—and two different tasks, one emphasizing manner more strongly and one emphasizing path more strongly. We analyzed the language and motion of the caregiver and the gazing behavior of the child to highlight the differences between the tutoring and the acquisition of the manner and path concepts. The results suggest that age is an important factor in the development of these action categories. The analysis of this corpus has also been exploited to develop an intelligent robotic behavior—the tutoring spotter system—able to emulate children's behaviors in a tutoring situation, with the aim of evoking in human subjects a natural and effective behavior in teaching to a robot. The findings related to the development of manner and path concepts have been used to implement new effective feedback strategies in the tutoring spotter system, which should provide improvements in human–robot interaction. (shrink)

The economic principle of communication, according to which successful communication can be reached by least effort, has been studied for verbal communication. With respect to nonverbal behavior, it implies that forms of iconic gestures change over the course of communication and become reduced in the sense of less pronounced. These changes and their effects on learning are currently unexplored in relevant literature. Addressing this research gap, we conducted a word learning study to test the effects of changing gestures on children’s (...) slow mapping. We applied a within-subject design and tested 51 children, aged 6.7 years, who learned unknown words from a story. The storyteller acted on the basis of two conditions: In one condition, in which half of the target words were presented, the story presentation was enhanced with progressively reduced iconic gestures ; in the other condition, half of the target words were accompanied by fully executed iconic gestures. To ensure a reliable gesture presentation, children were exposed to a recorded person telling a story in both conditions. We tested the slow mapping effects on children’s productive and receptive word knowledge three minutes as well as two to three days after being presented the story. The results suggest that children’s production of the target words, but not their understanding thereof, was enhanced by PRG. (shrink)

Language comprehension of action verbs recruits embodied representations in the brain that are assumed to invoke a mental simulation. This extends to abstract concepts, as well. We, therefore, argue that mental simulation works across levels of abstractness and involves higher-level schematic structures that subsume a generic structure of actions and events.

Research of tutoring in parent-infant interaction has shown that tutors – when presenting some action – modify both their verbal and manual performance for the learner (‘motherese’, ‘motionese’). Investigating the sources and effects of the tutors’ action modifications, we suggest an interactional account of ‘motionese’. Using video-data from a semi-experimental study in which parents taught their 8- to 11-month old infants how to nest a set of differently sized cups, we found that the tutors’ action modifications (in particular: high arches) (...) functioned as an orienting device to guide the infant’s visual attention (gaze). Action modification and the recipient’s gaze can be seen to have a reciprocal sequential relationship and to constitute a constant loop of mutual adjustments. Implications are discussed for developmental research and for robotic ‘Social Learning’. We argue that a robot system could use on-line feedback strategies (e.g. gaze) to pro-actively shape a tutor’s action presentation as it emerges. (shrink)