Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emotional design has become a new direction for most designers to improve their design thinking. In the emotional interaction design, the machine needs to capture the user's key information in real time, recognize the user's emotional state, and use (...) a variety of clues to finally determine the appropriate user model. Based on this background, this research uses a deep learning mechanism for more accurate and effective emotion recognition, thereby optimizing the design of the interactive system and improving the user experience. First of all, this research discusses how to use user characteristics such as speech, facial expression, video, heartbeat, etc., to make machines more accurately recognize human emotions. Through the analysis of various characteristics, the speech is selected as the experimental material. Second, a speech-based emotion recognition method is proposed. The mel-Frequency cepstral coefficient of the speech signal is used as the input of the improved long and short-term memory network. To ensure the integrity of the information and the accuracy of the output at the next moment, ILSTM makes peephole connections in the forget gate and input gate of LSTM, and adds the unit state as input data to the threshold layer. The emotional features obtained by ILSTM are input into the attention layer, and the self-attention mechanism is used to calculate the weight of each frame of speech signal. The speech features with higher weights are used to distinguish different emotions and complete the emotion recognition of the speech signal. Experiments on the EMO-DB and CASIA datasets verify the effectiveness of the model for emotion recognition. Finally, the feasibility of emotional interaction system design is discussed. (shrink)

This commentary focuses on how bottom-up neocortical models can be developed into eigenfunction expansions of probability distributions appropriate to describe short-term memory in the context of scalp EEG. The mathematics of eigenfunctions are similar to the top-down eigenfunctions developed by Nunez, despite different physical manifestations. The bottom-up eigenfunctions are at the local mesocolumnar scale, whereas the top-down eigenfunctions are at the global regional scale. Our respective approaches have regions of substantial overlap, and future studies may expand top-down eigenfunctions (...) into the bottom-up eigenfunctions, yielding a model of scalp EEG expressed in terms of columnar states of neocortical processing of attention and short-term memory. Footnotes1 The author is also affiliated with DRW Investments LLC, 311 S. Wacker Drive, Chicago, IL 60606. (shrink)

I present a framework for modeling memory, retrieval, perception, and their interactions. Recent versions of the models were inspired by Bayesian induction: We chose models that make optimal decisions conditioned on a memory/perceptual system with inherently noisy storage and retrieval. The resultant models are, fortunately, largely consistent with my models dating back to the 1960s, and are therefore natural successors. My recent articles have presented simplified models in order to focus on particular applications. This article takes a larger perspective and (...) places the individual models in a more global framework. I will discuss (1) the storage of episodic traces, the accumulation of these into knowledge (e.g., lexical/semantic traces in the case of words), and the changes in knowledge caused by learning; (2) the retrieval of information from episodic memory and from general knowledge; (3) decisions concerning storage, retrieval, and responding. Examples of applications include episodic recognition and cued and free recall, perceptual identification (naming, yes–no and forced‐choice), lexical decision, and long‐term and short‐term priming. (shrink)

The view that short-term memory should be conceived of as being a process based on the activation of long-term memory is inconsistent with neuropsychological evidence. Data from brain-damaged patients, showing specific patterns of impairment, are compatible with a vision of memory as a multiple-component system, whose different aspects, in neurologically unimpaired subjects, show a high degree of interaction.

People frequently gesture when a word is on the tip of their tongue (TOT), yet research is mixed as to whether and why gesture aids lexical retrieval. We tested three accounts: the lexical retrieval hypothesis, which predicts that semantically related gestures facilitate successful lexical retrieval; the cognitive load account, which predicts that matching gestures facilitate lexical retrieval only when retrieval is hard, as in the case of a TOT; and the motor movement account, which predicts that any motor movements should (...) support lexical retrieval. In Experiment 1 (a between‐subjects study; N = 90), gesture inhibition, but not neck inhibition, affected TOT resolution but not overall lexical retrieval; participants in the gesture‐inhibited condition resolved fewer TOTs than participants who were allowed to gesture. When participants could gesture, they produced more representational gestures during resolved than unresolved TOTs, a pattern not observed for meaningless motor movements (e.g., beats). However, the effect of gesture inhibition on TOT resolution was not uniform; some participants resolved many TOTs, while others struggled. In Experiment 2 (a within‐subjects study; N = 34), the effect of gesture inhibition was traced to individual differences in verbal, not spatial short‐term memory (STM) span; those with weaker verbal STM resolved fewer TOTs when unable to gesture. This relationship between verbal STM and TOT resolution was not observed when participants were allowed to gesture. Taken together, these results fit the cognitive load account; when lexical retrieval is hard, gesture effectively reduces the cognitive load of TOT resolution for those who find the task especially taxing. (shrink)

There has been a long-standing debate in philosophy and psychology about the role of representation in visual perception. Here, we argue on the basis of evidence from philosophy, psychology, and neuroscience that episodic and schematic memory representations are pivotal to the visual perception of objects and scenes. In the visual perception of objects and scenes, sensory information is initially matched with object and scene templates, or schemas, in long-term memory. The most relevant representations are then selected for encoding in (...) working memory. We furthermore argue that activations of episodic memory representations contribute to the fineness of grain of visual representations. The representational view of visual perception that emerges is what we call the "template tuning view." According to this view, prior information – specifically, long-term memories – shape the representational content of visual perception. In the final section of the chapter, we argue that unlike representational conceptions of visual perception, naïve and direct realist theories have difficulties accommodating these findings. (shrink)

ABSTRACTRecognising identity and emotion conveyed by the face is important for successful social interactions and has thus been the focus of considerable research. Debate has surrounded the extent...

The vigorous development of online education has produced massive amounts of education data. How to mine and analyze education big data has become an urgent problem in the field of education and big data knowledge engineering. As for the dynamic learning data, knowledge tracing aims to track learners’ knowledge status over time by analyzing the learners’ exercise data, so as to predict their performance in the next time step. Deep learning knowledge tracking performs well, but they mainly model the knowledge (...) components while ignoring the personalized information of questions and learners, and provide limited interpretability in the interaction between learners’ knowledge status and questions. A context-aware attentive knowledge query network model is proposed in this paper, which combines flexible neural network models with interpretable model components inspired by psychometric theory. We use the Rasch model to regularize the embedding of questions and learners’ interaction tuples, and obtain personalized representations from them. In addition, the long-term short-term memory network and monotonic attention mechanism are used to mine the contextual information of learner interaction sequences and question sequences. It can not only retain the ability to model sequences, but also use the monotonic attention mechanism with exponential decay term to extract the hidden forgetting behavior and other characteristics of learners in the learning process. Finally, the vector dot product is used to simulate the interaction between the learners’ knowledge state and questions to improve the interpretability. A series of experimental results on 4 real-world online learning datasets show that CAKQN has the best performance, and its AUC value is improved by an average of 2.945% compared with the existing optimal model. Furthermore, the CAKQN proposed in this paper can not only track learners’ knowledge status like other models but also model learners’ forgetting behavior. In the future, our research will have high application value in the realization of personalized learning strategies, teaching interventions, and resource recommendations for intelligent online education platforms. (shrink)

As an explicit organizing metaphor, memory aid, and conceptual framework, the prefrontal cortex may be viewed as a five-member ‘Executive Committee,’ as the prefrontal-control extensions of five sub-and-posterior-cortical systems: the ‘Perceiver’ is the frontal extension of the ventral perceptual stream which represents the world and self in object coordinates; the ‘Verbalizer’ is the frontal extension of the language stream which represents the world and self in language coordinates; the ‘Motivator’ is the frontal cortical extension of a subcortical extended-amygdala stream which (...) represents the world and self in motivational/emotional coordinates; the ‘Attender’ is the frontal cortical extension of a subcortical extended-hippocampal stream which represents the world and self in spatiotemporal coordinates and directs attention to internal and external events; and the ‘Coordinator’ is the frontal extension of the dorsal perceptual stream which represents the world and self in body- and eye-coordinates and controls willed action and working memory. This tutorial review examines the interacting roles of these five systems in perception, working memory, attention, long - term memory, motor control, and thinking. (shrink)

Neural models have proposed how short-term memory (STM) storage in working memory and long-term memory (LTM) storage and recall are linked and interact, but are realized by different mechanisms that obey different laws. The authors' data can be understood in the light of these models, which suggest that the authors may have gone too far in obscuring the differences between these processes.

This study investigates the role of locality (a task/material-related variable), demographic factors (age, education, and sex), cognitive capacities (verbal working memory [WM], verbal short-term memory [STM], speed of processing [SOP], and inhibition), and morphosyntactic category (time reference and grammatical aspect) in verb-related morphosyntactic production (VRMP). A sentence completion task tapping production of time reference and grammatical aspect in local and nonlocal configurations, and cognitive tasks measuring verbal WM capacity, verbal STM capacity, motor SOP, perceptual SOP, and inhibition were (...) administered to 200 neurotypical Greek-speaking participants, aged between 19 and 80 years. We fitted generalized linear mixed-effects models and performed path analyses. Significant main effects of locality, age, education, verbal WM capacity, motor SOP, and morphosyntactic category emerged. Production of time reference and aspect did not interact with any of the significant factors (i.e., age, education, verbal WM capacity, motor SOP, and locality), and locality did not interact with any memory system. Path analyses revealed that the relationships between age and VRMP, and between education and VRMP were partly mediated by verbal WM; and the relationship between verbal WM and VRMP was partly mediated by perceptual SOP. Results suggest that subject-, task/material- and morphosyntactic category-specific factors determine accuracy performance on VRMP; and the effects of age, education, and verbal WM on VRMP are partly indirect. The fact that there was a significant main effect of verbal WM but not of verbal STM on accuracy performance in the VRMP task suggests that it is predominantly the processing component (and not the storage component) of verbal WM that supports VRMP. Lastly, we interpret the results as suggesting that VRMP is also supported by a procedural memory system whose efficiency might be reflected in years of formal education. (shrink)

A modified visual search task demonstrates that humans are very good at resuming a search after it has been momentarily interrupted. This is shown by exceptionally rapid response time to a display that reappears after a brief interruption, even when an entirely different visual display is seen during the interruption and two different visual searches are performed simultaneously. This rapid resumption depends on the stability of the visual scene and is not due to display or response anticipations. These results are (...) consistent with the existence of an iterative hypothesis-testing mechanism that compares information stored in short-term memory (the perceptual hypothesis) with information about the display (the sensory pattern). In this view, rapid resumption occurs because a hypothesis based on a previous glance of the scene can be tested very rapidly in a subsequent glance, given that the initial hypothesis-generation step has already been performed. (shrink)

‘External memories’ raise a question about context: ‘external to what?’ External memory is a technical term applied to everything that can be memorized in an individual's environment. As a general rule I have decided to retain the technical terms that characterize the area of the topic under discussion. It was Ted Nelson who coined the word hypertext in 1967 to signify non-sequential writing as well as a computer technology that allowed the user to move about freely by means of (...) software links. I shall use the term software to mean computer programs. Electronic publishing started with the Xanadu system developed by Nelson. At the lowest level a hypertext is a system for managing databases that allows the user to connect together information screens using links. It is a collection of texts interconnected by means of interactive switches. This dynamic presentation of non-linear texts could not be printed on a conventional page. Hypertext allows the origin of a piece of information to be traced. In this article traceability is synonymous with explanation: ‘Why was this equipment designed in one way rather than another?’ It is about discovering the route of the design process that led to the eventual idea for the equipment. The necessary explanations are often required long after decisions were made, and this is why traceability is a matter of memory. The term agent is often used to mean those members of a community that act on its behalf. There are agents for a company or public department. There are biological agents. In a non-technological world traceability is ideally carried out by human agents who explain to other human agents. The term agent has been extended into the computer world to mean software to which the user delegates, sometimes unwittingly, tasks of varying complexity. These are then called software agents. In this article I refer to agent in its wider meaning, to be understood as an actor or an entity able to act in a limited context and having a quite specific role and appropriate resources. (shrink)

According to the concept of desirable difficulties, introducing difficulties in learning may sacrifice short-term performance in order to benefit long-term retention of learning. We describe three types of desirable difficulty effects: testing, generation, and varied conditions of practice. The empirical literature indicates that desirable difficulty effects are not always obtained and we suggest that cognitive load theory may be used to explain many of these contradictory results. Many failures to obtain desirable difficulty effects may occur under conditions (...) where working memory is already stressed due to the use of high element interactivity information. Under such conditions, the introduction of additional difficulties may be undesirable rather than desirable. Empirical evidence from diverse experiments is used to support this hypothesis. (shrink)

It is known that, on average, people adapt their choice of memory strategy to the subjective utility of interaction. What is not known is whether an individual's choices are boundedly optimal. Two experiments are reported that test the hypothesis that an individual's decisions about the distribution of remembering between internal and external resources are boundedly optimal where optimality is defined relative to experience, cognitive constraints, and reward. The theory makes predictions that are tested against data, not fitted to it. The (...) experiments use a no-choice/choice utility learning paradigm where the no-choice phase is used to elicit a profile of each participant's performance across the strategy space and the choice phase is used to test predicted choices within this space. They show that the majority of individuals select strategies that are boundedly optimal. Further, individual differences in what people choose to do are successfully predicted by the analysis. Two issues are discussed: the performance of the minority of participants who did not find boundedly optimal adaptations, and the possibility that individuals anticipate what, with practice, will become a bounded optimal strategy, rather than what is boundedly optimal during training. (shrink)

In debates in philosophy and cognitive science concerning short-term memory mechanisms and perceptual experience, most discussion has focused on the working memory and the various forms of sensory memory such as iconic memory. In this paper, I present a summary of some evidence for a proposed further form of memory termed conceptual short-term memory. I go on to outline some of the ways in which this additional distinctive sort of short-term memory might be of (...) relevance to ongoing philosophical debates, specifically in relation to questions about high-level perceptual phenomenology, the relationship between consciousness and reportability, and the boundary between cognition and perception. I conclude that conceptual short-term memory offers a promising new direction of research and philosophical investigation. (shrink)

This paper provides an interpretation of Hans-Jörg Rheinberger’s notions of epistemic things and historical epistemology . I argue that Rheinberger’s approach articulates a unique contribution to current debates about integrated HPS, and I propose some modifications and extensions of this contribution. Drawing on examples from memory research, I show that Rheinberger is right to highlight a particular feature of many objects of empirical research (“epistemic things”)—especially in the contexts of exploratory experimentation—namely our lack of knowledge about them. I argue that (...) this analysis needs to be supplemented with an account of what scientists do know, and in particular, how they are able to attribute rudimentary empirical contours to objects of research. These contours are closely connected to paradigmatic research designs, which in turn are tied to basic methodological rules for the exploration of the purported phenomena. I suggest that we engage with such rules in order to develop our own normative (epistemological) categories, and I tie this proposal to the idea of a methodological naturalism in philosophy of science. (shrink)

Music can express people’s thoughts and emotions. Music therapy is to stimulate and hypnotize the human brain by using various forms of music activities, such as listening, singing, playing and rhythm. With the empowerment of artificial intelligence, music therapy technology has made innovative development in the whole process of “diagnosis, treatment and evaluation.” It is necessary to make use of the advantages of artificial intelligence technology to innovate music therapy methods, ensure the accuracy of treatment schemes, and provide more paths (...) for the development of the medical field. This paper proposes an long short-term memory -based generation and classification algorithm for multi-voice music data. A Multi-Voice Music Generation system called MVMG based on the algorithm is developed. MVMG contains two main steps. At first, the music data are modeled to the MDPI and text sequence data by using an autoencoder model, including music features extraction and music clip representation. And then an LSTM-based music generation and classification model is developed for generating and analyzing music in specific treatment scenario. MVMG is evaluated based on the datasets collected by us: the single-melody MIDI files and the Chinese classical music dataset. The experiment shows that the highest accuracy of the autoencoder-based feature extractor can achieve 95.3%. And the average F1-score of LSTM is 95.68%, which is much higher than the DNN-based classification model. (shrink)

Previous studies of verbal short-term memory indicate that STM for serial order may be linked to language development and developmental language disorder. To clarify whether a domain-general mechanism is impaired in DLD, we studied the relations between age, non-verbal serial STM, and language competence. We hypothesized that non-verbal serial STM differences between groups of children with DLD and typically developing children are linked to their language acquisition differences. Fifty-one children with DLD and sixty-six TD children participated as part (...) of the HelSLI project in this cross-sectional study. The children were 4–6-year-old monolingual native Finnish speakers. They completed several tests of language and cognitive functioning, as well as new game-like tests of visual and auditory non-verbal serial STM. We used regression analyses to examine how serial STM moderates the effect of age on language. A non-verbal composite measure of serial visual and auditory STM moderated cross-sectional development of receptive language in the children with DLD. This moderation was not observed in the TD children. However, we found more rapid cross-sectional development of non-verbal serial STM in the TD children than in the children with DLD. The results suggest that children with DLD may be more likely to have compromised general serial STM processing and that superior non-verbal serial STM may be associated with better language acquisition in children with DLD. (shrink)

Future work is needed to establish that pure short-term memory is a coherent individual difference attribute that is separable from traditional compound short-term memory measures. Psychometric support for latent pure short-term memory capacity will provide an important starting point for future fine-grained analyses of the intrinsic factors that influence individual differences in math skills.

Miller (1956) summarized evidence that people can remember about seven chunks in short-term memory (STM) tasks. However, that number was meant more as a rough estimate and a rhetorical device than as a real capacity limit. Others have since suggested that there is a more precise capacity limit, but that it is only three to five chunks. The present target article brings together a wide variety of data on capacity limits suggesting that the smaller capacity limit is real. (...) Capacity limits will be useful in analyses of information processing only if the boundary conditions for observing them can be carefully described. Four basic conditions in which chunks can be identified and capacity limits can accordingly be observed are: (1) when information overload limits chunks to individual stimulus items, (2) when other steps are taken specifically to block the recoding of stimulus items into larger chunks, (3) in performance discontinuities caused by the capacity limit, and (4) in various indirect effects of the capacity limit. Under these conditions, rehearsal and long-term memory cannot be used to combine stimulus items into chunks of an unknown size; nor can storage mechanisms that are not capacity-limited, such as sensory memory, allow the capacity-limited storage mechanism to be refilled during recall. A single, central capacity limit averaging about four chunks is implicated along with other, noncapacity-limited sources. The pure STM capacity limit expressed in chunks is distinguished from compound STM limits obtained when the number of separately held chunks is unclear. Reasons why pure capacity estimates fall within a narrow range are discussed and a capacity limit for the focus of attention is proposed. Key Words: attention; enumeration; information chunks; memory capacity; processing capacity; processing channels; serial recall; short-term memory; storage capacity; verbal recall; working memory capacity. (shrink)

A symbolic computer model, employing the perceptual strategy, is presented for solving Tower of Hanoi problems. The model is calibrated—in terms of the number of problems solved, time taken, and number of moves made—to the performance of 20 normal subjects. It is then “lesioned” by increasing the decay rate of elements in working memory to model the performance of 20 patients with lesions to the prefrontal cortex. The model captures both the main effects of subject groups (patients and normal controls) (...) performance, and the subject groups (patients and normal controls) by problem difficulty interactions. This leads us to support the working memory hypothesis of frontal lobe functions, but for a narrow range of problems. (shrink)