Self-relevance has been demonstrated to impair instrumental learning. Compared to unfamiliar symbols associated with a friend, analogous stimuli linked with the self are learned more slowly. What is not yet understood, however, is whether this effect extends beyond arbitrary stimuli to material with intrinsically meaningful properties. Take, for example, stimulus valence an established moderator of self-bias. Does the desirability of to-be-learned material influence self-learning? Here, in conjunction with computational modelling (i.e. Reinforcement Learning Drift Diffusion Model analysis), a probabilistic selection (...) task was used to establish if and how stimulus valence (i.e. desirable/undesirable posters) impacts the acquisition of knowledge relating to object-ownership (i.e. owned-by-self vs. owned-by-friend). Several interesting results were observed. First, undesirable posters were learned more rapidly for self compared to friend, an effect that was reversed for desirable posters. Second, learning rates were accompanied by associated differences in reward sensitivity toward desirable and undesirable choice selections as a function of ownership. Third, decisional caution was greater for self-relevant (vs. friend relevant) responses. Collectively, these findings inform understanding of self-function and how valence and stimulus relevance mutually influence probabilistic learning. (shrink)

Shared intention normally leads to joint action. It does this, it is commonly said, only because it is a characteristically stable phenomenon, a phenomenon that tends to persist from the time it is formed until the time it is fulfilled. However, the issue of what the stability of shared intention comes down to remains largely undertheorized. My aim in this paper is to remedy this shortcoming. I argue that shared intention is a source of moral and epistemic reasons, that responsiveness (...) to such reasons on the part of each individual reinforces her own relevant attitudes, and that this enhances the stability of the shared intention as a whole. We thus discover an important self-reinforcing and stability-enhancing mechanism at the heart of shared intention and action. It follows from this proposal that the psychological commitment that an individual exhibits in joint action is normally a function not only of her own participatory intention, as often maintained, but also of her endorsement of the reasons created by her group’s shared intention to so act. (shrink)

Autonomous underwater vehicles are widely used to accomplish various missions in the complex marine environment; the design of a control system for AUVs is particularly difficult due to the high nonlinearity, variations in hydrodynamic coefficients, and external force from ocean currents. In this paper, we propose a controller based on deep reinforcement learning in a simulation environment for studying the control performance of the vectored thruster AUV. RL is an important method of artificial intelligence that can learn behavior through trial-and-error (...) interactions with the environment, so it does not need to provide an accurate AUV control model that is very hard to establish. The proposed RL algorithm only uses the information that can be measured by sensors inside the AUVs as the input parameters, and the outputs of the designed controller are the continuous control actions, which are the commands that are set to the vectored thruster. Moreover, a reward function is developed for deep RL controller considering different factors which actually affect the control accuracy of AUV navigation control. To confirm the algorithm’s effectiveness, a series of simulations are carried out in the designed simulation environment, which is a method to save time and improve efficiency. Simulation results prove the feasibility of the deep RL algorithm applied to the control system for AUV. Furthermore, our work also provides an optional method for robot control problems to deal with improving technology requirements and complicated application environments. (shrink)

This paper addresses weighting and partitioning in complex reinforcement learning tasks, with the aim of facilitating learning. The paper presents some ideas regarding weighting of multiple agents and extends them into partitioning an input/state space into multiple regions with di erential weighting in these regions, to exploit di erential characteristics of regions and di erential characteristics of agents to reduce the learning complexity of agents (and their function approximators) and thus to facilitate the learning overall. It analyzes, in reinforcement (...) learning tasks, di erent ways of partitioning a task and using agents selectively based on partitioning. Based on the analysis, some heuristic methods are described and experimentally tested. We nd that some o -line heuristic methods performed the best, signi cantly better than single-agent models. (shrink)

forcement learning algorithms that generate only reactive policies and existing probabilistic planning algorithms that requires a substantial amount of a priori knowledge in order to plan we devise a two stage bottom up learning to plan process in which rst reinforcement learn ing dynamic programming is applied without the use of a priori domain speci c knowledge to acquire a reactive policy and then explicit plans are extracted from the learned reactive policy Plan extraction is based on a beam search (...) algorithm that performs temporal projection in a restricted fashion guided by the value functions resulting from reinforcement learn ing dynamic programming.. (shrink)

The reinforcement sensitivity theory proposes that neurobiological systems mediate protective and appetitive behaviours and the functioning of these systems is associated to personality traits. In this manner, the RST is a link between neuroscience, behaviour, and personality. The theory evolved to the present revised version describing three systems: fight-flight-freezing, behavioural approach/activation, and behavioural inhibition. However, the most widely available measure of the theory, the BIS/BAS scales, only investigates two systems. Using a large longitudinal community survey, we found that the BIS/BAS (...) scales can be re-structured to investigate the three systems of the theory with a BIS scale, three BAS scales, and a separate fight-flight-freezing system scale. The re-structured scales were age, sex, and longitudinally invariant, and associations with personality and mental health measures followed theoretical expectations and previously published associations. The proposed framework can be used to investigate behavioural choices influencing physical and mental health and bridge historical with contemporary research. (shrink)

The absence of a clear influence of the responses modified by new connections created by LTP on the development of these connections casts doubt on an essential role of LTP in learning and memory formation without any association with reinforcement. The evidence for the involvement of the monoaminergic systems in synaptic potentiation in the cerebral cortex during learning is adduced, and their role in reinforcement system function is discussed.

Textbooks function as an important resource for teaching and learning of mathematics at the school level across the world. At least at the primary grades the contents of textbooks are situated in the larger society around the learners, in order that the learners can relate to what is taught to them. This opens the possibility for textbooks to uncritically reinforce the prevailing stereotypes or use the opportunity textbook provide to creatively break the stereotypes. Mathematics education research has engaged with (...) the question of gender stereotypes in mathematics textbooks which has had an impact how gender figures in textbooks. However, gender is neither a binary nor monolithic. In the Indian context, gender is not the only social hierarchy that operates. The National Curriculum Framework 2005(hence forth referred to as NCF 2005) addresses the question of prevailing stereotypes about children from social margins and says care must be taken to ensure that the curriculum, textbooks and classroom interaction do not reinforce the stereotypes. Moreover, in the last two decades transgender people have been able to demand recognition and acceptance at least in the higher educational spaces as transgender people. Given these, it would be important to understand how textbooks reflect the changes and demands. This paper analyses the content of the mathematics textbooks developed by the State Council of Education Research and Training (SCERT) of one Indian state, namely Kerala, to understand how they represent gender, caste, class and religious differences and to investigate if the textbooks are inclusive of the disabled learners. Based on the content analysis of the textbooks the paper argues that even as the textbooks try to ensure representation of both girls and boys in the pictures and word problems and make an attempt to be inclusive of different religions and marginalised cultures ( for example by incorporating the picture of Theyyam, which uses an art form of the marginalised people in Kerala) in a textbook, they end up strengthening the existing gender, class, religious stereotypes. It also draws attention to the complete absence of disabled children in the textbooks. (shrink)

It seems that the first two decades of the twenty first century demonstrate political mythology to be still functioning in the political life of the West. In this context, it is interesting to view the recent publications of Hans Blumenberg’s Nachlass: Präfiguration and Rigorismus der Wahrheit, as they reveal unpredicted complications for the interpretation of his philosophy of myth as well as of his political stances. They also evoke some more general questions concerning the role of myth in our contemporary (...) political life. The aim of this article is to present the paradoxes connected with the posthumously published Blumenberg critique of Hannah Arendt and to situate it in the wider context of twentieth century political thought, specifically the work of Sorel, Schmitt, Rosenberg and Cassirer. It is also to point to more general ethical and political ambiguities connected with the problem of political mythology in the present. (shrink)

We study a joint pricing and inventory control problem for perishables with positive lead time in a finite horizon periodic-review system. Unlike most studies considering a continuous density function of demand, in our paper the customer demand depends on the price of current period and arrives according to a homogeneous Poisson process. We consider both backlogging and lost-sales cases, and our goal is to find a simultaneously ordering and pricing policy to maximize the expected discounted profit over the planning (...) horizon. When there is no fixed ordering cost involved, we design a deep reinforcement learning algorithm to obtain a near-optimal ordering policy and show that there are some monotonicity properties in the learned policy. We also show that our deep reinforcement learning algorithm achieves a better performance than tabular-based Q-learning algorithms. When a fixed ordering cost is involved, we show that our deep reinforcement learning algorithm is effective and efficient, under which the problem of “curse of dimension” is circumvented. (shrink)

Artificial neural networks, in interaction with fuzzy logic, genetic algorithms, and fuzzy neural networks, represent an example of a modern interdisciplinary field, especially when it comes to solving certain types of engineering problems that could not be solved using traditional modeling methods and statistical methods. They represent a modern trend in practical developments within the prognostic modeling field and, with acceptable limitations, enjoy a generally recognized perspective for application in construction. Results obtained from numerical analysis, which includes analysis of the (...) behavior of reinforced concrete elements and linear structures exposed to actions of standard fire, were used for the development of a prognostic model with the application of fuzzy neural networks. As fire resistance directly affects the functionality and safety of structures, the significance which new methods and computational tools have on enabling quick, easy, and simple prognosis of the same is quite clear. This paper will consider the application of fuzzy neural networks by creating prognostic models for determining fire resistance of eccentrically loaded reinforced concrete columns. (shrink)

The generation of the pitch control signal in a wind turbine (WT) is not straightforward due to the nonlinear dynamics of the system and the coupling of its internal variables; in addition, they are subjected to the uncertainty that comes from the random nature of the wind. Fuzzy logic has proved useful in applications with changing system parameters or where uncertainty is relevant as in this one, but the tuning of the fuzzy logic controller (FLC) parameters is neither straightforward nor (...) an easy task. On the other hand, reinforcement learning (RL) allows systems to automatically learn, and this capability can be exploited to tune the FLC. In this work, a WT pitch control architecture that uses RL to tune the membership functions and scale the output of a fuzzy controller is proposed. The RL strategy calculates the fuzzy controller gains in order to reduce the output power error of the WT according to the wind speed. Different reward mechanisms based on the output power error have been considered. Simulation results with different wind profiles show that this architecture performs better (123.7 W) in terms of power errors than an FLC without RL (133.2 W) or a simpler PID (208.8 W). Even more, it provides a smooth response and outperforms other hybrid controllers such as RL-PID and radial basis function neural network control. (shrink)

This paper addresses automatic partitioning in complex reinforcement learning tasks with multiple agents, without a priori domain knowledge regarding task structures. Partitioning a state/input space into multiple regions helps to exploit the di erential characteristics of regions and di erential characteristics of agents, thus facilitating learning and reducing the complexity of agents especially when function approximators are used. We develop a method for optimizing the partitioning of the space through experience without the use of a priori domain knowledge. The (...) method is experimentally tested and compared to a number of other algorithms. As expected, we found that the multi-agent method with automatic partitioning outperformed single-agent learning. (shrink)

Reinforcement learning systems usually assume that a value function is defined over all states (or state-action pairs) that can immediately give the value of a particular state or action. These values are used by a selection mechanism to decide which action to take. In contrast, when humans and animals make decisions, they collect evidence for different alternatives over time and take action only when sufficient evidence has been accumulated. We have previously developed a model of memory processing that includes (...) semantic, episodic and working memory in a comprehensive architecture. Here, we describe how this memory mechanism can support decision making when the alternatives cannot be evaluated based on immediate sensory information alone. Instead we first imagine, and then evaluate a possible future that will result from choosing one of the alternatives. Here we present an extended model that can be used as a model for decision making that depends on accumulating evidence over time, whether that information comes from the sequential attention to different sensory properties or from internal simulation of the consequences of making a particular choice. We show how the new model explains both simple immediate choices, choices that depend on multiple sensory factors and complicated selections between alternatives that require forward looking simulations based on episodic and semantic memory structures. In this framework, vicarious trial and error is explained as an internal simulation that accumulates evidence for a particular choice. We argue that a system like this forms the “missing link” between more traditional ideas of semantic and episodic memory, and the associative nature of reinforcement learning. (shrink)

With the advent of the era of artificial intelligence, deep reinforcement learning has achieved unprecedented success in high-dimensional and large-scale artificial intelligence tasks. However, the insecurity and instability of the DRL algorithm have an important impact on its performance. The Soft Actor-Critic algorithm uses advanced functions to update the policy and value network to alleviate some of these problems. However, SAC still has some problems. In order to reduce the error caused by the overestimation of SAC, we propose a new (...) SAC algorithm called Averaged-SAC. By averaging the previously learned action-state estimates, it reduces the overestimation problem of soft Q-learning, thereby contributing to a more stable training process and improving performance. We evaluate the performance of Averaged-SAC through some games in the MuJoCo environment. The experimental results show that the Averaged-SAC algorithm effectively improves the performance of the SAC algorithm and the stability of the training process. (shrink)

Sagvolden, Johansen, Aase, and Russell (Sagvolden et al.) examine attention-deficit/hyperactivity disorder (ADHD) at levels of analysis ranging from neurotransmitters to behavior. At the behavioral level they attribute aspects of ADHD to anomalies of delay-of-reinforcement gradients. With a normal gradient, responses followed after a long delay by a reinforcer may share in the effects of that reinforcer; with a diminished or steepened gradient they may fail to do so. Steepened gradients differentially select rapidly emitted responses (hyperactivity), and they limit the effectiveness (...) with which extended stimuli become conditioned reinforcers, so that observing behavior is less well maintained (attention deficit). Impulsiveness also follows from steepened gradients, which increase the effectiveness of smaller, more immediate consequences relative to larger, more delayed ones. Individuals who vary in the degree to which their delay gradients are steepened will show different balances between hyperactivity and attention deficit. Given the range of ADHD phenomena addressed, it may be unnecessary to appeal to additional behavioral processes such as extinction deficit. Extinction deficit is more likely a derivative of attention deficit, in that failure to attend to stimuli differentially correlated with extinction should slow its progress. The account suggests how relatively small differences in delay gradients early in development might engender behavioral interactions leading to very large differences later on. The steepened gradients presumably originate in properties of neurotransmitter function, but behavioral interventions that use consistently short delays of reinforcement to build higher-order behavioral units as a scaffolding to support complex cognitive and social skills may nonetheless be feasible. Key Words: ADHD; attention deficit; delay gradient; exponential decay; extinction deficit; hyperactivity; impulsiveness; intervention; observing responses; self-control. Footnotes1000 Jeffrey Gray was the BBS Editor for this treatment. When he accepted Sagvolden et al. for publication, he invited Catania, who was one of the reviewers of the original submission, to prepare a precommentary. Com-mentators were then invited to respond to the Sagvolden et al. article, to the Catania precommentary, or to both. (shrink)

The large scale, time varying, and diversification of physically coupled networked infrastructures such as power grid and transportation system lead to the complexity of their controller design, implementation, and expansion. For tackling these challenges, we suggest an online distributed reinforcement learning control algorithm with the one-layer neural network for each subsystem or called agents to adapt the variation of the networked infrastructures. Each controller includes a critic network and action network for approximating strategy utility function and desired control law, (...) respectively. For avoiding a large number of trials and improving the stability, the training of action network introduces supervised learning mechanisms into reduction of long-term cost. The stability of the control system with learning algorithm is analyzed; the upper bound of the tracking error and neural network weights are also estimated. The effectiveness of our proposed controller is illustrated in the simulation; the results indicate the stability under communication delay and disturbances as well. (shrink)

In the metaphor of behavioral momentum, the rate of a free operant in the presence of a discriminative stimulus is analogous to the velocity of a moving body, and resistance to change measures an aspect of behavior that is analogous to its inertial mass. An extension of the metaphor suggests that preference measures an analog to the gravitational mass of that body. The independent functions relating resistance to change and preference to the conditions of reinforcement may be construed as (...) convergent measures of a single construct, analogous to physical mass, that represents the effects of a history of exposure to the signaled conditions of reinforcement and that unifies the traditionally separate notions of the strength of learning and the value of incentives. Research guided by the momentum metaphor encompasses the effects of reinforcement on response rate, resistance to change, and preference and has implications for clinical interventions, drug addiction, and self-control. In addition, its principles can be seen as a modern, quantitative version of Thorndike's (1911) Law of Effect, providing a new perspective on some of the challenges to his postulation of strengthening by reinforcement. Key Words: behavioral momentum; clinical interventions; drug addiction; preference; reinforcement; resistance to change; response strength; self-control. (shrink)

Stimuli that resemble humans, but are not perfectly human-like, are disliked compared to distinctly human and nonhuman stimuli. Accounts of this “Uncanny Valley” effect often focus on how changes in human resemblance can evoke different emotional responses. We present an alternate account based on the novel hypothesis that the Uncanny Valley is not directly related to ‘human-likeness’ per se, but instead reflects a more general form of stimulus devaluation that occurs when inhibition is triggered to resolve conflict between competing (...) stimulus-related representations. We consider existing support for this inhibitory-devaluation hypothesis and further assess its feasibility through tests of two corresponding predictions that arise from the link between conflict-resolving inhibition and aversive response: 1) that the pronounced disliking of Uncanny-type stimuli will occur for any image that strongly activates multiple competing stimulus representations, even in the absence of any human-likeness, and 2) that the negative peak of an ‘Uncanny Valley’ should occur at the point of greatest stimulus-related conflict and not (in the presence of human-likeness) always closer to the ‘human’ end of a perceptual continuum. We measured affective responses to a set of line drawings representing nonhuman animal-animal morphs, in which each continuum midpoint was a bistable image (Exp. 1), as well as to sets of human-robot and human-animal computer-generated morphs (Exp. 2). Affective trends depicting classic Uncanny Valley functions occurred for all continua, including the nonhuman stimuli. Images at continua midpoints elicited significantly more negative affect than images at endpoints, even when the continua included a human endpoint. This illustrates the feasibility of the inhibitory-devaluation hypothesis and the need for further research into the possibility that the strong dislike of Uncanny-type stimuli reflects the negative affective consequences of cognitive inhibition. (shrink)

When consciousness is examined using subjective ratings, the extent to which processing is conscious or unconscious is often estimated by calculating task performance at the subjective threshold or by calculating the correlation between accuracy and awareness. However, both these methods have certain limitations. In the present article, we propose describing task accuracy and awareness as functions of stimulus intensity as suggested by Koch and Preuschoff . The estimated lag between the curves describes how much stimulus intensity must increase (...) for awareness to change proportionally as much as accuracy and the slopes of the curves are used to assess how fast accuracy and awareness increases and whether awareness is dichotomous. The method is successfully employed to assess consciousness characteristics on data from four different awareness scales. (shrink)

Perceptual processes differ from cognitive, this paper argues, in functioning to be causally controlled by proximal stimuli, and being modular, at least in a modest sense that excludes their being isotropic in Jerry Fodor's sense. This claim agrees with such theorists as Jacob Beck and Ben Phillips that a function of stimulus-control is needed for perceptual status. In support of this necessity claim, I argue, inter alia, that E.J. Green's recent architectural account misclassifies processes deploying knowledge of grammar (...) as perceptual. _Pace_ Beck and Phillips, however, I argue a function of stimulus-control is insufficient for perceptual as opposed to cognitive status. One consideration in favour of such insufficiency, noted but (I argue) not convincingly rebutted by these theorists, concerns perpetually grounded demonstrative thought. Two other considerations trade on the fact that a function of stimulus-control can arise not from blind nature but intentional design or social institutions, where so-functioning processes may but need not be perceptual. I offer two cases where such processes are cognitive, viz. skilful play-by-play announcing of ongoing events, and voluntary visualizing of ongoing events under the guidance of apt play-by-play announcements, dubbed announcement-driven visualizing (ADV). The cognitive status of these three diverse phenomena cannot be explained by an absence of a perception-like representational format or content (for ADV has such) or by a presence of personal-level mental states causally mediating between stimuli and outputs (for perception has such). A bettter explanation invokes, I argue, the non-modular character of the generating process. (shrink)

We have analyzed many discrimination learning difficulties as reflecting multiple stimulus control topographies (SCTs). Nevin & Grace's analysis offers new variables to consider in the design of stimulus-control shaping procedures and cross-setting generalization of newly established behavior. A multiple-SCT perspective also suggests that fixed-trial discrimination procedures may offer advantages for reconciling momentum theory and partial reinforcement extinction effects.

In this paper it is argued that certain stimulus-response learning models which are adequate to represent finite automata (acceptors) are not adequate to represent noninitial state input-output automata (transducers). This circumstance suggests the question whether or not the behavior of animals if satisfactorily modelled by automata is predictive. It is argued in partial answer that there are automata which can be explained in the sense that their transition and output functions can be described (roughly, Hempel-type covering law explanation) while (...) their behaviors are in principle not predictable short of possession of their complete histories or of information concerning present internal states by indirect observation. (shrink)

Shors & Matzel propose that hippocampal LTP increases the effective salience of discrete external stimuli and thereby facilitates the induction of memories at distant places. In line with this suggestion, a neural network model of associative learning and hippocampal function assumes that LTP increases hippocampal error signals to the cortex, thereby facilitating stimulus configuration in association cortex. Computer simulations show that under these assumptions the model correctly describes the effect of LTP induction and blockade in classical discriminations and (...) place learning. (shrink)

Steady-state visual evoked potentials are commonly used for functional objective diagnostics. In general, the main response at the stimulation frequency is used. However, some studies reported the main response at the second harmonic of the stimulation frequency. The aim of our study was to analyze the influence of the stimulus design on the harmonic components of ssVEPs. We studied 22 subjects using a circular layout. At a given eccentricity, the stimulus was presented according to a 7.5 Hz square (...) wave with 50% duty cycle. To analyze the influence of the stimulus eccentricity, a background luminance of 30 cd/m2 was added to suppress foveal stray light effects; to analyze the influence of simultaneous foveal and peripheral stimulations, stimulations are performed without stray light suppression. For statistical analysis, medians M of the amplitude ratios for amplitudes at the second harmonic to the first harmonic and the probability of the occurrence of the main response at the second harmonic P are calculated. For stimulations with foveal stray light suppression, the medians were M0–1.6° = 0.45, M1.6–3.5° = 0.45, M3.5–6.4° = 0.76, M6.4–10.9° = 0.72, and M10.9–18° = 0.48, and the probabilities were P0–1.6° = 0.05, P1.6–3.5° = 0.05, P3.5–6.4° = 0.32, P6.4–10.9° = 0.29, and P10.9–18° = 0.30. For stimulations without foveal stray light suppression, the medians M were M0–1.6° = 0.29, M1.6–3.5° = 0.37, M3.5–6.4° = 0.98, M6.4–10.9° = 1.08, and M10.9–18° = 1.24, and the probabilities were P0–1.6° = 0.09, P1.6–3.5° = 0.05, P3.5–6.4° = 0.50, P6.4–10.9° = 0.55, and P10.9–18° = 0.55. In conclusion, the stimulus design has an influence on the harmonic components of ssVEPs. An increase in stimulation eccentricity during extrafoveal stimulation leads to a transition of the main response to the second harmonic. The effect is enhanced by a simultaneous foveal stimulation. (shrink)