Two experiments were run to demonstrate the presence of a partial reinforcement extinction effect (PREE) and a partial punishment effect (PPE) 4 weeks after training in a 1-trial/day procedure. In the PREE paradigm, two groups of animals were trained to run a straight alley for food reward; one group was rewarded on every trial (CRF), whereas the other was rewarded on only 50% of the trials (PRF). In the test phase, extinction, no reward was present (...) on any trial. Four weeks after the end of acquisition, and subsequent to 7 days of CRF retraining, PRF animals showed greater resistance to extinction than did their CRF controls. The PPE paradigm resembled the PREE paradigm in all aspects, except that mild electric foot shock, together with the food reward, was used instead of nonreward on 50% of the acquisition trials. In the test phase, all animals were shocked and given a food reward on every trial. Four weeks after the end of acquisition and retraining, the animals with previous experience of shock in acquisition showed greater resistance to punishment than did their controls. (shrink)

This commentary considers factors that may account for the inconsistency between the behavioral momentum formulation and the partial reinforcement extinction effect. The method of testing, the variability of the schedule, the nature of the response-contingency, and response effort are considered. Some applications to real-world problems are also discussed.

Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). (...) In this article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model. (shrink)

Studies suggest that reward and emotion are interdependent. However, there are discrepancies regarding the interaction between these variables. Some researchers speculate that the inconsistent findings may be due to different targets being used. Although reward and emotion both affect attention, it is not clear whether their impacts are independent. This study examined the impact of reward anticipation on emotion processing for different targets. A cue-target paradigm was used, and behavior and eye-tracking data were recorded in an emotion (...) or sex recognition task under the conditions of reward and non-reward anticipation. The results showed that when the target was related to the emotional attribute of the stimulus, the reward promoted the processing target information, thereby generating reward-oriented attention. When the target was unrelated to the emotional attributes of the stimulus, the reward did not promote the processing target information, and at the same time, individuals had negative emotional biases toward the emotional faces. The results revealed that, in addition to affecting the attention to emotional faces independently, the target regulated the promotion of reward anticipation to emotional attention and attention bias towards negative stimuli. (shrink)

We have analyzed many discrimination learning difficulties as reflecting multiple stimulus control topographies (SCTs). Nevin & Grace's analysis offers new variables to consider in the design of stimulus-control shaping procedures and cross-setting generalization of newly established behavior. A multiple-SCT perspective also suggests that fixed-trial discrimination procedures may offer advantages for reconciling momentum theory and partial reinforcement extinction effects.

In spite of these efforts in the 1920s and 1930s to initiate ongoing research on contraception, the subject of birth control remained a problem of concern primarily to the social activist rather than to the research scientist or practicing physician.80 In the 1930s, as has been shown, American scientists turned to the study of other aspects of reproductive physiology, while American physicians, anxious to eliminate the moral and medical dangers of contraception, only reluctantly accepted birth control as falling within their (...) professional domain.As a result, the problem of cheap, effective, and safe contraception was not solved by these earliest attempts. Consideration of the subject was initiated afresh by private philanthropy after World War II, sparked by a new wave of interest in population studies.81 Summarizing such efforts to support research in the reproductive sciences, a recent Ford Foundation study has noted: “To initiate and sustain serious research in the reproductive sciences has required for more than half a century concerted effort by interested individuals and private organizations, mainly from outside the mainstreams of the biomedical research community.”82The early laboratory research on chemical contraception described in this paper was but one important outcome of the concerted effort made by reformers in the 1920s to eliminate a variety of social problems thought to derive from excessive fertility. Scientific arguments and expertise were employed to advocate reform as well as to define the appropriate solution to such social problems. Scientists were recruited as advocates for the movement, but they were also employed as researchers in laboratory investigations sponsored by these same reformers.Sponsors of these early laboratory studies noted the difficulty of obtaining first-class investigators.83 The routine analyses necessary for such research, as well as the traditional scientific aversion to applied problems, provide only a partial explanation for this response. The real difficulty lay in recruiting investigators to a field that had previously been taboo. Once opened up — first as socially relevant, and finally as scientifically sound — there was much interest in this area, and the appeal to researchers of the scientific issues surrounding fertility and reproduction soon surpassed that of the reforming value of birth control.A survey of the kinds of experimental investigations sponsored by birth control advocates indicates the range of physiological problems explored by contraceptive research. The most definitive work was done on the efficacy and safety of spermicides, but the potential of other contraceptive methods was also examined. Investigators attempted to develop spermatoxins that would effectively immunize women against sperm, and they also tried to elucidate the mechanism of hormonal control of reproduction. In fact, speculations about the possible hormonal manipulation of fertility were expressed at the Seventh International Birth Control Conference held in Zurich in 1930.84In the 1920s, clinical studies were undertaken to assess the effectiveness of the various birth control methods. Laboratory investigators complemented this work by screening spermicides for safety and testing for their ability to kill sperm. There were a variety of birth control preparations on the market, but no one really knew whether these were effective or even safe.85 Although the physiology of other major organ systems was well advanced, the scientific study of reproductive physiology and contraceptive technology was clearly in its infancy in this period. Routine analyses simply could not be conducted, because the fundamental research establishing baselines had not yet been done. Scientists used this fact to redirect attention to basic research on reproduction.Laboratory research on contraception indicated important unexplored areas for physiological investigation. Social activists, who had encouraged prominent scientists to become interested in both the social value and the genetic implications of birth control, found these investigators revising the goals of their research. The biologists had formed their own network and had begun to seek out funding, reformulating the justification for sponsorship of further investigations. The eugenic motivations underlying these studies, which had initially made them theoretically attractive to biologists, were gradually eroded. Concern with “human evolution” ceded its place to interest in physiological mechanisms. Crew and others began to note that the use of biological theory to justify essentially political decisions had serious limitations. Biologists had become uncomfortable with those very arguments which had originally captured their interest. Recognition of the potential political abuse wrought by applying scientific principles to society was expressed by Crew just one year after the Zurich meeting. Referring to previous assessments of the role of sex in reproduction, he generalized: “In the past the biologist has justified feudalism, Manchester Liberalism, socialism and every other type of social organization and political programme by reference to selected biological phenomena.”86 By 1932 Crew had also begun to question the biological logic of regulated breeding, and had made it clear to his American sponsor that there was no simple correspondence between the practice of birth control and the genetic improvement of the human race.87Biologists further began to recognize, however, that although the hopeful genetic solution to human problems was probably an illusion, contraception still remained one tangible means to alleviate, human misery. Some laboratory scientists, like Crew, acknowledged the applicability of their own particular skills to this problem. For a few brief years, social needs and scientific goals were mutually supportive and closely intertwined. But as laboratory researchers gained interest in the study of reproduction and established their own priorities in this field, they temporarily withdrew from the arena of debate over birth control as an important mechanism for social reform.With the rise of Hitler, the genetic arguments for birth control rapidly lost their appeal. But by that time the scientific problem of how to achieve effective contraception had entered the professional consciousness. Both physicians and scientists began to be aware of birth control as a subject within their domain of expertise, although outside the principal focus of their research. Scientific discussion of birth control permanently altered from a question of justification to a problem of method: How could one achieve reliable and safe contraception? This had been Sanger's and Dickinson's goal from the beginning. Laboratory scientists had indeed been persuaded to undertake this work; this research had in turn affected biologists' perceptions of the whole field of reproductive physiology, encouraging further study of reproductive mechanisms. The promise of new knowledge provided for continued funding of this research, despite the caution by scientists that the social benefits would not be as immediate or as far-reaching as advocates and they themselves had first argued.The activities of birth control activists and their supporting agencies, and the financial backing of private contributors and foundations, notably the Rockefeller philanthropies, provided an important new stimulus to the development of research on the biology of reproduction in the late 1920s and early 1930s. Biologists were able to claim an enlarged realm of issues for scientific study through their activities as advocates and as investigators for the birth control movement. At the same time, they promised as-yet-undiscovered possibilities for regulating human reproduction once its physiology was understood. The knowledge and control that they promised lay in understanding the whole reproductive cycle — not merely in evaluating the toxic effects of presumptive spermicides.Chemical spermicides never summoned the interest of scientists as the contraceptive pill was to do, yet that research did reinforce the widespread perceptions of scientific research as essential to social reform. Spermicide investigations focused research efforts in reproductive biology by challenging traditional taboos, defining problems for further study, and providing laboratory investigators the opportunity to assert the social and scientific value of their own skills. Crew echoed this attitude as he observed in 1934:Man has turned from the adventurous conquest of his environment to the conquest of himself. To-day is the day of biological invention, eagerly used for the control of the undesirable and the unwanted. Sex and reproduction are no longer hedged around by myth and taboo; they are no longer accepted as mysteries that defy understanding. They are matters inviting examination and explanation; they are regarded as expressions of physico-chemical forces, the nature of which is to be displayed. It is accepted that when knowledge is sufficient, control will be absolute, and, though knowledge is not yet sufficient, readers... must be persuaded to the view that this will not always be so.88The synergism between reproductive biology and social needs has been temporary and sporadic, but recurring, since the 1920s. Scientific research programs have clearly been influenced by issues raised by public debate. Nevertheless, reproductive biologists have continued to assert their own professional goals. For the most part, they have rejected problems without inherent scientific interest and have spurned applied research except as it has had a direct bearing on current research themes. This attitude, apparent among American and British investigators in the 1930s, created the intellectual context for the invention and acceptance of the technically sophisticated oral contraceptive pill. It did not foster the production and improvement of the simple, safe, effective, and cheap vaginal contraceptive desired by early birth control advocates. (shrink)

Self-relevance has been demonstrated to impair instrumental learning. Compared to unfamiliar symbols associated with a friend, analogous stimuli linked with the self are learned more slowly. What is not yet understood, however, is whether this effect extends beyond arbitrary stimuli to material with intrinsically meaningful properties. Take, for example, stimulus valence an established moderator of self-bias. Does the desirability of to-be-learned material influence self-learning? Here, in conjunction with computational modelling (i.e. Reinforcement Learning Drift Diffusion Model analysis), a probabilistic (...) selection task was used to establish if and how stimulus valence (i.e. desirable/undesirable posters) impacts the acquisition of knowledge relating to object-ownership (i.e. owned-by-self vs. owned-by-friend). Several interesting results were observed. First, undesirable posters were learned more rapidly for self compared to friend, an effect that was reversed for desirable posters. Second, learning rates were accompanied by associated differences in reward sensitivity toward desirable and undesirable choice selections as a function of ownership. Third, decisional caution was greater for self-relevant (vs. friend relevant) responses. Collectively, these findings inform understanding of self-function and how valence and stimulus relevance mutually influence probabilistic learning. (shrink)

Profit Sharing is one of the reinforcement learning methods. An agent, as a learner, selects an action with a state-action value and receives rewards when it reaches a goal state. Then it distributes receiving rewards to state-action values. This paper discusses how to set the initial value of a state-action value. A distribution function ƒ( x ) is called as the reinforcement function. On Profit Sharing, an agent learns a policy by distributing rewards with the reinforcement function. (...) On Markov Decision Processes (MDPs), the reinforcement function ƒ( x ) = 1/ L x is useful, and on Partially Observable Markov Decision Processes (POMDPs), ƒ( x ) = 1/ L w is useful, where L is the sufficient number of rules at each state, and W is the length of an episode. If episodes are always long, the value of the reinforcement function is little. So the differences of rule values become little, and the agent learns little by using the roulette selection as an action selection. This problem is called as Learning Speed Problem. If the value of the reinforcement function for an action is very higher than its state-action value, an agent will not select other action. There is a problem when its action is not a optimal action. This problem is called as Past Experiences Problem. This paper shows that both Learning Speed Problem and Past Experiences Problem are caused by the bad setting between the initial values of a state-action values and the function values of a reinforcement function. We propose how to set the initial values of a state-action values at each state. The experiment shows that an agent can learn correctly even if the length of episode is large. And shows the effectiveness on both MDPs and POMDPs. Our proposed method focuses on the initialization of state-action values and does not limit reinforcement functions. So it can apply to any reinforcement function. (shrink)

Rewards are reinforcement mechanisms that organizations use to shape desirable employee behaviors. However, rewards may also have unintended consequences, such as building expectations for receiving extra benefits and weakening employee barriers to unethical acts. This article investigates the dark side of the reward–behavior association, and exploring what is referred to as the reward–theft parity effect (RTPE). The authors hypothesize that receiving rewards induces a corresponding type of theft. In Study 1, survey results (n = 634) show initial (...) support for the RTPE between rewards and same‐category theft, and that perceptions of procedural justice attenuate this association. Study 2 involved a 2 (reward) × 2 (theft opportunity) between‐subjects experimental design (n = 300) and provides further support for the RTPE by revealing greater theft under a reward–theft parity condition than under a reward–theft disparity condition. Study 3 replicates these findings in a field experiment (n = 285) and finds that the RTPE is exacerbated when the reward–behavior task is perceived as unjust. Collectively, these findings provide new theoretical insights while also giving managers a more complete understanding of the dark side of behavioral reinforcement mechanisms. (shrink)