According to the reward-prediction error hypothesis of dopamine, the phasic activity of dopaminergic neurons in the midbrain signals a discrepancy between the predicted and currently experienced reward of a particular event. It can be claimed that this hypothesis is deep, elegant and beautiful, representing one of the largest successes of computational neuroscience. This paper examines this claim, making two contributions to existing literature. First, it draws a comprehensive historical account of the main steps that led to the formulation and subsequent (...) success of the RPEH. Second, in light of this historical account, it explains in which sense the RPEH is explanatory and under which conditions it can be justifiably deemed deeper than the incentive salience hypothesis of dopamine, which is arguably the most prominent contemporary alternative to the RPEH. (shrink)

1. Introduction 2. Reward-Guided Decision Making 3. Content in the Model 4. How to Deflate a Metarepresentational Reading Proust and Carruthers on metacognitive feelings 5. A Deflationary Treatment of RPEs? 5.1 Dispensing with prediction errors 5.2 What is use of the RPE focused on? 5.3 Alternative explanations—worldly correlates 5.4 Contrast cases 6. Conclusion Appendix: Temporal Difference Learning Algorithms.

University of Oslo and Norwegian School of Economics and Business Administration, Norway We look at how one should reward effort without rewarding talent. One way to approach this issue is to ask how an increase in one individual's effort should be allowed to affect the post-tax income of others. The article provides characterizations of three main classes of redistribution mechanism on the basis of how these answer this question. Key Words: reward • effort • responsibility • equal opportunity • distributive (...) justice. (shrink)

Systems of logico-probabilistic reasoning characterize inference from conditional assertions that express high conditional probabilities. In this paper we investigate four prominent LP systems, the systems _O, P_, _Z_, and _QC_. These systems differ in the number of inferences they licence _. LP systems that license more inferences enjoy the possible reward of deriving more true and informative conclusions, but with this possible reward comes the risk of drawing more false or uninformative conclusions. In the first part of the paper, we (...) present the four systems and extend each of them by theorems that allow one to compute almost-tight lower-probability-bounds for the conclusion of an inference, given lower-probability-bounds for its premises. In the second part of the paper, we investigate by means of computer simulations which of the four systems provides the best balance of reward versus risk. Our results suggest that system _Z_ offers the best balance. (shrink)

Reward and punishment sensitivity seem important traits in understanding behavior in general and psychopathology in particular. Though the definitions used for reward and punishment sensitivity differentiate between responsivity and motivation, the measures thus far used to assess these constructs do not. Further, specificity of the type of reward and punishment in questionnaires might result in measurement bias especially when examining the relationship with psychopathology. Therefore, we developed a stimulus-independent multidimensional questionnaire of reward and punishment sensitivity that differentiates between responsivity and (...) motivation. This study addresses the psychometric qualities of this newly developed reward and punishment responsivity and motivation questionnaire. On the basis of exploratory ordinal factor analysis that was used to examine the quality of the initial pool of 39 items, the number of items was reduced to 18. Confirmatory ordinal factor analysis on the remaining items in an independent sample supported a 18-item four-factor model, and showed acceptable to good internal reliability. The relationship between the subscales of the RPRM-Q and often used questionnaires was examined in the combined sample, which showed some first support for the ability of the new questionnaire to differentiate between responsivity and motivation to approach/avoid. The findings indicate that the RPRM-Q might be a helpful instrument to further test the relevance of punishment and reward sensitivity in psychopathology. (shrink)

In philosophy, the textbook case for the discussion of human motivation is the examination (and almost always, the refutation) of psychological egoism. The arguments have become part of the folklore of our tribe, from their inclusion in countless introductory texts. [...] One of my central aims has been to define the issues empirically, so we do not just settle them by definition. Although I am inclined at present to put my bets on the reward-event theory, with its internalism, monism, and (...) causal primacy of satisfaction, I think we are very far from knowing enough to settle these questions concerning motivation, human or otherwise. The winds of science will blow where they may. In the meantime, we can be a bit more circumspect about what we put in our tribal folklore. (shrink)

Based on brain imaging findings, we present a model according to which addiction emerges as an imbalance in the information processing and integration among various brain circuits and functions. The dysfunctions reflect (a) decreased sensitivity of reward circuits, (b) enhanced sensitivity of memory circuits to conditioned expectations to drugs and drug cues, stress reactivity, and (c) negative mood, and a weakened control circuit. Although initial experimentation with a drug of abuse is largely a voluntary behavior, continued drug use can eventually (...) impair neuronal circuits in the brain that are involved in free will, turning drug use into an automatic compulsive behavior. The ability of addictive drugs to co‐opt neurotransmitter signals between neurons (including dopamine, glutamate, and GABA) modifies the function of different neuronal circuits, which begin to falter at different stages of an addiction trajectory. Upon exposure to the drug, drug cues or stress this results in unrestrained hyperactivation of the motivation/drive circuit that results in the compulsive drug intake that characterizes addiction. (shrink)

The audit profession’s technical and ethical standards require the application of professional skepticism throughout the financial statement audit process, as auditor skepticism is essential for detecting financial statement fraud and protecting the investing public. However, recent research suggests that audit supervisors often punish staff for exercising skepticism, presenting auditors with an ethical conflict between acting in their own self-interest and acting in a way that improves audit quality and protects the public. This research also suggests that supervisors who reward appropriate (...) skeptical behavior, regardless of the outcome, appear to develop staff that are more likely to detect and convey fraud red flags to their superiors. Building on this research, we use a case-based survey to identify the characteristics of audit supervisors (audit seniors and managers) who are more likely to reward appropriate skepticism, even if it ultimately does not identify a misstatement. We find that trait skepticism, especially suspending one’s judgment, positively drives the evaluations of professional skepticism in our setting. Also, we observe that when supervisors believe that their own audit partner will view the skepticism favorably, they “pay it forward” by rewarding their own staff who engage in skepticism. Our findings identify the characteristics that audit firms may want to develop and foster in auditors rising to supervisory levels. (shrink)

Research on academic pay finds an unexplained gender pay gap that has not fully dissolved over time and that appears to increase with years of experience. In this study, I consider how role congruity bias contributes to this pay gap. Bias is more likely to manifest in a context where there is some ambiguity about performance and where stereotypes are stronger. I predict that bias in the attribution of credit for coauthored research leads to lower returns to research for female (...) professors. To test this prediction, I use a sample of Canadian faculty in accounting, where research is typically coauthored, where females are underrepresented at the most senior ranks, and where many universities evaluate merit in research, teaching, and service to determine salary increases. In regressions of salary on individual and institutional determinants of salary, I find that women earn marginally less than men. However, the pay gap is only evident for women who publish in a selective list of journals; for the subset of faculty with no publications from this list, there are no significant differences in salary. For researching faculty, the pay gap relates specifically to research productivity. While women publish less on average than men, the returns to their research are also lower. In particular, the relation between the individual’s research ranking and salary is significantly lower for women who publish a higher proportion of their work with men, than for all other faculty. Additional analysis of salary and coauthor patterns confirms that women receive significantly less credit for coauthored articles they publish with men than those they publish with other women but that no similar variations in reward are evident for men across publishing patterns. These findings suggest bias in the attribution of credit for coauthored research in the determination of salary, consistent with role congruity theory, and provide an important potential explanation for why salaries for women vary systematically from those of men even after considering productivity. Providing lower rewards for equal work represents a continuing ethical issue in academe and compounds the challenges women already facing in the profession. (shrink)

When faced with intertemporal choices, which have consequences that unfold over time, we often discount the future, preferring smaller immediate rewards often at the expense of long-term benefits. How psychologically connected one feels to one’s future self-influences such temporal discounting. Psychological connectedness consists in sharing psychological properties with past or future selves, but connectedness comes in degrees. If one feels that one is not psychologically connected to one’s future self, one views that self like a different person and is (...) less likely to wait for the future reward. Increasing perceived psychological connectedness to one’s future self may lead to more far-sighted decisions. Episodic prospection may help in this regard. Episodic prospection is our ability to ‘pre-experience’ the future by mentally simulating it, drawing on information from episodic memory and other sources. Episodic memory and prospection are thought to involve a special form of consciousness, which underpins the capacity to appreciate the connection between one’s past, present, and future selves. Simulating the future self through prospection may increase felt psychological connectedness and support future-oriented decision-making. Yet this is where a puzzle arises. The imagery of episodic memory and prospection is perspectival: often one views the visualised scenario from a detached perspective, seeing oneself from-the-outside as if viewing another person. The aim of this paper is to characterise how the perspectival imagery of prospection relates to psychological connectedness, and to show that even though such imagery involves a detached perspective it can still be used to help reward one’s future self. (shrink)

The brain and emotion treats emotions as states elicited by reinforcers (reward or punishment), but it is unclear how this view can do justice to the diversity of emotions. It is also unclear how such a view distinguishes emotions from states such as hunger and thirst. A complementary approach to understanding emotions may begin by considering emotions as aspects of social cognition.

Rewards research typically examines how well incentives increase employees' productivity. By comparison, research in business ethics research focuses more on employees' ethical behavior than on their productivity per se. Yet, despite the bounty of literature in these two areas, little research specifically (a) links incentives to (un)ethical behavior and (b) focuses on relationships other than that between the employee-employer. This article reviews this neglect in detail, urges that future research address these gaps, and proposes a diagnostic model for use (...) by scholars to guide this future research and by practitioners to implement rewards in their organizations. (shrink)

In the absence of explicit punitive sanctions, why do individuals voluntarily participate in intergroup warfare when doing so incurs a mortality risk? Here we consider the motivation of individuals for participating in warfare. We hypothesize that in addition to other considerations, individuals are incentivized by the possibility of rewards. We test a prediction of this “cultural rewards war-risk hypothesis” with ethnographic literature on warfare in small-scale societies. We find that a greater number of benefits from warfare is associated (...) with a higher rate of death from conflict. This provides preliminary support for the relationship between rewards and participation in warfare. (shrink)

One of the challenges in the motivation literature is examining the simultaneous effect of different motivational mechanisms on overall motivation and performance. The motivational congruence theory addresses this by stipulating that different motivational mechanisms can reinforce each other if they have similar effects on the perceived locus of causality. Reward salience and choice are two motivational mechanisms which their joint effects have been long debated. Built upon the motivational congruence effect, a recent empirical study affirms that a salient reward in (...) a condition characterized by lack of choice and a non-salient reward in a condition characterized by provision of choice both increase overall motivation and performance. In this study, we examine the effect of reward salience and choice on overall motivation and performance in a controlling context, an effect which has not been studied before. A 2 (choice: present, absent) × 3 (reward: salient, non-salient, none) factorial design was conducted to examine research hypotheses. The results show that under controlling conditions, salient reward improves overall motivation and performance compared to non-salient and no-reward conditions. (shrink)

The experience of a reward appears to enhance memory for recent prior events, adaptively making that information more available to guide future decision-making. Here, we tested whether reward enhances memory for associative item-location information and also whether the effect of reward spreads to other categorically-related but unrewarded items. Participants earned either points (Experiment 1) or money (Experiment 2) through a time-estimation reward task, during which stimuli-location pairings around a 2D-ring were shown followed by either high-value or low-value rewards. All (...) stimuli were then tested for location memory or recognition (yes/no), immediately and after a 24-hour delay. Across both experiments (combined analysis), there was a robust improvement in location memory following high-value rewards, even though evidence supporting this effect was reliable in Experiment 2 but not in Experiment 1. The memory-enhancing effect of reward was observed on both the immediate and delayed location-memory tests. Reward-enhanced memory for both directly rewarded stimuli and categorically related stimuli that were not directly rewarded. No reliable effect of reward value on yes/no recognition-memory performance was observed in either experiment. We hypothesise that reward enhances the consolidation of recent experience and conceptually related memories to make these more available for future decisions. (shrink)

Rewards are reinforcement mechanisms that organizations use to shape desirable employee behaviors. However, rewards may also have unintended consequences, such as building expectations for receiving extra benefits and weakening employee barriers to unethical acts. This article investigates the dark side of the reward–behavior association, and exploring what is referred to as the reward–theft parity effect (RTPE). The authors hypothesize that receiving rewards induces a corresponding type of theft. In Study 1, survey results (n = 634) show initial (...) support for the RTPE between rewards and same‐category theft, and that perceptions of procedural justice attenuate this association. Study 2 involved a 2 (reward) × 2 (theft opportunity) between‐subjects experimental design (n = 300) and provides further support for the RTPE by revealing greater theft under a reward–theft parity condition than under a reward–theft disparity condition. Study 3 replicates these findings in a field experiment (n = 285) and finds that the RTPE is exacerbated when the reward–behavior task is perceived as unjust. Collectively, these findings provide new theoretical insights while also giving managers a more complete understanding of the dark side of behavioral reinforcement mechanisms. (shrink)

Humans expect resources to be distributed fairly. They also show biases to construe all acts as intentional. This study investigates whether every unequal distribution is initially assumed to be intentional unfairness. Study 1 presents a control group of adults with a movie showing one individual accidentally receiving less reward than expected for a task. The experimental group was shown the same scenario, except that the individual was now in the presence of an additional person who received the full reward. Despite (...) the similarity of the scenarios, as predicted, participants in the control condition responded as if the disappointing reward was accidental, while those in the experimental condition responded as if the act was intentional: Their tendency to avoid the “perpetrator” did not differ from that of participants in another control condition who saw an intentionally unfair reward distribution. In Study 2, 7- and 8-year-old children’s results replicated those of adults. Implications for social and moral cognition are discussed. (shrink)

Managers and others have long debated the merits of different reward systems, such as piecework, hourly rates, bonuses, stock options, and the like. They have usually focused on the efficiency of these systems, but they have also had to consider their side effects on relationships, trust, and calls for fair treatment. Such debates local to every organisation play out the issues of rewards and equity in market-based societies as a whole. This paper examines the concept of equity in the (...) distribution of resources in a society of capitalists. It begins with a discussion of the nature of individual capitalism in modern societies. Then, using production and consumption as the two basic functions of a society, it presents a schema for analysing equity. It concludes with a suggestion for overcoming income inequity. (shrink)

Miller and Rodgers (2001) proposed a central nervous system based Ontogenetic Bonding System that operates across the life course to promote succorant, 1 affiliative, sexual, and nurturant bonds. I discuss features of this theoretical framework that can inform Depue & Morrone-Strupinsky's (D&M-S's) model. Most important, I suggest that the affiliative reward processes D&M-S describe are better conceptualized as subserving the affect/motivation of affection. Footnotes1 “Succorance” is a term coined by Murray (1938) to describe a general tendency to seek the help (...) and protection of others. (shrink)

The Nuffield Council on Bioethics’ 2011 report, Human Bodies: Donation for Medicine and Research, proposes a system for examining the ethical implications of different types of incentives for the provision of human tissue for use in medicine and research. The cornerstone of this system is the principle of altruism which, the Council recommends, should, where possible, remain the starting point for any such tissue provision. Using the Council's example of ova provision for research as an area in which altruism-based (...) class='Hi'>rewards might be departed from, this article argues that such a system has the potential to become inconsistent and unnecessarily complex. It suggests that the outcomes-focussed and motivations-focussed justifications the Council provides do not sit easily within the fast-paced and unpredictable area of biotechnology research. Further, it may undermine the focus on autonomy that is enshrined in the relevant legislation. This article suggests that a fair system for incentivising and rewarding the provision of human tissue in research should be developed, which focuses on elements of this role that are common to all tissue providers. (shrink)

The reward infrastructure in science centres on publication, in which journal editors play a key role. Reward distribution hinges on value assessments performed by editors, who draw from plural value systems to judge manuscripts. This conceptual paper examines the numerous biases and other factors that affect editorial decisions. Hybrid and often conflicting value systems contribute to an infrastructure in which editors manage reward through editorial review, commissioned commentaries and reviews and weighing of peer review judgments. Taken together, these systems and (...) processes push the editor into a role resembling censorship. Editors and authors both experience this phenomenon as an unintended side-effect of the reward infrastructure in science. To work towards a more constructive editor-author relationship, we propose a conversation, an exchange between editor and author in which value is collectively assessed as obligatory passage points in the publishing process are traversed. This paper contributes to the discourse on editorial practices by problematising editorial paradigms in a new way and suggesting solutions to entrenched problems. (shrink)

Rolls's book depends significantly on a definition relating emotion to reward and learning. This definition confuses two separable concepts, and may result in the exclusion of notions of emotion and motivation from lower animals that may possess limited learning capacities. A more useful definition might revolve around the notion that emotions are states that function to optimize the performance of behavior.

Internal rewards are the psychological benefits one receives by performing certain other-regarding actions. Internal rewards include such benefits as the avoidance of guilt, the avoidance of painful memories, and the attainment of warm, fuzzy feelings. Despite the limitations of social psychology, Sober and Wilson believe that evolutionary theory can show that it is more likely for benevolent other-regarding motivational mechanisms to have evolved, thereby supporting the altruist’s claim. Here, I will argue for two related theses. First, if internal (...) reward explanations pose a problem for social psychology, then they also pose a problem for evolutionary theory. Second, there is no need to think that internal reward explanations pose a problem for altruists because these explanations either do not inform us about what our ultimate motives really are or they unreasonably define out of existence the possibility of altruism. (shrink)

People differ in the extent to which they discount the values of future rewards. Behavioural economists measure these differences in terms of functions that describe rates of reduced valuation in the future – temporal discounting – as these vary with time. They measure differences in preference for risk – differing rates of probability discounting – in terms of similar functions that describe reduced valuation of rewards as the probability of their delivery falls. So-called ‘impulsive’ people, including people disposed (...) to addiction, tend to choose rewards in patterns that are described by unusually steeply sloped discount functions. The empirical literature is ambivalent as to whether this applies to pathological gamblers (PGs) who manifest most other behavioural patterns associated with addiction, with different, equally careful studies suggesting opposite conclusions (Petry & Casarella 1999; Holt, Green & Myerson 2003). This puzzle may arise from the fact that most previous experiments on discounting behaviour were not designed so as to allow effects of high temporal discounting to be distinguished from effects of low probability discounting. (Addiction has sometimes been associated with the latter because it seems to involve, at least in the starting addict, under appreciation of risk.) Our research project investigates both forms of discounting behaviour and their relationship to severity of risk of PG in a community sample of South African gamblers. (shrink)

Can an agent's intelligence level be negative? We extend the Legg-Hutter agent-environment framework to include punishments and argue for an affirmative answer to that question. We show that if the background encodings and Universal Turing Machine (UTM) admit certain Kolmogorov complexity symmetries, then the resulting Legg-Hutter intelligence measure is symmetric about the origin. In particular, this implies reward-ignoring agents have Legg-Hutter intelligence 0 according to such UTMs.

Unlike other creatures, humans developed the ability to cooperate with genetically unrelated strangers and a tendency to comply with social norms. However, humans deviate from social norms in various situations. This study used the modified ultimatum game to explore why humans deviate from social norms and how their prosocial behavior can be promoted. In Study 1, participants were asked to imagine working with an anonymous counterpart to complete a task and obtain a certain amount of money (e.g., ¥10). The computer (...) divided the money randomly in favor of the participant (e.g., 9:1 or 8:2). Participants should decide whether to accept or reject such a self-benefiting division. In the non-risk condition, an absolutely fair redivision of money would take place if participants reject self-benefiting division (e.g. 5:5 or 6:4). By contrast, in the risk condition, other-benefiting redivision of money (e.g., 1:9 or 2:8) would take place if participants rejected the self-benefiting division. Results involving 40 college students showed the main effect of condition. The frequency of accepting self-benefiting division in the risk condition was higher than that in the non-risk condition. As such, compliance with social norms is based on the preservation of material resources. In Study 2, we used economic or moral rewards to compensate for economic loss following compliance with the norm. Results involving 28 college students revealed a significant effect of compensation. The rewards, including moral praise, effectively decreased selfish choices. These findings extend previous studies on social norm compliance by emphasizing the importance of internal, fairness-based balance between material and moral needs, as well as the role of moral praise in promoting prosocial behavior. (shrink)

Can humans get arbitrarily capable reinforcement learning agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal? This question impacts how far RL can be scaled, and whether alternative paradigms must be developed in order to build safe artificial general intelligence. In this paper, we study when an RL agent has an instrumental goal to tamper with its reward process, and describe design principles that prevent instrumental (...) goals for two different types of reward tampering. Combined, the design principles can prevent reward tampering from being an instrumental goal. The analysis benefits from causal influence diagrams to provide intuitive yet precise formalizations. (shrink)

Past studies have shown that the perceived time of actions is retrospectively influenced by post-action events. The current study examined whether rewarding performance feedback altered the reported time of action. In Experiment 1, participants performed a speeded button press task and received monetary reward for a presumed “fast,” or a monetary punishment for a presumed “slow” response. Rewarded trials resulted in the false perception that the response action occurred earlier than punished trials. In Experiments 2 and 3, the need for (...) a speeded response and reward were independently manipulated in order to decouple the cognitive and reward components in the feedback signal. When tested independently, neither variable affected the judged time of action. We conclude that meaningful feedback is only used when made salient by reward, to modulate the judged time of an action. (shrink)

Acts of helping others are often based on mixed motivations. Based on this claim, it has been argued that the use of a financial reward to incentivize organ donation is compatible with promoting altruism in organ donation. In its report Human Bodies: Donation for Medicine and Research, the Nuffield Council on Bioethics uses this argument to justify its suggestion to pilot a funeral payment scheme to incentivize people to register for deceased organ donation in the UK. In this article, I (...) cast a sceptical eye on the above Nuffield report's argument that its proposed funeral payment scheme would prompt deceased organ donations that remain altruistic . Specifically, I illustrate how this scheme may prompt various forms of mixed motivations which would not satisfy the report's definition of altruism. Insofar as the scheme produces an expectation of the reward, it stands diametrical to promoting an ‘altruistic perspective’. My minimal goal in this article is to argue that altruism is not motivationally compatible with reward as an incentive for donation. My broader goal is to argue that if a financial reward is used to incentivize organ donation, then we should recognize that the donation system is no longer aiming to promote altruism. Rewarded donation would not be altruistic but it may be ethical given a persistent organ shortage situation. (shrink)

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously (...) held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications. (shrink)