BACKGROUND: The nucleus accumbens (NAc) mediates reward learning and motivation. Despite an abundance of neuropeptides, peptidergic neurotransmission from the NAc has not been integrated into current models of reward learning. The existence of a sparse population of neurons containing corticotropin releasing factor (CRF) has been previously documented. Here we provide a comprehensive analysis of their identity and functional role in shaping reward learning.
METHODS: To do this, we took a multidisciplinary approach that included florescent in situ hybridization (Nmice ≥ 3), tract tracing (Nmice = 5), ex vivo electrophysiology (Ncells ≥ 30), in vivo calcium imaging with fiber photometry (Nmice ≥ 4) and use of viral strategies in transgenic lines to selectively delete CRF peptide from NAc neurons (Nmice ≥ 4). Behaviors used were instrumental learning, sucrose preference and spontaneous exploration in an open field.
RESULTS: Here we show that the vast majority of NAc CRF-containing (NAcCRF) neurons are spiny projection neurons (SPNs) comprised of dopamine D1-, D2- or D1/D2-containing SPNs that primarily project and connect to the ventral pallidum and to a lesser extent the ventral midbrain. As a population, they display mature and immature SPN firing properties. We demonstrate that NAcCRF neurons track reward outcomes during operant reward learning and that CRF release from these neurons acts to constrain initial acquisition of action-outcome learning, and at the same time facilitates flexibility in the face of changing contingencies.
CONCLUSIONS: We conclude that CRF release from this sparse population of SPNs is critical for reward learning under normal conditions.

UNASSIGNED: Current frameworks propose that delusions result from aberrant belief updating due to altered prediction error (PE) signaling and misestimation of environmental volatility. We aimed to investigate whether behavioral and neural signatures of belief updating are specifically related to the presence of delusions or generally associated with manifest schizophrenia.
UNASSIGNED: Our cross-sectional design includes human participants (n[female/male] = 66[25/41]), stratified into four groups: healthy participants with minimal (n = 22) or strong delusional-like ideation (n = 18), and participants with diagnosed schizophrenia with minimal (n = 13) or strong delusions (n = 13), resulting in a 2 × 2 design, which allows to test for the effects of delusion and diagnosis. Participants performed a reversal learning task with stable and volatile task contingencies during fMRI scanning. We formalized learning with a hierarchical Gaussian filter model and conducted model-based fMRI analysis regarding beliefs of outcome uncertainty and volatility, precision-weighted PEs of the outcome- and the volatility-belief.
UNASSIGNED: Patients with schizophrenia as compared to healthy controls showed lower accuracy and heightened choice switching, while delusional ideation did not affect these measures. Participants with delusions showed increased precision-weighted PE-related neural activation in fronto-striatal regions. People with diagnosed schizophrenia overestimated environmental volatility and showed an attenuated neural representation of volatility in the anterior insula, medial frontal and angular gyrus.
UNASSIGNED: Delusional beliefs are associated with altered striatal PE-signals. Juxtaposing, the potentially unsettling belief that the environment is constantly changing and weaker neural encoding of this subjective volatility seems to be associated with manifest schizophrenia, but not with the presence of delusional ideation.

Adverse childhood experiences (ACEs) are a major risk factor for the development of multiple psychopathological conditions, but the mechanisms underlying this link are poorly understood. Associative learning encompasses key mechanisms through which individuals learn to link important environmental inputs to emotional and behavioral responses. ACEs may impact the normative maturation of associative learning processes, resulting in their enduring maladaptive expression manifesting in psychopathology. In this review, we lay out a systematic and methodological overview and integration of the available evidence of the proposed association between ACEs and threat and reward learning processes. We summarize results from a systematic literature search (following PRISMA guidelines) which yielded a total of 81 articles (threat: n=38, reward: n=43). Across the threat and reward learning fields, behaviorally, we observed a converging pattern of aberrant learning in individuals with a history of ACEs, independent of other sample characteristics, specific ACE types, and outcome measures. Specifically, blunted threat learning was reflected in reduced discrimination between threat and safety cues, primarily driven by diminished responding to conditioned threat cues. Furthermore, attenuated reward learning manifested in reduced accuracy and learning rate in tasks involving acquisition of reward contingencies. Importantly, this pattern emerged despite substantial heterogeneity in ACE assessment and operationalization across both fields. We conclude that blunted threat and reward learning may represent a mechanistic route by which ACEs may become physiologically and neurobiologically embedded and ultimately confer greater risk for psychopathology. In closing, we discuss potentially fruitful future directions for the research field, including methodological and ACE assessment considerations.

Stressors can initiate a cascade of central and peripheral changes that modulate mesocorticolimbic dopaminergic circuits and, ultimately, behavioral response to rewards. Driven by the absence of conclusive evidence on this topic and the Research Domain Criteria framework, random-effects meta-analyses were adopted to quantify the effects of acute stressors on reward responsiveness, valuation, and learning in rodent and human subjects. In rodents, acute stress reduced reward responsiveness (g = -1.43) and valuation (g = -0.32), while amplifying reward learning (g = 1.17). In humans, acute stress had marginal effects on valuation (g = 0.25), without affecting responsiveness and learning. Moderation analyses suggest that acute stress neither has unitary effects on reward processing in rodents nor in humans and that the duration of the stressor and specificity of reward experience (i.e., food vs drugs) may produce qualitatively and quantitatively different behavioral endpoints. Subgroup analyses failed to reduce heterogeneity, which, together with the presence of publication bias, pose caution on the conclusions that can be drawn and point to the need of guidelines for the conduction of future studies in the field.

BACKGROUND: Decades of research have firmly established that cognitive health and cognitive treatment services are a key need for people living with psychosis. However, many current clinical programs do not address this need, despite the essential role that an individual\'s cognitive and social cognitive capacities play in determining their real-world functioning. Preliminary practice-based research in the Early Psychosis Intervention Network early psychosis intervention network shows that it is possible to develop and implement tools that delineate an individuals\' cognitive health profile and that help engage the client and the clinician in shared decision-making and treatment planning that includes cognitive treatments. These findings signify a promising shift toward personalized cognitive health.
METHODS: Extending upon this early progress, we review the concept of interindividual variability in cognitive domains/processes in psychosis as the basis for offering personalized treatment plans. We present evidence from studies that have used traditional neuropsychological measures as well as findings from emerging computational studies that leverage trial-by-trial behavior data to illuminate the different latent strategies that individuals employ.
UNASSIGNED: We posit that these computational techniques, when combined with traditional cognitive assessments, can enrich our understanding of individual differences in treatment needs, which in turn can guide evermore personalized interventions.
CONCLUSIONS: As we find clinically relevant ways to decompose maladaptive behaviors into separate latent cognitive elements captured by model parameters, the ultimate goal is to develop and implement approaches that empower clients and their clinical providers to leverage individual\'s existing learning capacities to improve their cognitive health and well-being.

In classical cerebellar learning, Purkinje cells (PkCs) associate climbing fiber (CF) error signals with predictive granule cells (GrCs) that were active just prior (∼150 ms). The cerebellum also contributes to behaviors characterized by longer timescales. To investigate how GrC-CF-PkC circuits might learn seconds-long predictions, we imaged simultaneous GrC-CF activity over days of forelimb operant conditioning for delayed water reward. As mice learned reward timing, numerous GrCs developed anticipatory activity ramping at different rates until reward delivery, followed by widespread time-locked CF spiking. Relearning longer delays further lengthened GrC activations. We computed CF-dependent GrC→PkC plasticity rules, demonstrating that reward-evoked CF spikes sufficed to grade many GrC synapses by anticipatory timing. We predicted and confirmed that PkCs could thereby continuously ramp across seconds-long intervals from movement to reward. Learning thus leads to new GrC temporal bases linking predictors to remote CF reward signals-a strategy well suited for learning to track the long intervals common in cognitive domains.

UNASSIGNED: Behavioral activation is an evidence-based treatment for depression. Theoretical considerations suggest that treatment response depends on reinforcement learning mechanisms. However, which reinforcement learning mechanisms are engaged by and mediate the therapeutic effect of behavioral activation remains only partially understood, and there are no procedures to measure such mechanisms.
UNASSIGNED: To perform a pilot study to examine whether reinforcement learning processes measured through tasks or self-report are related to treatment response to behavioral activation.
UNASSIGNED: The pilot study enrolled 13 outpatients (12 completers) with major depressive disorder, from July of 2018 through February of 2019, into a nine-week trial with BA. Psychiatric evaluations, decision-making tests and self-reported reward experience and anticipations were acquired before, during and after the treatment. Task and self-report data were analysed by using reinforcement-learning models. Inferred parameters were related to measures of depression severity through linear mixed effects models.
UNASSIGNED: Treatment effects during different phases of the therapy were captured by specific decision-making processes in the task. During the weeks focusing on the active pursuit of reward, treatment effects were more pronounced amongst those individuals who showed an increase in Pavlovian appetitive influence. During the weeks focusing on the avoidance of punishments, treatment responses were more pronounced in those individuals who showed an increase in Pavlovian avoidance. Self-reported anticipation of reinforcement changed according to formal RL rules. Individual differences in the extent to which learning followed RL rules related to changes in anhedonia.
UNASSIGNED: In this pilot study both task- and self-report-derived measures of reinforcement learning captured individual differences in treatment response to behavioral activation. Appetitive and aversive Pavlovian reflexive processes appeared to be modulated by separate psychotherapeutic interventions, and the modulation strength covaried with response to specific interventions. Self-reported changes in reinforcement expectations are also related to treatment response.
UNASSIGNED: Set Your Goal: Engaging in GO/No-Go Active Learning, #NCT03538535, http://www.clinicaltrials.gov.

Recent experiments and theories of human decision-making suggest positive and negative errors are processed and encoded differently by serotonin and dopamine, with serotonin possibly serving to oppose dopamine and protect against risky decisions. We introduce a temporal difference (TD) model of human decision-making to account for these features. Our model involves two critics, an optimistic learning system and a pessimistic learning system, whose predictions are integrated in time to control how potential decisions compete to be selected. Our model predicts that human decision-making can be decomposed along two dimensions: the degree to which the individual is sensitive to (1) risk and (2) uncertainty. In addition, we demonstrate that the model can learn about the mean and standard deviation of rewards, and provide information about reaction time despite not modeling these variables directly. Lastly, we simulate a recent experiment to show how updates of the two learning systems could relate to dopamine and serotonin transients, thereby providing a mathematical formalism to serotonin\'s hypothesized role as an opponent to dopamine. This new model should be useful for future experiments on human decision-making.

Learning, an important activity for both human and animals, has long been a focal point of research. During the learning process, subjects assimilate not only their own information but also information from others, a phenomenon known as social learning. While numerous studies have explored the impact of social feedback as a reward/punishment during learning, few studies have investigated whether social feedback facilitates or inhibits the learning of environmental rewards/punishments. This study aims to test the effects of social feedback on economic feedback and its cognitive processes by using the Iowa Gambling Task (IGT). One hundred ninety-two participants were recruited and categorized into one non-social feedback group and four social feedback groups. Participants in the social feedback groups were informed that after the outcome of each choice, they would also receive feedback from an online peer. This peer was a fictitious entity, with variations in identity (novice or expert) and feedback type (random or effective). The Outcome-Representation Learning model (ORL model) was used to quantify the cognitive components of learning. Behavioral results showed that both the identity of the peer and the type of feedback provided significantly influenced the deck selection, with effective social feedback increasing the ratio of chosen good decks. Results in the ORL model showed that the four social feedback groups exhibited lower learning rates for gain and loss compared to the nonsocial feedback group, which suggested, in the social feedback groups, the impact of the recent outcome on the update of value decreased. Parameters such as forgetfulness, win frequency, and deck perseverance in the expert-effective feedback group were significantly higher than those in the non-social feedback and expert-random feedback groups. These findings suggest that individuals proactively evaluate feedback providers and selectively adopt effective feedback to enhance learning.

Major depressive disorder (MDD) is one of the leading causes of disability-adjusted life years. Emerging evidence indicates the presence of reward processing abnormalities in MDD. An important scientific question is whether the abnormalities are due to reduced sensitivity to received rewards or reduced learning ability. Motivated by the probabilistic reward task (PRT) experiment in the EMBARC study, we propose a semiparametric inverse reinforcement learning (RL) approach to characterize the reward-based decision-making of MDD patients. The model assumes that a subject\'s decision-making process is updated based on a reward prediction error weighted by the subject-specific learning rate. To account for the fact that one favors a decision leading to a potentially high reward, but this decision process is not necessarily linear, we model reward sensitivity with a non-decreasing and nonlinear function. For inference, we estimate the latter via approximation by I-splines and then maximize the joint conditional log-likelihood. We show that the resulting estimators are consistent and asymptotically normal. Through extensive simulation studies, we demonstrate that under different reward-generating distributions, the semiparametric inverse RL outperforms the parametric inverse RL. We apply the proposed method to EMBARC and find that MDD and control groups have similar learning rates but different reward sensitivity functions. There is strong statistical evidence that reward sensitivity functions have nonlinear forms. Using additional brain imaging data in the same study, we find that both reward sensitivity and learning rate are associated with brain activities in the negative affect circuitry under an emotional conflict task.