This paper presents an interdisciplinary framework, Machine Psychology, which integrates principles from operant learning psychology with a particular Artificial Intelligence model, the Non-Axiomatic Reasoning System (NARS), to advance Artificial General Intelligence (AGI) research. Central to this framework is the assumption that adaptation is fundamental to both biological and artificial intelligence, and can be understood using operant conditioning principles. The study evaluates this approach through three operant learning tasks using OpenNARS for Applications (ONA): simple discrimination, changing contingencies, and conditional discrimination tasks. In the simple discrimination task, NARS demonstrated rapid learning, achieving 100% correct responses during training and testing phases. The changing contingencies task illustrated NARS\'s adaptability, as it successfully adjusted its behavior when task conditions were reversed. In the conditional discrimination task, NARS managed complex learning scenarios, achieving high accuracy by forming and utilizing complex hypotheses based on conditional cues. These results validate the use of operant conditioning as a framework for developing adaptive AGI systems. NARS\'s ability to function under conditions of insufficient knowledge and resources, combined with its sensorimotor reasoning capabilities, positions it as a robust model for AGI. The Machine Psychology framework, by implementing aspects of natural intelligence such as continuous learning and goal-driven behavior, provides a scalable and flexible approach for real-world applications. Future research should explore using enhanced NARS systems, more advanced tasks and applying this framework to diverse, complex tasks to further advance the development of human-level AI.

Anhedonia and avolition are emotions frequently endorsed by individuals with stress related disorders. Kappa opioid receptor (KOR) activation can induce negative emotions and recent clinical evidence suggests that KOR antagonism can alleviate anhedonia in a transdiagnostic cohort of patients. However, the behavioral consequences of KOR activation and antagonism in modulating motivation, as assessed by schedule-controlled behavioral performance without preexisting conditions (stress or substance use), have not been formally assessed. To address this gap in the literature, this report utilized male and female Sprague Dawley rats to (1) evaluate the impact of the selective KOR agonist U50,488, on the performance of animals responding for sucrose pellets under a progressive ratio (PR) schedule and (2) determine the effects of the short-acting KOR antagonist LY2444296 alone and on U50,488 mediated reductions in PR performance. Overall, U50,488 5 mg/kg significantly reduced the breakpoint and number of rewards obtained by animals. This occurred in the absence of motor impairment and independent of evidence for satiation. LY2444296 did not alter PR performance when administered alone but effectively blocked the deficits induced by U50,488. To further delineate the behavioral alterations that underlie these reductions in responding, a more detailed analysis was conducted on PR performance in the first 15 min of the session, the period of time when animals obtained the most reinforcers. During this period, U50,488 increased the length of the post-reinforcement pause and reduced the running rate on PR schedules. These changes in behavior produced by acute activation of KORs are consistent with a reduction of effort-related motivation in rodents. These data contribute to the understanding of how KORs modulate motivation, which is critical to future efforts to evaluate performance in the context of stress and assess how KOR antagonists alleviate anhedonic behaviors associated with stress.

Aspirin (Acetylsalicylic acid, ASA), one of the widely used non-steroid anti-inflammatory drugs can easily end up in sewage effluents and thus it becomes necessary to investigate the effects of aspirin on behaviour of aquatic organisms. Previous studies in mammals have shown ASA to alter fear and anxiety-like behaviours. In the great pond snail Lymnaea stagnalis, ASA has been shown to block a \'sickness state\' induced by lipopolysaccharide injection which upregulates immune and stress-related genes thus altering behavioural responses. In Lymnaea, eliciting physiological stress may enhance memory formation or block its retrieval depending on the stimulus type and intensity. Here we examine whether ASA will alter two forms of associative-learning memory in crayfish predator-experienced Lymnaea when ASA exposure accompanies predator-cue-induced stress during the learning procedure. The two trainings procedures are: 1) operant conditioning of aerial respiration; and 2) a higher form of learning, called configural learning, which here is dependent on evoking a fear response. We show here that ASA alone does not alter homeostatic aerial respiration, feeding behaviour or long-term memory (LTM) formation of operantly conditioned aerial respiration. However, ASA blocked the enhancement of LTM formation normally elicited by training snails in predator cue. ASA also blocked configural learning, which makes use of the fear response elicited by the predator cue. Thus, ASA alters how Lymnaea responds cognitively to predator detection.

Reinforcement learning inspires much theorizing in neuroscience, cognitive science, machine learning, and AI. A central question concerns the conditions that produce the perception of a contingency between an action and reinforcement-the assignment-of-credit problem. Contemporary models of associative and reinforcement learning do not leverage the temporal metrics (measured intervals). Our information-theoretic approach formalizes contingency by time-scale invariant temporal mutual information. It predicts that learning may proceed rapidly even with extremely long action-reinforcer delays. We show that rats can learn an action after a single reinforcement, even with a 16-min delay between the action and reinforcement (15-fold longer than any delay previously shown to support such learning). By leveraging metric temporal information, our solution obviates the need for windows of associability, exponentially decaying eligibility traces, microstimuli, or distributions over Bayesian belief states. Its three equations have no free parameters; they predict one-shot learning without iterative simulation.

Synaptic plasticity in the mesolimbic dopamine (DA) system contributes to the neural adaptations underlying addictive behaviors and relapse. However, the specific behavioral relevance of glutamatergic excitatory drive onto dopamine D1 receptor (D1R)-expressing neurons in mediating the reinforcing effect of cocaine remains unclear. Here, we investigated how midbrain AMPAR and NMDAR function modulate cocaine reward-related behavior using mutant mouse lines lacking the glutamate receptor genes Gria1 or Grin1 in D1R-expressing neurons (GluA1D1CreERT2 or GluN1D1CreERT2, respectively). We found that conditional genetic deletion of either GluA1 or GluN1 within this neuronal sub-population did not impact the ability of acute cocaine injection to increase intracranial self-stimulation (ICSS) ratio or reduced brain reward threshold compared to littermate controls. Additionally, our data demonstrate that deletion of GluA1 and GluN1 receptor subunits within D1R-expressing neurons did not affect cocaine reinforcement in an operant self-administration paradigm, as mutant mice showed comparable cocaine responses and intake to controls. Given the pivotal role of glutamate receptors in mediating relapse behavior, we further explored the impact of genetic deletion of AMPAR and NMDAR onto D1R-expressing neurons on cue-induced reinstatement following extinction. Surprisingly, deletion of AMPAR and NMDAR onto these neurons did not impair cue-induced reinstatement of cocaine-seeking behavior. These findings suggest that glutamatergic activity via NMDAR and AMPAR in D1R-expressing neurons may not exclusively mediate the reinforcing effects of cocaine and cue-induced reinstatement.

BACKGROUND: Substance use disorder is characterized by long-lasting changes in reward-related brain regions, such as the nucleus accumbens. Previous work has shown that cocaine exposure induces plasticity in broad, genetically defined cell types in the nucleus accumbens; however, in response to a stimulus, only a small percentage of neurons are transcriptionally active-termed an ensemble. Here, we identify an Arc-expressing neuronal ensemble that has a unique trajectory of recruitment and causally controls drug self-administration after repeated, but not acute, cocaine exposure.
METHODS: Using Arc-CreERT2 transgenic mice, we expressed transgenes in Arc+ ensembles activated by cocaine exposure (either acute [1 × 10 mg/kg intraperitoneally] or repeated [10 × 10 mg/kg intraperitoneally]). Using genetic, optical, and physiological recording and manipulation strategies, we assessed the contribution of these ensembles to behaviors associated with substance use disorder.
RESULTS: Repeated cocaine exposure reduced the size of the ensemble while simultaneously increasing its control over behavior. Neurons within the repeated cocaine ensemble were hyperexcitable, and their optogenetic excitation was sufficient for reinforcement. Finally, lesioning the repeated cocaine, but not the acute cocaine, ensemble blunted cocaine self-administration. Thus, repeated cocaine exposure reduced the size of the ensemble while simultaneously increasing its contributions to drug reinforcement.
CONCLUSIONS: We showed that repeated, but not acute, cocaine exposure induced a physiologically distinct ensemble characterized by the expression of the immediate early gene Arc, which was uniquely capable of modulating reinforcement behavior.

In classical cerebellar learning, Purkinje cells (PkCs) associate climbing fiber (CF) error signals with predictive granule cells (GrCs) that were active just prior (∼150 ms). The cerebellum also contributes to behaviors characterized by longer timescales. To investigate how GrC-CF-PkC circuits might learn seconds-long predictions, we imaged simultaneous GrC-CF activity over days of forelimb operant conditioning for delayed water reward. As mice learned reward timing, numerous GrCs developed anticipatory activity ramping at different rates until reward delivery, followed by widespread time-locked CF spiking. Relearning longer delays further lengthened GrC activations. We computed CF-dependent GrC→PkC plasticity rules, demonstrating that reward-evoked CF spikes sufficed to grade many GrC synapses by anticipatory timing. We predicted and confirmed that PkCs could thereby continuously ramp across seconds-long intervals from movement to reward. Learning thus leads to new GrC temporal bases linking predictors to remote CF reward signals-a strategy well suited for learning to track the long intervals common in cognitive domains.

In honey bees, most studies of circadian rhythms involve a locomotion test performed in a small tube, a tunnel, or at the hive entrance. However, despite feeding playing an important role in honey bee health or fitness, no demonstration of circadian rhythm on feeding has been performed until recently. Here, we present the BeeBox, a new laboratory platform for bees based on the concept of the Skinner box, which dispenses discrete controlled amounts of food (sucrose syrup) following entrance into an artificial flower. We compared caged groups of bees in 12 h-12 h light/dark cycles, constant darkness and constant light and measured average hourly syrup consumption per living bee. Food intake was higher in constant light and lower in constant darkness; mortality increased in constant light. We observed rhythmic consumption with a period longer than 24 h; this is maintained in darkness without environmental cues, but is damped in the constant light condition. The BeeBox offers many new research perspectives and numerous potential applications in the study of nectar foraging animals.

We are studying the mechanisms of H-reflex operant conditioning, a simple form of learning. Modelling studies in the literature and our previous data suggested that changes in the axon initial segment (AIS) might contribute. To explore this, we used blinded quantitative histological and immunohistochemical methods to study in adult rats the impact of H-reflex conditioning on the AIS of the spinal motoneuron that produces the reflex. Successful, but not unsuccessful, H-reflex up-conditioning was associated with greater AIS length and distance from soma; greater length correlated with greater H-reflex increase. Modelling studies in the literature suggest that these increases may increase motoneuron excitability, supporting the hypothesis that they may contribute to H-reflex increase. Up-conditioning did not affect AIS ankyrin G (AnkG) immunoreactivity (IR), p-p38 protein kinase IR, or GABAergic terminals. Successful, but not unsuccessful, H-reflex down-conditioning was associated with more GABAergic terminals on the AIS, weaker AnkG-IR, and stronger p-p38-IR. More GABAergic terminals and weaker AnkG-IR correlated with greater H-reflex decrease. These changes might potentially contribute to the positive shift in motoneuron firing threshold underlying H-reflex decrease; they are consistent with modelling suggesting that sodium channel change may be responsible. H-reflex down-conditioning did not affect AIS dimensions. This evidence that AIS plasticity is associated with and might contribute to H-reflex conditioning adds to evidence that motor learning involves both spinal and brain plasticity, and both neuronal and synaptic plasticity. AIS properties of spinal motoneurons are likely to reflect the combined influence of all the motor skills that share these motoneurons. KEY POINTS: Neuronal action potentials normally begin in the axon initial segment (AIS). AIS plasticity affects neuronal excitability in development and disease. Whether it does so in learning is unknown. Operant conditioning of a spinal reflex, a simple learning model, changes the rat spinal motoneuron AIS. Successful, but not unsuccessful, H-reflex up-conditioning is associated with greater AIS length and distance from soma. Successful, but not unsuccessful, down-conditioning is associated with more AIS GABAergic terminals, less ankyrin G, and more p-p38 protein kinase. The associations between AIS plasticity and successful H-reflex conditioning are consistent with those between AIS plasticity and functional changes in development and disease, and with those predicted by modelling studies in the literature. Motor learning changes neurons and synapses in spinal cord and brain. Because spinal motoneurons are the final common pathway for behaviour, their AIS properties probably reflect the combined impact of all the behaviours that use these motoneurons.

Evaluative Conditioning (EC) refers to changes in our liking or disliking of a stimulus due to its pairing with other positive or negative stimuli. In addition to stimulus-based mechanisms, recent research has shown that action-based mechanisms can also lead to EC effects. Research, based on action control theories, has shown that pairing a positive or negative action with a neutral stimulus results in EC effects (Stimulus-Response binding). Similarly, research studies using Operant Conditioning (OC) approaches have also observed EC effects. The aim of the present study is to directly compare EC effects elicited by two different response-based approaches - S-R bindings and OC. To this end, participants were randomly assigned to an S-R binding procedure and an OC procedure. EC effects were measured in conditions and compared. Implications for EC theory are discussed.