Authors: Antoinette Poulton, Robert Hester
Categories: Original Manuscript, dopamine, impulsivity, learning, reward, substance use disorders, AcademicSubjects/SCI01880
Source: Social Cognitive and Affective Neuroscience
Doi: 10.1093/scan/nsz077
Substance dependence constitutes a profound societal burden. Although large numbers of individuals use licit or illicit substances, few transition to dependence. The specific factors influencing this transition are not well understood. Substance-dependent individuals tend to be swayed by the immediate rewards of drug taking, but are often insensitive to delayed negative consequences of their behavior. Dependence is consequently associated with impulsivity for reward and atypical learning from feedback. Behavioral impulsivity is indexed using tasks measuring spontaneous decision-making and capacity to control impulses. While evidence indicates drug taking exacerbates behavioral impulsivity for reward, animal and human studies of drug naïve populations demonstrate it might precede any drug-related problems. Research suggests dependent individuals are also more likely to learn from rewarding (relative to punishing) feedback. This may partly explain why substance-dependent individuals fail to modify their behavior in response to negative outcomes. This enhanced learning from reward may constitute a further pre-existing risk factor for substance dependence. Although impulsivity for reward and preferential learning from rewarding feedback are both underpinned by a compromised dopaminergic system, few studies have examined the relationship between these two mechanisms. The interplay of these processes may help enrich understanding of why some individuals transition to substance dependence.
Keywords: substance use disorders, impulsivity, learning, reward, dopamine
Drug misuse is a major health issue with wide-ranging economic and social implications. It is a risk factor for a range of diseases and disorders, including injuries, cancers, HIV infection and mental illness (Rehm et al., 2017; Peacock et al., 2018). Healthy years lost due to alcohol, tobacco and illicit drug use were 85.0, 170.9 and 27.8 million, respectively, in 2015 (Peacock et al., 2018). This represents a substantial economic and social burden. Despite the costs, licit and illicit drugs are widely used. Worldwide, 43% of the population consumes alcohol, 15.2% smoke daily and 5% use illicit drugs (United Nations Office on Drugs and Crime [UNODC], 2017; Peacock et al., 2018; World Health Organisation [WHO], 2018). Relatively, few of these people will, however, become dependent drug users. Globally, 5.2% of individuals are diagnosed with alcohol-use disorders, while 11% of those who use illicit substances are estimated to be dependent (UNODC, 2017; WHO, 2018). Nevertheless, if the considerable burden of drug misuse is to be alleviated, unraveling the specific constellation of factors influencing the transition from drug use to substance use disorder is imperative.
Substance dependence refers to the continued and maladaptive use of drugs of abuse, medications or toxins, despite impairments at clinically significant levels (American Psychiatric Association [APA], 2013). Substance-dependent individuals tend to consume these substances in larger amounts than anticipated, and often continue using them despite being aware of the associated negative consequences (APA, 2013). Factors related to impulsivity, impulse control and learning thus appear to be implicated. Specifically, dependent individuals are particularly impulsive for the short-term rewards of drug taking. Moreover, their persistent maladaptive drug use implies a diminished capacity to control this impulsivity. At the same time, continued drug use despite negative consequences suggests dependent individuals may also be characterized by aberrant learning such that rewarding outcomes influence them more than punishing ones.
This commentary will examine behavioral impulsivity for reward, operationalized as impulsive decision-making and impaired inhibitory control, and behavioral learning from reward, operationalized as preferential learning from rewarding (relative to punishing) outcomes. The evidence that links each of these constructs to substance dependence will be considered. The neurobiological underpinnings of these concepts, including the role of dopamine, will be described in order to demonstrate a potential link between these mechanisms. It is proposed that individual differences in the expression of behavioral impulsivity may be related to individual differences in behavioral learning from rewarding (relative to punishing) feedback. Furthermore, this relationship may constitute a pre-existing vulnerability for substance dependence. While the relationship between impulsivity and substance dependence has been explored previously, we extend upon previous reviews by suggesting the link between behavioral impulsivity for reward and behavioral learning from reward warrants further investigation.
Impulsivity is recognized as a multidimensional concept and is therefore broadly defined. It usually refers to acting hastily or rashly and without forethought or contemplating outcomes (Evenden, 1999). Although beneficial in some professional and social situations, it is the negative aspects of impulsivity, particularly as they relate to personality disorders, attention-deficit hyperactivity disorder (ADHD), gambling, and substance use disorders, which attract clinical intervention (Evenden, 1999; Bickel et al., 2012). In personality psychology, impulsivity is commonly assessed using self-report measures, which provide an indication of an individual’s trait-level impulsivity (Cyders and Coskunpinar, 2011; Gullo and Potenza, 2014). In cognitive psychology, it is assessed at the behavioral level using computer-based tasks that tap various components of the concept (Cyders and Coskunpinar, 2011; Gullo and Potenza, 2014; Weafer et al., 2014). There is often no or only low correlation between self-report and behavioral measures of impulsivity (Reynolds et al., 2006; Cyders and Coskunpinar, 2011; Gullo and Potenza, 2014; MacKillop et al., 2016); as such, some have questioned the validity of viewing impulsivity as a unitary construct (Cyders and Coskunpinar, 2011). It can be argued, however, that while self-reports provide an overview of an individual’s general tendency to be impulsive, behavioral tasks capture in-the-moment fluctuations in components of impulsivity, as well as variations that might arise in response to various stimuli (Cyders and Coskunpinar, 2011; Gullo and Potenza, 2014). Although both self-report and behavioral impulsivity have been implicated in substance use disorder pathology (Verdejo-García et al., 2008; Gullo et al., 2014; Gullo and Potenza, 2014; Weafer et al., 2014), it is fluctuations in aspects of impulsivity and how these might map onto variations in substance intake that often attracts the interest of cognitive psychologists (Jones et al., 2018), particularly as this might provide insight into when and how to intervene.
In this section, we focus on two aspects of impulsivity—impulsivity for reward (or impulsive choice) and impulse control (or impulsive action)—which have been identified as distinct components of impulsive behavior and are core processes linked to drug misuse (Dawe et al., 2004; Reynolds et al., 2006; Verdejo-García et al., 2008; Gullo et al., 2014; Weafer et al., 2014; MacKillop et al., 2016). Specifically, we consider behavioral manifestations of these components of impulsivity; that is, impulsive decision-making and impaired inhibitory control. At the neurobiological level, these tasks are theorized to, respectively, tap the influence of bottom-up midbrain reward processes and the relative efficacy of prefrontal top-down executive control mechanisms (Weafer et al., 2014). According to the competing brain regions’ hypothesis, vulnerability for transition to substance use disorders is theorized to arise from imbalances across these distinct yet interrelated brain networks (Jentsch and Taylor, 1999; Bechara, 2005; Bickel et al., 2007). The impaired response inhibition and salience attribution theory likewise highlights the respective and interactive roles of midbrain and frontal regions in addiction as well as the involvement of mesolimbic and mesocortical dopaminergic circuits (Goldstein and Volkow, 2002; Goldstein and Volkow, 2011; Zilverstand et al., 2018).
Impulsive decision-making is defined as a propensity to favor immediate reward regardless of delayed outcomes (Dom et al., 2007; Fernie et al., 2010; Murphy and Garavan, 2011). It is described as sensitivity to proximate reward, and insensitivity to delayed consequences (De Wit, 2009). In drug taking, this equates to increased sensitivity to the rewarding aspects of drug use, including euphoria and relief from withdrawal, which occur almost immediately, and insensitivity to negative outcomes, such as loss of job, family and friends, which have a gradual onset and occur at some later moment in time (Kirby and Petry, 2004). Behavioral impulsive decision-making is typically assessed using delayed discounting tasks (Field et al., 2007; Bickel et al., 2012). Such tasks examine the point at which individuals choose an immediate reward in preference to waiting for a larger one available after some delay; this point quantifies how sharply future rewards are devalued as a function of the delay (Field et al., 2007; MacKillop et al., 2011). It denotes an individual’s delay discounting rate (DDR), and thereby provides a measure of impulsive decision-making (Kirby et al., 1999; Bickel et al., 2008). Generally, hypothetical monetary rewards are employed in delayed discounting tasks, as experiments comparing how individuals discount hypothetical vs real rewards have found little evidence to suggest DDRs differ significantly across conditions (Bickel et al., 2002; Madden et al., 2003).
Impulsive decision-making can be conceptualized as varying along a spectrum. Individuals with clinically diagnosed dependence problems typically have high DDRs, thereby signifying greater impulsivity for immediate reward (De Wit, 2009; MacKillop et al., 2011). For example, compared to controls, significantly higher DDRs have been identified in alcohol-dependent individuals attempting to abstain (Mitchell et al., 2007); cocaine-dependent individuals, including those attempting abstinence (Heil et al., 2006); heroin users (Kirby et al., 1999); and methamphetamine-dependent participants attempting to abstain (Hoffman et al., 2006). Indeed, a meta-analysis of 64 studies detailed clear links between high DDRs and substance dependence in clinical populations, including those attempting abstinence (MacKillop et al., 2011). At the same time, individuals identified as subclinical drug users generally have lower DDRs than dependent users, but higher rates than controls (MacKillop et al., 2011). Murphy and Garavan (2011) found that individual differences in DDRs differentiated nondependent problem and nonproblem alcohol drinkers, and predicted alcohol consumption, drinking behavior, and alcohol-related issues. Similarly, Field et al. (2007) revealed there was a significant association between weekly alcohol consumption and DDRs among adolescents, such that nondependent heavy drinkers had higher rates than light drinkers. Thus, while research supports the notion that substance-dependent individuals and those attempting abstinence are characterized by high DDRs, there is also evidence that impulsive decision-making tasks can differentiate between dependent users, nondependent problematic users and nondependents. It has, as a consequence, been proposed as a behavioral marker of addiction that warrants further investigation (Bickel et al., 2014).
The impulsive decision-making symptomology of substance-dependent individuals and those attempting abstinence is considered both a cause and consequence of drug use. Individuals who discount future possible rewards in favor of more immediate ones appear susceptible to drug use problems (Monterosso et al., 2001). In a six-year prospective study of 947 adolescents, Audrain-McGovern et al. (2009) discovered high DDRs differentiated smoking vs nonsmoking trajectories. Kollins (2003) found that elevated DDRs were associated with self-reported younger age at first use of marijuana, cigarettes and alcohol, as well as the number of illicit drugs used. Animal studies have demonstrated that high impulsivity in drug-naïve rats, as measured by a delayed reward paradigm, predicts both self-administered drug acquisition and escalation of use (Perry et al., 2005; Dalley et al., 2007; Anker et al., 2009; MacKillop et al., 2011; Jupp and Dalley, 2014). Collectively, these studies suggest impulsive decision-making plays a causal role in the development of substance-misuse behaviors. They thus also highlight the potential predictive value of impulsive decision-making tasks in terms of identifying individuals likely to transition from drug use to substance dependence. At the same time, drug taking appears to exacerbate impulsive decision-making. Although single doses of ethanol, delta^9^-tetrahydrocannabinol, and d-amphetamine did not increase DDRs in healthy nondrug-taking individuals (De Wit and Richards, 2004; Bidwell et al., 2013), rats became more impulsive with repeated self-administration of the stimulant d-amphetamine, regardless of the level of their intake (Gipson and Bardo, 2009). Similarly, self-administered cocaine produced lasting elevated delayed discounting in rats (Mendez et al., 2010). Thus, behavioral evidence suggests that those vulnerable to substance dependence have enhanced impulsivity for reward and that continued drug use further exacerbates this impulsivity, which presumably renders abstinence particularly difficult.
Impaired inhibitory control, or cognitive disinhibition, refers to an inability to successfully inhibit a dominant behavioral or prepotent response (Dom et al., 2007; Field et al., 2007; Murphy and Garavan, 2011). It describes a tendency to act without thinking, and to succumb to, rather than overcome, urges. Typically, stop signal, go/no-go and go/stop tasks are employed in the investigation of behavioral inhibitory control (Goudriaan et al., 2008; Fernie et al., 2010; Bickel et al., 2012). These tasks require participants to respond rapidly to specific frequently appearing stimuli, but to inhibit responses to others that are presented less often (Murphy and Garavan, 2011). The imbalance in the occurrence of each type of stimuli creates a response prepotency that manifests in a difficulty inhibiting responses when required (Murphy and Garavan, 2011). Deficits in inhibitory control have been found in a number of disorders characterized by impulsivity, including ADHD and Tourette’s syndrome, as well as drug misuse (Kamarajan et al., 2005).
Impaired inhibitory control has been identified in individuals with a range of substance-dependence problems (Bickel et al., 2012). Compared to controls, individuals dependent on cocaine (Garavan et al., 2003; Verdejo-García et al., 2007), amphetamine and methamphetamine (Verdejo-García et al., 2008), opiates (Forman et al., 2004; Goldstein and Volkow, 2011), and alcohol (Kamarajan et al., 2005), all showed compromised go/no-go performance. Additionally, Spinella (2002) found smoking status and quantity smoked was associated with go/no-go inhibitory errors. Typical go/no-go tasks appear insensitive, however, at distinguishing nondependent harmful substance users from low-level users. For example, Murphy and Garavan (2011) found go/no-go inhibitory performance did not discriminate nondependent harmful drinkers from low-level drinkers in a logistic regression analysis. Similarly, Fernie et al. (2010) reported poor performance on inhibitory control tasks did not predict alcohol use or misuse. Rossiter et al., 2012 also found inhibitory control performance under neutral conditions did not differentiate nondependent harmful and low-level drinkers. The healthy population, however, are presumably characterized by a range of inhibitory control abilities. Fernie et al. (2010) postulate that in the absence of immediate reward or punishment, go/no-go tasks might be insensitive to detecting response inhibition deficits in the nondependent population. Indeed, harmful nondependent alcohol drinkers have demonstrated improved inhibitory accuracy under rewarding conditions compared to neutral ones, and this effect was significantly different to the performance of low-level drinkers (Rossiter et al., 2012). Thus, while neutral condition go/no-go tasks are not sensitive enough to reveal a range of inhibitory control abilities in nondependent samples, a reward condition appears to provide such individuals with incentive to be impulsive if they are so inclined.
Relatively, few human behavioral studies examine the direct contribution pre-existing cognitive inhibitory control deficits have on vulnerability for drug use, misuse and transition to dependence. Indirect evidence comes from studies involving high-risk groups. For example, Verdejo-García et al. (2008) reviewed several studies that taken together show drug-naïve offspring of substance-dependent parents have greater inhibitory deficits than controls, and that this predicts an elevated incidence of subsequent drug use issues. Likewise, substance use and misuse prevalence is higher among adolescents and adults diagnosed with ADHD (Molina et al., 2007), a disorder associated with go/no-go inhibitory control deficits (Fisher et al., 2011). A number of studies have found a link between neurobehavioral disinhibition in early adolescent—which comprises measures of executive functioning, affect regulation and behavior control—and the incidence of subsequent substance use disorder (Tarter, et al., 2004a,b; Chapman et al., 2007). It is difficult to disentangle the relative contributions of executive dysfunction, emotional dysregulation and disruptive behavior symptomology in these studies. Nonetheless, these papers are often taken as evidence that pre-existing impulsivity contributes to substance-use-disorder vulnerability (Verdejo-García et al., 2008). More direct evidence comes from a longitudinal study of 498 children that showed go/no-go response inhibition in early adolescence predicted illicit drug use and the onset of problems related to alcohol consumption, independent of familial risk and ADHD or conduct disorder symptoms (Nigg et al., 2006).
Animal studies also provide evidence that pre-existing inhibitory control deficits are related to vulnerability for drug use (Perry and Carroll, 2008). Poor inhibitory control in drug naïve rats, for example, was associated with an increased tendency to initiate and maintain self-administration of nicotine (Diergaarde et al., 2008), and predicted higher levels of intravenous cocaine self-administration and escalation of use (Dalley et al., 2007). Evidence also suggests that drug taking directly influences inhibitory control (Verdejo-García et al., 2008). Fillmore and Rush (2006) have demonstrated, for instance, that even small amounts of alcohol attenuate inhibitory control. Moreover, the degree of inhibitory control deficit in chronic cocaine users has been found to correspond proportionally to lifetime cocaine consumption (Colzato et al., 2007). Behavioral evidence thus suggests pre-existing poor inhibitory control is associated with vulnerability for substance-dependence issues, and that drug use further exacerbates this impairment.
In summary, two potent behavioral measures of impulsivity, namely delayed discounting and inhibitory control tasks utilizing reward contingencies, respectively, distinguish individuals in the healthy population who are impulsive for immediate reward and those who have a reduced capacity to control their impulses. While evidence suggests that in substance-dependent populations, chronic drug taking exacerbates this behavioral tendency, animal and human studies demonstrate that in drug naïve populations, it might also precede any drug-related problems. Although other factors are undoubtedly at play, behavioral impulsivity might thus serve as a cognitive marker for susceptibility to future drug use and misuse.
Delayed discounting decisions are mediated via an interaction between prefrontal cognitive and subcortical affective neural circuits (McClure et al., 2004; Hoffman et al., 2008; MacKillop et al., 2012). The affective system, which includes the ventral striatum, amygdala, and anterior insula cortex, is responsible for processing motivation and intrinsic value, including reward (Hoffman et al., 2008; Garavan, 2011; Bickel et al., 2012; MacKillop et al., 2012). It is consequently sensitive to stimuli salience, and shows increased activation when individuals choose immediate rewards (Hariri et al., 2006; Bickel et al., 2012). The cognitive circuit, which includes the dorsal anterior cingulate cortex (dACC) and dorsolateral prefrontal cortex (dlPFC), is involved in comparing alternatives and assessing outcomes (McClure et al., 2004; Hoffman et al., 2008; Bickel et al., 2012; MacKillop et al., 2012). This frontal system shows increased activation when individuals choose delayed rewards (McClure et al., 2004; Hariri et al., 2006; Garavan, 2011; Bickel et al., 2012). In healthy individuals, interplay between these systems facilitates advantageous decision-making (McClure et al., 2004; Hariri et al., 2006). In substance-dependent individuals, both systems show dysfunction; specifically, the affective system is hyperactive, while the cognitive system is hypoactive (Hariri et al., 2006; Hoffman et al., 2008; Garavan, 2011). This dual dysfunction is argued to result in the heightened impulsivity exemplified in delayed discounting tasks (Hoffman et al., 2008).
The dACC also plays a key role in inhibitory control tasks (Kaufman et al., 2003; Forman et al., 2004; Bickel et al., 2012). This area detects response conflict when there is a mismatch between an intended action and its outcome, signaling the right inferior frontal regions of the need for upregulation of cognitive control (Hester and Garavan, 2004; Ridderinkhof et al., 2004; Yücel and Lubman, 2007). Increased activation of the dACC is seen in healthy individuals during errors of commission on go/no-go tasks (Forman et al., 2004). Substance dependence, however, attenuates dACC conflict monitoring, thereby resulting in impaired inhibitory control (Verdejo-García et al., 2007; Yücel and Lubman, 2007). Substance-dependent individuals also show reduced dlPFC activity, which corresponds with the failure to upregulate cognitive control in reaction to the response conflict engendered by inhibitory tasks (Garavan et al., 2008).
It is difficult to determine whether the neural activation patterns seen in substance-dependent populations during delayed discounting and inhibitory control tasks are a consequence of chronic drug use, or if they are a pre-existing risk factor for drug use and misuse (Dalley et al., 2011). Behavioral evidence allows for both possibilities. While it has been argued that chronic drug use results in cortical dysfunction, several recent studies propose there are pre-existing neural deficits in individuals who develop drug-related problems (Dawe et al., 2004; Verdejo-García et al., 2008; De Wit, 2009). A study of adolescents in their mid-teens, for example, showed that activation pattern deficits in the prefrontal cortex during a go/no-go task predicted alcohol and drug use over the following 18 months, even when controlling for past use (Mahmood et al., 2013). Similarly, attenuated prefrontal activation during a go/no-go task in adolescents aged 12–14 years predicted transition to heavy alcohol use more than 4 years later (Norman et al., 2011). More recently, results of a large multidimensional longitudinal study revealed activation patterns of brain regions associated with behavioral measures of impulsivity can predict binge alcohol drinking in adolescence (Whelan et al., 2014). Thus, there is evidence to suggest atypical neural activation during tasks that measure impulsivity predates drug misuse. This lends further support to the notion that behavioral measures of impulsivity are useful tools for identifying those in the healthy population who might be at risk of developing drug-related problems.
Dopamine is implicated in behavioral impulsivity (McClure et al., 2004; Buckholtz et al., 2010). Dopaminergic projections originating in the midbrain substantia nigra and ventral tegmental areas innervate brain regions associated with impulsivity (Dalley et al., 2011). Individual differences in dopamine receptor availability in these regions appear central to variations in impulsive behavior (Goudriaan et al., 2008; Volkow and Baler, 2012). Animal studies have shown that impulsive rats have significantly lower dopamine D2 and D3 receptor availability than their nonimpulsive counterparts (Dalley et al., 2007). In humans, lower availability of D2 and D3 receptors in midbrain areas has likewise been found to correlate inversely with impulsivity levels (Buckholtz et al., 2010; Ghahremani et al., 2012). Low-receptor availability is argued to result in poor dopaminergic tone that contributes to a chronic reward deficit, which, in turn, is argued to predispose such individuals to reward-orientated impulsive behavior in an effort to normalize this deficiency (Young et al., 2004; Goudriaan et al., 2008).
Dopamine has also been identified as a key element in vulnerability for drug-related problems (McClure et al., 2004; Volkow and Baler, 2012). A number of drugs directly impact dopaminergic pathways by blocking the reuptake of dopamine (Hoffman et al., 2006). This results in increased concentrations of the neurotransmitter in mesocorticolimbic areas, including the dACC and ventral striatum, and this is immediately rewarding (Hester and Garavan, 2004; Urban et al., 2010; Goldstein and Volkow, 2011; Nutt et al., 2015). The magnitude of drug-induced dopamine release in the striatum has been found to correlate inversely with D2 and D3 receptor availability (Buckholtz et al., 2010; Nutt et al., 2015). Thus, in addition to poor impulse control, individuals with low-receptor expression have an elevated dopaminergic response to drugs (Buckholtz et al., 2010). Indeed, drug-naïve individuals with low D2 receptor availability self-report increased subjective euphoric effects of psycho-stimulants compared to those with high D2 receptor availability (Volkow et al., 1999). Moreover, the degree of dopamine release in striatal areas has been found to predict the intensity of subjective desire for drugs (Buckholtz et al., 2010). Interestingly, similar processes have been noted in the literature pertaining to obesity. Individuals with reduced dopamine receptor availability have been shown to have a greater propensity to overeat, and this behavior is proposed to compensate for hyporesponsive reward circuitry (Burger and Stice, 2011). As with drugs, ratings of meal pleasantness are associated with the magnitude of dopamine release in the striatum (Burger and Stice, 2011).
In sum, individuals with low D2 receptor expression are argued to engage in impulsive rewarding behavior in an effort to normalize neural reward circuits temporarily (Young et al., 2004). Drug taking, which results in dopamine release in the striatum, is one such rewarding activity (Young et al., 2004). Low D2 receptor availability is thus considered a risk factor for substance dependence (Noble, 2003). Conversely, high D2 receptor availability has been found to be a protective factor against dependence problems in individuals considered at risk due to family history (Volkow et al., 2006).
Dopamine D2 receptor availability is likely a product of a combination of a number of gene variants and various environmental factors. Nonetheless, a recent meta-analysis showed expression of the D2 receptor in healthy individuals is influenced by the single-nucleotide polymorphism (SNP) rs1800497, often termed Taq1A (Gluskin and Mickey, 2016). Reduced striatal D2 receptor availability has been linked to possession of the minor A1 allele of this SNP (Bühler et al., 2015; Eisenstein et al., 2016). Possession of the A1 allele is proposed to result in increased synthesis of dopamine (Bühler et al., 2015). It has been associated with more impulsive choices on delayed discounting tasks in the healthy population (Eisenberg et al., 2007). At the same time, alcohol-dependent individuals with this allele made significantly more errors of commission on inhibitory control tasks than controls (Rodríguez-Jiménez et al., 2006). Compared to healthy controls, possession of the allele has been found to be significantly higher in individuals dependent on alcohol, cocaine, nicotine and opiates, and in those who misuse several substances concurrently (Noble, 2000, 2003). Moreover, adolescents who possessed the allele and were deemed at risk of substance dependence due to family background were more likely than noncarriers to have consumed alcohol, been intoxicated, used more illicit substances, used nicotine regularly and experienced marijuana-induced highs (Conner et al., 2005). In a large meta-analysis of 55 studies, Young et al. (2004) found possession of the allele was a marker for substance use and severe misuse. Interestingly, possession of this allele has also been implicated in mechanisms leading to obesity (Burger and Stice, 2011).
Behavioral impulsivity for reward can thus be understood as a multifaceted and potentially genetically determined mechanism that plays a part in rendering individuals susceptible to problematic drug use. As such, it predates substance dependence, though it is undoubtedly exacerbated by chronic use. This provides further support for using behavioral tasks of impulsivity to assist in identifying individuals in the healthy population who might be at risk of drug-related problems.
The involvement of dopamine in impulsive behavior and substance dependence provides clues to another cognitive factor that might contribute to susceptibility to drug use and misuse. While low D2 receptor availability creates chronic neural dopaminergic deficits that incline at-risk individuals to impulsive behaviors, including drug-related activities, dopaminergic function also underpins reinforcement learning or learning from feedback (Young et al., 2004; Rustemeier et al., 2012).
Schultz (1998, 2007) has demonstrated how dopaminergic neurons of the substantia nigra and ventral tegmental area respond to reward and reward-predicting stimuli. These neurons appear to learn from feedback, encoding the difference between reward and unpredicted rewards (i.e. those that are unexpected based on previous experience) are better than expected and elicit increased dopamine release in the midbrain areas (positive prediction error); predicted rewards (i.e. those that are expected based on previous experience) provoke no response beyond baseline firing; and, predicated rewards that fail to eventuate create a depressed dopaminergic response (negative prediction error; Schultz, 1998; Frank and Claus, 2006; Schultz, 2007; Rustemeier et al., 2012). Increases and decreases in dopamine in midbrain areas are, respectively, positively and negatively reinforcing, and this drives adaptive activity leading to reward is likely to be repeated, but actions that provoke less reward than expected are likely to be avoided (Frank and Claus, 2006; Baker et al., 2011; Baker et al., 2013). Those particularly at risk of drug-related problems, who experience an increased release of dopamine in midbrain areas in response to drugs (for example, due to low D2 receptor expression), are likely to generate exaggerated reinforcing positive prediction errors when taking drugs. This amplified response may guide future behavior such that these individuals are predisposed toward repeating this rewarding behavior (Chiu et al., 2008).
Critically, the ability to learn from both positive and negative reinforcers may be related to dopamine receptor availability (Baker et al., 2013). Utilizing a probabilistic learning task (PLT), Klein et al. (2007) investigated the learning preferences of individuals with low D2 receptor expression in the midbrain. According to this paradigm, participants can be categorized according to their learning some tend to select the stimuli rewarded more frequently and are thus learning from reward, while others attempt to avoid stimuli associated with poor reward, and are therefore learning from aversive outcome (Baker et al., 2013). Klein et al. (2007) found participants with low D2 receptor availability were less able to learn to avoid negative outcomes in the PLT compared to controls. Moreover, while controls demonstrated no clear significant preference for learning from positive compared to negative feedback, low D2 receptor participants were significantly more likely to learn from positive compared to negative feedback (Klein et al., 2007). Learning from feedback is thus hypothesized to be reliant upon effective dopaminergic low D2 receptor availability results in poor tone and is associated with attenuated learning from negative reinforcers and increased sensitivity to positive reinforcers (Klein et al., 2007). This may explain why substance-dependent individuals, who are typically characterized by poor dopaminergic tone, appear overly sensitive to the rewards of drug taking, but are insensitive to the negative consequences of their behavior (Klein et al., 2007). Furthermore, it also suggests nondependent persons with a genetically mediated predisposition for impulsive behavior (and drug use and misuse problems) might also have a reduced ability to learn from negative feedback, but an enhanced capacity for learning from positive feedback.
Although a preferential bias for learning from rewarding relative to punishing outcomes explains why some individuals may progress from initial drug use to more frequent use, it does not necessarily account for why persons with substance use disorders persist at their behavior despite often catastrophic, delayed negative consequences. Animal models provide some insight into this phenomenon. After an extended period of cocaine self-administration, for example, highly impulsive rats demonstrated greater resistance to punishment than their less impulsive counterparts (Frank et al., 2004). When high- and low-impulsivity rats with equivalent cocaine intake were subjected to foot shocks, both groups decreased their intake, but then resumed self-administration after a period of enforced abstinence (Economidou et al., 2009). Following a subsequent punishment and abstinence phase, however, only the highly impulsive animals reinstated self-administration a second time (Economidou et al., 2009). The Iowa gambling task (IGT) has been used to investigate punishment resistance in substance-dependent individuals (Grant et al., 2000; Bechara, 2003; Dom et al., 2005; Yücel and Lubman, 2007; Goudriaan et al., 2008). The task requires participants to learn to choose cards from advantageous and disadvantageous decks in order to maximize net gains and minimize net losses (Bechara et al., 2001; Bechara, 2003; Fridberg et al., 2010). Substance-dependent individuals have consistently been shown to perform poorly on the IGT, generally favoring choices that result in small, immediate gains but large losses over time (Grant et al., 2000; Dom et al., 2005; Fridberg et al., 2010). It is hypothesized that such individuals persist at immediately rewarding behavior despite aversive delayed consequences because they are insensitive to punishing feedback (Grant et al., 2000; Fridberg et al., 2010). Additionally, a large subgroup of substance-dependent individuals has been found to have an abnormally elevated physiological response to reward during a variation of this task, thereby reflecting hypersensitivity to reward (Bechara et al., 2002). Collectively, these results inform the substance-dependence behavior of these they appear unduly influenced by the immediate rewards of drug taking and fail to learn from the delayed negative consequences of their actions (Bechara et al., 2002).
Although results from studies utilizing both the PLT and IGT provide insight into preferential learning from rewarding and punishing feedback, there are some irregularities that suggest these results should be treated cautiously. Rustemeier et al. (2012), for example, were unable to identify preferential learning from positive or negative feedback using the PLT in their alcohol-dependent sample. At the same time, the complexity of the IGT makes it difficult to determine whether disadvantageous choices reflect poor learning from negative feedback, deliberate risk-taking or even difficulty using risk to inform decision-making (Upton et al., 2011). In fact, several authors have proposed that the IGT taps risk discounting (Monterosso et al., 2001; Dom et al., 2007). The probabilistic nature of both the PLT and IGT means it is difficult to assess if these tasks solely quantify learning or if they are also a measure of risk-taking. Certainly, risk-taking is linked to impulsivity in the substance-dependence literature (Jupp and Dalley, 2014), but an investigation into the association between impulsivity for, and learning from, reward needs to minimize any risk-taking confound. Novel learning tasks are thus required in order to investigate this phenomenon more fully in dependent and healthy human populations.
Recently, consideration has been given to how neuroeconomics, particularly prospect theory, might extend our understanding of substance use and misuse (Monterosso et al., 2012; Melrose et al., 2015; Guttman et al., 2018). A central tenet of neuroeconomics concerns valuation and how this is represented at the neural level (Monterosso et al., 2012). Valuation is described as a process by which individuals weigh up—or value—alternatives so as to select the most advantageous option (Guttman et al., 2018). It is thought to be impacted by various factors, including an individual’s unique point of reference (which shifts over time), the timing of the outcome and the risk or certainty involved; it can also be shaped by framing, imposed either externally or internally (Smith and Huettel, 2010; Melrose et al., 2015; Guttman et al., 2018). Prospect theory has been proposed as a means by which these factors—and others—might be taken into consideration when examining the seemingly irrational decision-making of individuals (Melrose et al., 2015). For instance, prospect theory provides a mechanism for explaining why discounting—in delay discounting tasks—does not occur at a constant rate (Monterosso et al., 2012; Melrose et al., 2015; Guttman et al., 2018). It accounts for why individuals choose 16 tomorrow, but they select 15 in 99 days (Monterosso et al., 2012; Guttman et al., 2018). At least, one commentator has speculated prospect theory might also offer insight into delayed discounting, inhibitory control and reinforcement learning processes in the context of addiction (Monterosso et al., 2012). Importantly, given addictive behavior can be interpreted as a succession of decisions with changing reference points, and thus shifting gain vs loss valuations, prospect theory might ultimately also provide a framework for understanding within-person substance-misuse trajectories (Melrose et al., 2015). At present, however, relatively few empirical papers consider even between-subject differences in the area of addiction from the perspective of neuroeconomics (Smith and Huettel, 2010; Melrose et al., 2015). It is nonetheless an interesting area for future research.
Studies investigating impulsivity, that is, impulsive decision-making and impaired inhibitory control, as well as those examining learning from feedback, consistently reveal reward sensitivity is heightened among individuals with substance use disorders. Individuals characterized by subclinical problematic drug use also exhibit elevated sensitivity for reward. Critically, healthy individuals considered at risk of developing a substance use disorder—due to dopaminergic deficits related to low D2/3 receptor expression—likewise demonstrate increased reward sensitivity. This has led to assertions that sensitivity for reward constitutes an important indicator of substance use disorder vulnerability. Certainly, increased reward sensitivity may explain why individuals with, or at risk of developing, a substance use disorder tend to consume drugs in larger amounts and for longer periods than anticipated, but this construct fails to account for the propensity of these individuals to continue taking drugs despite associated negative consequences. We propose this aspect of substance use disorder may relate to punishment insensitivity. Indeed, animal and human studies indicate substance use disorder attenuates sensitivity to punishing reinforcers. Importantly, substance use disorder at-risk healthy individuals (with dopaminergic deficits associated with low D2/3 receptor expression) also demonstrate reduced learning from negative feedback. Although reward sensitivity and punishment insensitivity are both underpinned by a compromised dopaminergic system, few studies using behavioral tasks take both these processes into account when considering substance-use-disorder susceptibility. While further research is required, if the same dopamine-related mechanism mediating reward sensitivity also underpins punishment insensitivity, then both these processes may serve as dual pre-existing markers for substance use disorder vulnerability.
This research was supported by an Australian National Health and Medical Research Council grant (1050766) and an Australian Research Council fellowship (FT110100088). The funding bodies had no role in writing the report or in the decision to submit the manuscript for publication.
No conflict declared.
Antoinette Poulton, Melbourne School of Psychological Sciences, University of Melbourne, Parkville 3010, VIC, Australia.
Robert Hester, Melbourne School of Psychological Sciences, University of Melbourne, Parkville 3010, VIC, Australia.