Extinction learning deficit in a rodent model of attention-deficit hyperactivity disorder
© Brackney et al.; licensee BioMed Central Ltd. 2012
Received: 5 March 2012
Accepted: 25 November 2012
Published: 13 December 2012
Skip to main content
© Brackney et al.; licensee BioMed Central Ltd. 2012
Received: 5 March 2012
Accepted: 25 November 2012
Published: 13 December 2012
Deficient operant extinction has been hypothesized to be constitutive of ADHD dysfunction. In order to elucidate the behavioral mechanisms underlying this deficit, the performance of an animal model of ADHD, the spontaneously hypertensive rat (SHR), was compared against the performance of a control strain, the Wistar-Kyoto rat (WKY) during extinction.
Following extensive training of lever pressing under variable interval schedules of food reinforcement (reported previously), SHR and WKY rats were exposed to two sessions of extinction training. Extinction data was analyzed using the Dynamic Bi-Exponential Refractory Model (DBERM) of operant performance. DBERM assumes that operant responses are organized in bouts separated by pauses; during extinction, bouts may decline across multiple dimensions, including frequency and length. DBERM parameters were estimated using hierarchical Bayesian modeling.
SHR responded more than WKY during the first extinction session. DBERM parameter estimates revealed that, at the onset of extinction, SHR produced more response bouts than WKY. Over the course of extinction, response bouts progressively shortened for WKY but not for SHR.
Based on prior findings on the sensitivity of DBERM parameters to motivational and schedule manipulations, present data suggests that (1) more frequent response bouts in SHR are likely related to greater incentive motivation, and (2) the persistent length of bouts in SHR are likely related to a slower updating of the response-outcome association. Overall, these findings suggest specific motivational and learning deficits that may explain ADHD-related impairments in operant performance.
Multiple theories postulate abnormalities in operant conditioning as a behavioral phenotype of attention deficit hyperactivity disorder (ADHD) [1, 2]. Some of these theories involve response extinction—the decline in behavior once reinforcement is discontinued—as an aspect of operant performance compromised in ADHD [3, 4]. In this regard, theories often make conflicting predictions. Sagvolden and colleagues’ dynamic developmental theory , for instance, predicts slower extinction in individuals with ADHD; Tripp and Wickens’ dopamine transfer deficit theory  makes the opposite prediction. Empirical evidence that would adjudicate this dispute is surprisingly scarce and difficult to interpret. For instance, Sagvolden and colleagues  showed that children with ADHD responded more during extinction than controls; extinction contingencies, however, alternated with positive reinforcement, and schedule interaction effects were not ruled out. Other studies have shown stronger emotional responses to non-reinforcement in children with ADHD [6–8], but very weak differences relative to controls in extinction performance . Research on reversal, omission , and Pavlovian extinction learning  provide indirect evidence that operant extinction may be slower in individuals with ADHD.
The spontaneously hypertensive rat (SHR), a common animal model of ADHD , typically emits higher rates of operant responding under maintenance (when reinforcement is effective and performance is relatively stable) and extinction conditions, relative to the Wistar-Kyoto (WKY) control strain [12, 13]. Differences in response rate during extinction, however, only support very limited inferences on extinction deficits, for two reasons. First, response rate during extinction is highly dependent on preceding maintenance response rates . Differences in responding during extinction, therefore, may be due to differences in responsiveness to reinforcement during maintenance, and not due to a fundamental problem with extinction itself . Second, the response rate measure conflates various performance parameters that may be differentially sensitive to extinction contingencies. These parameters arise from the organization of operant behavior in bouts of responses separated by relatively long pauses [16–18].
The Bi-Exponential Refractory Model (BERM) [18, 19] of steady-state free-operant maintenance provides estimates of response-bout parameters, including bout initiation rate, response rate within bouts, and average bout length. BERM has been described in detail elsewhere . Briefly, BERM assumes that animals initiate bouts following one Poisson process and, once a bout is initiated, emit responses according to a second, faster, Poisson process. A bout may be exited from with some probability following each response. BERM’s dynamic generalization, DBERM, assumes that a subset of BERM parameters may decline exponentially over time in the absence of reinforcement, such as during extinction. These dynamic components allow for the dissociation of starting parameter estimates (which are dependent on maintenance conditions) from their rate of decline over the course of extinction. This dissociation facilitates the identification of differences between groups, such as the WKY and SHR, in the microstructural dynamics responsible for the decline in response rate during extinction, while minimizing confounds that may arise from unequal starting values.
BERM parameters are differentially sensitive to various experimental manipulations [18, 19]. For instance bout initiation rate appears to be particularly sensitive to motivational changes such as food deprivation and reduced reinforcement density. Bout length and within-bout response rate appear to be sensitive primarily to reinforcement contingencies , and may thus be indicative of the strength of the response-reinforcer association . BERM and DBERM parameters may, therefore, identify behavioral and cognitive endophenotypes underlying differences in performance between animal models of psychiatric disorders and their controls.
The purpose of the present study was to identify the parameters of operant behavior, as characterized by DBERM, that underlie the observed differences in response rate between SHR and WKY during extinction. Using DBERM allows for the isolation of effects of maintenance contingencies on extinction performance. In addition, strain differences in DBERM parameters may suggest differential sensitivities to contingencies of reinforcement, or differences in the underlying behavioral processes controlling behavior.
Twelve male young-adult (PND 94–95) rats, 6 SHR/NCrl (Charles River Laboratories, US) and 6 WKY/NHsd (Harlan Laboratories, US), were used. WKY served as normoactive control for SHR . Rats were food restricted and maintained at approximately 85% of their ad libitum weight based on a logistic function fitted to the growth curves provided by breeders. Each rat’s weight was assessed daily, and any negative difference from his expected weight was compensated with an equal weight of food presented an hour after the experimental session terminated. Every rat was fed a minimum of 2 g of post-session food each day, and weighed approximately 240 g at the start of the experiment. Before extinction testing, all subjects participated in a previous study , in which additional details on the subjects and apparatus are described. All procedures in the present study were conducted according to the guidelines of the National Institutes of Health, which were approved by the Institutional Animal Care and Use Committee at Arizona State University.
Experimental sessions were conducted in six MED Associates® operant chambers with the standard dual lever configuration; each retractable lever flanked a food receptacle aperture. Food dispenser activation deposited a single 45-mg food pellet into receptacle. During sessions, only the right lever (nearest the door) was extended.
Prior to extinction testing, all subjects were trained for 54 days on a multiple variable interval (VI) schedule of food reinforcement, operative on a single lever, with schedules ranging between VI 12-s to VI 192-s. The purpose of that training was to assess changes in BERM parameters across multiple schedules of reinforcement in SHR and control strains. The outcome of this assessment is reported in detail elsewhere . Because the multiple-VI training protocol is also described in detail in that report, it is only outlined here. Each session was preceded by a 5-min acclimation period in which no programmed events occurred. Lever access followed the acclimation period for 65 min or 40 reinforcer deliveries, whichever occurred first. A VI schedule was randomly selected without replacement at the beginning of each session from a list of five available schedules (VI 12, 24, 48, 96, or 192-s). An inter-trial interval of 20 s followed each reinforcer (food pellet) delivery, during which the lever was retracted. Each of the five VI schedules was continuously signaled by a pulsing tone of a distinct frequency (3–12 kHz). After eight reinforcers were delivered on a schedule, a new schedule was selected without replacement. Each lever press was followed by a 0.11-s refractory period in which lever pressing was not effective. This refractory period was imposed to prevent the recording of artifactual lever bounces.
Extinction sessions began the day after the final VI training session. Extinction sessions were 65-min long (excluding the preceding 5-min acclimation period), and were identical to VI maintenance sessions in all respects except that only the tone signaling the VI 192-s schedule was sounded and lever pressing had no programmed consequences. Two extinction sessions were conducted over consecutive days.
Data were analyzed at two levels. At a macrostructural level, the effects of strain (SHR vs. WKY), extinction session (EXT1 vs. EXT2), and time in extinction (eight 8.125-min bins in each session) on log response rate were assessed using a mixed-design ANOVA. Response rates were calculated for each individual rat by dividing the total number of responses within each bin by 8.125 min. ANOVA was conducted in IBM SPSS Statistics® v. 20.
DBERM was fit to extinction IRTs to identify the sources of ostensible between-strain differences in response rate. DBERM assumes that free-operant performance is described by four separate parameters, three of which may decline as a function of time in extinction t. They are the bout initiation rate b t , or rate at which the subject engages in lever-pressing activity; the within-bout response rate w t , or speed at which the lever is activated while engaged; the average bout length L t , which is the mean number of responses in a bout after the response that initiated the bout; and the refractory period δ, or minimum time required to emit a response [18–20].
Estimated DBERM parameter medians (95% credible interval) for each strain in EXT1
Mean bout length† (resp)
2.81 (1.79 – 4.27)
Within-bout response rate (min-1)
253.71 (90.76 – 687.57)
175.68 (99.49 – 307.82)
Bout-initiation rate (min-1)
42.74 (27.54 -70.92)*
14.70 (8.76 – 25.14)*
Half-life of L (min)
2794.48 (79.50 – 6.9 × 105)*
19.37 (9.35 – 55.72)*
Half-life of w (min)
42.94 (9.77 – 244.32)
21.28 (5.80 – 104.20)
Half-life of b (min)
7.77 (4.81 – 12.88)
4.69 (1.58 – 11.26)
Refractory period (s)
0.11 (0.11 – 0.11)
0.12 (0.10 – 0.15)
Asymptotic response rate (min-1)
2.37 (0.81 – 6.76)
1.64 (1.07 – 2.50)
where C(x) denotes the recovery coefficient for parameter x. For example, if the baseline bout length on EXT2 was half of that on EXT1, then C(L 0 ) would be 0.5. This model therefore assumes a multiplicative change of parameters between sessions, analogous to the within-session exponential decay assumed by DBERM. If a parameter did not change between sessions, the estimate of its recovery coefficient would be close to 1.
Estimated recovery coefficient medians (95% credible interval) for each strain in EXT2
C(L 0 )
1.44 (0.89 – 2.41)*
0.50 (0.25 – 1.04)*
C(w 0 )
0.85 (0.55 – 1.34)
0.93 (0.76 – 1.14)
C(b 0 )
0.73 (0.55 – 0.94)
1.24 (0.51 – 3.41)
C(H L )
0.15 (0.00 – 128.70)
1.27 (0.46 – 9.47)
C(H w )
0.52 (0.18 – 1.48)
1.51 (0.77 – 6.65)
C(H b )
0.44 (0.22 – 0.89)
0.43 (0.15 – 1.05)
1.00 (0.98 – 1.00)
1.09 (0.85 – 1.36)
0.37 (0.26 – 0.58)*
0.70 (0.49 – 1.02)*
For a given DBERM parameter x, the measure of interest was the mean of the posterior distribution of each strain, μx(SHR) and μx(WKY). Inferences about the differences between strains were based on the posterior distributions of the (unstandardized) effect size, E x = μx(SHR) – μx(WKY). The mean and the central 95% credible interval (CI) of each of these posterior distributions were estimated using Markov chain Monte Carlo (MCMC) [22, 26, 27]. The 95% CI contains 95% of the mass of the posterior distribution. A significant difference between the strains was declared for model parameter x if the 95% CI of E x did not include 0. The method for MCMC sampling is described in detail elsewhere . A total of 20100 MCMC samples of the joint posterior distribution were collected.
In order to aid interpretation, the posterior estimates of the log-normal group mean parameters, effect sizes, and their 95% CIs were back-transformed to the linear scale using exponentiation. The back-transformed μ x is an estimate of the median DBERM parameter x for a strain on the linear scale. The linear effect size is an estimate of the ratio of the (linear) group medians of the two strains, and a significant difference implies that the 95% CI of this ratio does not include 1.
Mean (+/− SEM) response rates during maintenance and extinction sessions 1 and 2 (EXT1 and EXT2) are shown as dots in Figure 1. Each extinction session was divided into eight bins of 8.125 min. A 2 × 2 × 8 (strain: SHR vs. WKY, session: EXT1 vs. EXT2, bin: 1 vs. 2 vs. … 8) mixed-design ANOVA was conducted with log response rate as the dependent measure. A significant main effect of strain was detected, F(1,10) = 6.93, p = 0.025, as well as a significant main effect of session, F(1, 10) = 93.57, p < 0.001. A strain × session interaction effect was also detected, F(1, 10) = 5.82, p = 0.037. Simple main effects tests revealed that SHR responded substantially more than WKY during EXT1, F(1, 10) = 10.79, p = 0.008, but not during EXT2, F(1,10) = 2.40, p = 0.152.
A significant main effect of bin, F(7, 70) = 71.52, p < 0.001, and a session × bin interaction, F(7,70) = 2.36, p = 0.032, were also observed. A pairwise comparison of response rates in bin 1 across sessions found that initial responding in EXT1 was significantly greater (p = 0.004) than in EXT2. In EXT1, pairwise comparisons between consecutive bins indicated that response rate in each bin was significantly higher (p ≤ 0.05) than the response rate in the next bin for all but bins 5 vs. 6. In contrast, pairwise comparisons of consecutive bins in EXT2 revealed a significant difference (p ≤ 0.05) of response rate only between bins 1 vs. 2, 2 vs. 3, and 7 vs. 8, suggesting that response rate had reached near asymptotic levels after the third bin. There were no significant strain × bin or strain × bin × session interaction effects on log response rate. Combined, these results indicate that (1) SHR responded substantially more than WKY in EXT1, (2) this difference between strains subsided by EXT2, (3) regardless of strain, responding at the start of the session was substantially faster in EXT1 than EXT2, and (4) response rate decayed to near asymptotic levels substantially faster in EXT2 than in EXT1.
DBERM was fit to 19,184 individual IRTs collected from all animals in both EXT sessions. Parameter estimates for individual subjects may be found in the supplementary material - Additional file 1: Tables S1 and S2. Table 1 summarizes the posterior distributions of parameters for EXT1, on the linear scale. The estimate of the SHR median b 0 was almost 3 times larger than that of WKY, indicating that, at the onset of extinction, SHR produced substantially more response bouts than WKY. The half-lives of L t , w t , and b t for EXT1 are also included in Table 1. Only H L varied significantly between strains: the median bout length of SHR was virtually constant during extinction (H L = 2794.48 min)a, whereas WKY’s declined to half of its baseline estimate within 20 min. It is possible that SHR’s long H L was due to the group’s low L 0 . That is, SHR could have emitted too few within-bout responses for the parameters associated with the within-bout state to be estimated accurately. However, a simulation experiment (described in section 2 of the supplementary material) demonstrated that even with a lower L 0 , H L can be estimated with reasonable accuracy, suggesting that the lack of evidence for SHR’s L 0 declining during the session, and the differences in H L between SHR and WKY, were not statistical artifacts.
Parameter estimates for EXT2 are conveyed as recovery coefficients (Equation 4) in Table 2. Significant between-strain differences were observed only in C(L 0 ) and C(Ω). These results indicate that (1) bout length at the onset of extinction declined less between sessions for SHR than for WKY, and (2) asymptotic response rate appeared to decline more between sessions for SHR than for WKY.
The goodness of fit of DBERM was validated using posterior predictive check. Ten thousand samples of the joint posterior distribution of DBERM parameters of individual rats were randomly selected without replacement from the MCMC output. For each sample, the following Monte Carlo simulation of EXT1 and EXT2 was conducted. For each rat, an IRT starting at session time t n (IRT n , where n is a response counter) was generated using the following algorithm: (1) Update the rat’s dynamic parameters (L t , w t , b t ) at t n . This was accomplished by substituting t n and the sampled DBERM parameters for the rat into Equation 3. Sampled recovery coefficients were also used to compute DBERM parameters for EXT2. (2) Randomly sample the bout state at t n . The probability of being in the within-bout state was p t ; the probability of being in the between-bout state was 1 – p t . (3) Randomly sample a pause of duration τ n from an exponential distribution with mean pause of either 1/w t or 1/b t depending on the sampled bout state. (4) Add δ to the sample of τ n to give IRT n . After generating IRT n , the session timer was increased to tn+1 = t n + IRT n , and IRTn+1 was generated by repeating from step 1 with tn+1. This iterative procedure was carried out starting from t1 = 0, for each rat, for each of the two extinction sessions, until the session time t exceeded the experimental session time of 65 min. This generated a prediction of how every rat would respond conditional on the observed experimental data and the model constraints imposed by DBERM. The simulated overall response rates for the SHR and WKY groups, averaged over the 10000 samples of the posterior distribution, are reported in Figure 1 as curves. Simulations closely tracked the changes in response rate observed during both extinction sessions, thus validating DBERM and the extinction parameter estimates in Tables 1 and 2 as reasonable descriptions of extinction in SHR and WKY.
A previous paper by Cheung et al.  references the current data set for illustrative purposes. Differences in data analysis between this paper and  are discussed in supplemental material section 4.
This study was aimed at identifying the aspects of operant responding that are responsible for the difference in extinction performance between SHR and WKY. As this study and others [17–19] indicate, free operant responding is composed of multiple underlying components that change over the course of extinction. These changes may reflect the numerous processes implicated in extinction learning, such as reductions in arousal or incentive motivation [28–30], and learning new associations in an unreinforced context [31, 32], among many others (see [31, 33] for reviews). A precise mapping of these processes to changes in response-bout microstructure during extinction  would provide a foundation for inferences about the processes responsible for differences in performance between strains.
In EXT1 of the current study, SHR responded more than WKY, which is consistent with prior studies that compared the extinction performance of these strains [3, 34, 35]. DBERM parameter estimation revealed that SHR emitted more response bouts at the onset of extinction. Because deprivation level and reinforcement density covary positively with bout initiation rate [16, 18, 20, 36–39] the difference in mean b 0 between strains suggests a heightened motivation for incentives in SHR. This heightened motivation has been proposed as a fundamental component of operant hyperactivity in SHR . Effects on b 0 are likely carried over from differences in maintenance performance, which means that the difference in b 0 between strains is not unique to extinction performance. During extinction the context of reinforcement incites responding [28, 40], presumably influencing b. In this study, we observed no significant difference in H b and C(b0) between strains, which also suggests that differences in extinction performance between SHR and WKY cannot be explained by differences in the decline of context-elicited arousal .
We also observed significant difference of C(Ω) between strains. The steeper reduction of Ω in SHR between extinction sessions undermines the notion that Ω is simply the operant level of the response. Parameter Ω and its change between sessions may depend on unknown factors that decline at a slow rate over extinction, such as the occasion-setting properties of the context . Further investigation into the appropriate estimation and the theoretical interpretation of Ω is required before sound behavioral inferences based on Ω can be made.
Of the theoretical perspectives that specifically address response extinction, the present findings are most consistent with Sagvolden’s dynamic development theory of ADHD . This theory predicts slower extinction in individuals with ADHD compared to neurotypical controls. The dynamic development theory assumes that individuals with ADHD have shortened delay of reinforcement gradients. A shorter delay of reinforcement gradient would attenuate the length of response sequences that are reinforced , which would be reflected in shorter initial bout lengths (L 0 ). While a significant difference of L 0 was not observed in EXT1, we did detect a trend in L 0 consistent with Sagvolden’s theory and with estimates from maintenance performance .
DBERM parameter estimation also demonstrated that bout length decayed over the course of extinction for WKY, but not for SHR. Bout length is particularly sensitive to schedule manipulations even when reinforcement rate remains constant [18, 39, 42]. Such sensitivity suggests that bout length is an indicator of the strength of the response-outcome association [1, 18]. During extinction, the response-outcome association is expected to weaken (even if it is not completely obliterated ), a process that may be expressed as a progressive reduction in bout length. Therefore, it is likely that, after operant conditioning on VI schedules, the absence of decay in bout length in SHR during extinction is indicative of a slower updating of the response-reinforcer association. This slower process may reflect, in turn, a fundamental learning deficit. The strain difference in the between-session change of L 0 suggests that the slow associative updating process attributed to SHR within EXT1 also operated between extinction sessions.
An alternative explanation of the constant bout length observed in SHR during extinction may attribute such effect to the low baseline bout length of SHR. That is, SHR bouts may have declined very little in length, relative to WKY bouts, because SHR bouts were already very short, and not because of any learning deficit in SHR. This explanation is unlikely to be valid for two reasons, besides the statistical non-significance in the difference of L0 between strains. First, median DBERM parameter estimates indicate that WKY bouts declined in length at such rate that SHR and WKY bouts were of equal length around the middle of the EXT1 (more precisely, after 31.73 min). That is, the same parameter estimates from which the constant bout length of SHR is inferred suggest that WKY bouts were shorter than SHR bouts during the second half of EXT1. If SHR bouts were too close to the minimal bout length, then the model would have underestimated WKY bout length and response rate during the second half of EXT1, a divergence that is not observed in the simulation (cf. Figure 1). Second, we simulated extinction performance using individual SHR parameters, except that H L was sampled from WKY estimates (described in supplemental material, section 2). When parameters were recovered from the simulation, we found no evidence of bias: the credible intervals of the recovered parameters enveloped the parameters that produced the simulated data. We did not find evidence of bias even when the generative SHR L0 was shortened by a factor of 3. This absence of bias suggests that the DBERM was sensitive enough that, had bout length in SHR declined at a rate comparable to that of WKY, DBERM would have detected it.
It may seem surprising that the SHR median within-bout response rate is estimated at over 250 resp/min. Note however that the minimum time between responses (δ) has been observed to be on the order of 0.1 s across multiple experiments [18, 19], indicating a “top speed” of 600 resp/min, although this rate is not typically sustained over long intervals due to the pause-and-bout pattern of responding. In addition, within-bout response rates exclude δ (cf. Equation 1). With these considerations, it can be found that the currently observed within-bout response rates are well within a plausible range.
Potential issues regarding data interpretation may also arise from the multiple-VI training that preceded the current experiment. It is unlikely that the particular order in which VI schedules were presented in the last maintenance session influenced extinction performance, because such order was randomized in every multiple-VI session. However, it is possible that the current results are idiosyncratic to extinction following multiple (as opposed to simple) VI training. To assess the generality of our findings, similar comparisons of SHR and WKY during extinction should be conducted following training with a single VI.
It should also be noted that DBERM is a descriptive model. Currently it has 8 parameters describing dynamic responding within an extinction session (Table 1). Previous work using Akaike information criterion with model selection suggests that all 8 parameters describing within-session responding are required . Instead of having a new set of 8 DBERM parameters for each extinction session, we used recovery coefficients for each DBERM parameters as a way to describe their changes across multiple extinction sessions (Table 2). The recovery coefficients assumed that DBERM parameters change multiplicatively across extinction sessions. Further experiments will be needed to examine how well multiplicative recovery coefficients describe extinction data beyond the first two extinction sessions.
At the beginning of extinction SHR emitted more response bouts (b 0 ) whose length declined at a slower rate H L and C(L0)]. This implies that SHR emitted more responses than WKY following the discontinuation of operant reinforcement. The difference in baseline bout-initiation rate most likely reflects differences in maintenance performance carried over to extinction, and not differences in extinction learning itself. In contrast, the persistent length of bouts in SHR suggests deficient extinction learning in SHR. Because changes in bout length are primarily related to schedule effects [18, 20, 39, 42], it is likely that the persistent length of SHR bouts reflects a low sensitivity to changes in reinforcement schedule. This inference is consistent with findings of slower autoshaping in adult SHR , and with the slow responsiveness of individuals with ADHD to changes in reinforcement contingencies [9, 10, 44]. Thus, the identification of extinction learning deficits in SHR supports its use as an animal model of ADHD-related learning deficits, which may be involved in some varieties of hyperactivity . In general terms, this evidence provides further support for the use of SHR as an animal model of ADHD [11, 45].
a Estimates of H L were capped at 5.6 × 106 min (approximately 10 years). Half lives are capped because of the possibility of infinite half-lives, which can lead to the posterior distributions of the decay rate parameters (γ, α, β) to be improper, i.e., the posterior distribution integrates to infinity .
Attention deficit hyperactivity disorder
Bi-exponential refractory model
Dynamic bi-exponential refractory model
Extinction sessions 1 and 2
Spontaneously hypertensive rat strain
Wistar-Kyoto rat strain
Variable interval schedule.
This research was supported by DA011064 (Timothy Cheung, awarded to Janet Neisewander), DA032632 and MH094562 (Federico Sanabria). The authors would like to thank Janet Neisewander, Peter Killeen, Gabriel Mazur and Elizabeth Watterson for their helpful comments and support while developing this manuscript. The authors would also like to thank Richard Denton, Greg Edwards, Kristina Gerencser, Allison Lucas, Jonathan Schiro, and Lauren Shields for data collection.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.