Schultz W. Neuronal reward and decision signals: from theories to data. Physiol Rev. 2015;95(3):853–951.
Article
CAS
PubMed
PubMed Central
Google Scholar
Kim HF, Hikosaka O. Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards. Brain. 2015;138(7):1776–800.
Article
PubMed
PubMed Central
Google Scholar
Chase HW, Kumar P, Eickhoff SB, Dombrovski AY. Reinforcement learning models and their neural correlates: an activation likelihood estimation meta-analysis. Cogn Affect Behav Neurosci. 2015;15(2):435–59.
Article
PubMed
PubMed Central
Google Scholar
Ikemoto S, Bonci A. Neurocircuitry of drug reward. Neuropharmacology. 2014;76:329–41.
Article
CAS
PubMed
Google Scholar
Lee D, Seo H, Jung MW. Neural basis of reinforcement learning and decision making. Annu Rev Neurosci. 2012;35:287–308.
Article
CAS
PubMed
PubMed Central
Google Scholar
Daw ND, Dayan P. The algorithmic anatomy of model-based evaluation. Philos Trans R Soc Lond B Biol Sci. 2014;369(1655):20130478.
Article
PubMed
PubMed Central
Google Scholar
O’Doherty JP. Contributions of the ventromedial prefrontal cortex to goal-directed action selection. Ann N Y Acad Sci. 2011;1239(1):118–29.
Article
PubMed
Google Scholar
Frank MJ, Claus ED. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev. 2006;113(2):300–26.
Article
PubMed
Google Scholar
Niv Y, Daw ND, Joel D, Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology. 2007;191(3):507–20.
Article
CAS
PubMed
Google Scholar
Salamone JD. The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res. 1994;61(2):117–33.
Article
CAS
PubMed
Google Scholar
Salamone JD, Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res. 2002;137(1):3–25.
Article
CAS
PubMed
Google Scholar
Salamone JD, Correa M. The mysterious motivational functions of mesolimbic dopamine. Neuron. 2012;76:470–85.
Article
CAS
PubMed
PubMed Central
Google Scholar
Guitart-Masip M, Beierholm UR, Dolan R, Duzel E, Dayan P. Vigor in the face of fluctuating rates of reward: an experimental examination. J Cogn Neurosci. 2011;23(12):3933–8.
Article
PubMed
Google Scholar
Beierholm U, Guitart-Masip M, Economides M, Chowdhury R, Düzel E, Dolan R, Dayan P. Dopamine modulates reward-related vigor. Neuropsychopharmacology. 2013;38:1495–503.
Article
CAS
PubMed
PubMed Central
Google Scholar
Floresco SB. The nucleus accumbens: an interface between cognition, emotion, and action. Annu Rev Psychol. 2015;66:25–52.
Article
PubMed
Google Scholar
Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD. Mesolimbic dopamine signals the value of work. Nat Neurosci. 2016;19(1):117–26.
Article
CAS
PubMed
Google Scholar
Mowrer OH. A stimulus-response analysis of anxiety and its role as a reinforcing agent. Psychol Rev. 1939;46:553–65.
Article
Google Scholar
Bolles RC. The avoidance learning problem. Psychol Learn Motiv. 1972;6:97–139.
Article
Google Scholar
Grossberg S. A neural theory of punishment and avoidance, I: qualitative theory. Math Biosci. 1972;15(1):39–67.
Article
Google Scholar
Johnson JD, Li W, Li J, Klopf AH. A computational model of learned avoidance behavior in a one-way avoidance experiment. Adapt Behav. 2001;9(2):91–104.
Article
Google Scholar
Maia TV. Two-factor theory, the actor-critic model, and conditioned avoidance. Learn Behav. 2010;38(1):50–67.
Article
PubMed
Google Scholar
Moutoussis M, Bentall RP, Williams J, Dayan P. A temporal difference account of avoidance learning. Network. 2008;19(2):137–60.
Article
PubMed
Google Scholar
Boureau YL, Dayan P. Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology. 2010;36(1):74–97.
Article
PubMed
PubMed Central
CAS
Google Scholar
Guitart-Masip M, Duzel E, Dolan R, Dayan P. Action versus valence in decision making. Trends Cogn Sci. 2014;18(4):194–202.
Article
PubMed
PubMed Central
Google Scholar
Bandler R, Keay KA, Floyd N, Price J. Central circuits mediating patterned autonomic activity during active vs. passive emotional coping. Brain Res Bull. 2000;53(1):95–104.
Article
CAS
PubMed
Google Scholar
McNaughton N, Corr PJ. A two-dimensional neuropsychology of defense: fear/anxiety and defensive distance. Neurosci Biobehav Rev. 2004;28:285–305.
Article
PubMed
Google Scholar
Bolles RC. Species-specific defense reactions and avoidance learning. Psychol Rev. 1970;77(1):32–48.
Article
Google Scholar
Blanchard RJ, Flannelly KJ, Blanchard DC. Defensive behaviors of laboratory and wild rattus norvegicus. J Comp Psychol. 1986;100(2):101–7.
Article
CAS
PubMed
Google Scholar
Mobbs D, Kim JJ. Neuroethological studies of fear, anxiety, and risky decision-making in rodents and humans. Curr Opin Behav Sci. 2015;5:8–15.
Article
Google Scholar
Maier SF, Amal J, Baratta MV, Paul E, Watkins LR. Behavioral control, the medial prefrontal cortex, and resilience. Dialogues Clin Neurosci. 2006;8(4):397–406.
PubMed
PubMed Central
Google Scholar
Maier SF, Watkins LR. Stressor controllability and learned helplessness: the roles of the dorsal raphe nucleus, serotonin, and corticotropin-releasing factor. Neurosci Biobehav Rev. 2005;29(4):829–41.
Article
CAS
PubMed
Google Scholar
Huys QJ, Dayan P. A Bayesian formulation of behavioral control. Cognition. 2009;113(3):314–28.
Article
PubMed
Google Scholar
Frank MJ, Fossella JA. Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology. 2011;36(1):133–52.
Article
PubMed
Google Scholar
Blackburn JR, Pfaus JG, Phillips AG. Dopamine functions in appetitive and defensive behaviours. Prog Neurobiol. 1992;39:247–79.
Article
CAS
PubMed
Google Scholar
Brooks AM, Berns GS. Aversive stimuli and loss in the mesocorticolimbic dopamine system. Trends Cogn Sci. 2013;17(6):281–6.
Article
PubMed
Google Scholar
Holly EN, Miczek KA. Ventral tegmental area dopamine revisited: effects of acute and repeated stress. Psychopharmacology. 2016;233(2):163–86.
Article
CAS
PubMed
Google Scholar
Lammel S, Lim BK, Malenka RC. Reward and aversion in a heterogeneous midbrain dopamine system. Neuropharmacology. 2014;76:351–9.
Article
CAS
PubMed
Google Scholar
McCutcheon JE, Ebner SR, Loriaux AL, Roitman MF. Encoding of aversion by dopamine and the nucleus accumbens. Front Neurosci. 2012;6:137.
Article
PubMed
PubMed Central
Google Scholar
Pignatelli M, Bonci A. Role of dopamine neurons in reward and aversion: a synaptic plasticity perspective. Neuron. 2015;86(5):1145–57.
Article
CAS
PubMed
Google Scholar
Schultz W. Dopamine reward prediction-error signalling: a two-component response. Nat Rev Neurosci. 2016;17:183–95.
Article
CAS
PubMed
Google Scholar
Sutton RS, Barto AG. Reinforcement learning: an introduction. Cambridge: MIT Press; 1998.
Google Scholar
Kaelbling LP, Littman ML, Moore AW. Reinforcement learning: a survey. J Artif Intell Res. 1996;4:237.
Google Scholar
Doya K. What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Netw. 1999;12(7):961–74.
Article
PubMed
Google Scholar
Daw ND, Niv Y, Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci. 2005;8(12):1704–11.
Article
CAS
PubMed
Google Scholar
Dickinson A, Balleine BW. The role of learning in motivation. In: Gallistel CR, editor. Steven’s handbook of experimental psychology. New York: Wiley; 2002. p. 497–533.
Google Scholar
Dolan RJ, Dayan P. Goals and habits in the brain. Neuron. 2013;80(2):312–25.
Article
CAS
PubMed
PubMed Central
Google Scholar
Bellman RE. Dynamic programming. Princeton: Princeton University Press; 1957.
Google Scholar
Sutton RS. Learning to predict by the methods of temporal differences. Mach Learn. 1988;3(1):9–44.
Google Scholar
Barto AG, Sutton RS, Anderson CW. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans Syst Man Cybern. 1983;13:835–46.
Google Scholar
Watkins CJCH. Learning from delayed rewards. Ph.D. Thesis, University of Cambridge; 1989.
Dayan P. Exploration from generalization mediated by multiple controllers. In: Baldassare G, Mirolli M, editors. Intrinsically motivated learning in natural and artificial systems. Berlin: Springer; 2013. p. 73–91.
Chapter
Google Scholar
Howard RA. Information value theory. IEEE Trans Syst Sci Cybern. 1966;2:22–6.
Article
Google Scholar
Gittins JC. Bandit processes and dynamic allocation indices. J R Stat Soc. 1979;41(2):148–77.
Google Scholar
Sutton RS. Integrated architecture for learning, planning, and reacting based on approximating dynamic programming. In: Porter BW, Mooney RJ, editors. Proceedings of the seventh international conference on machine learning. Morgan Kaufman Publishers, Inc. 1990. p. 216–24.
Dayan P, Sejnowski TJ. Exploration bonuses and dual control. Mach Learn. 1996;25(1):5–22.
Google Scholar
Dayan P, Berridge KC. Model-based and model-free pavlovian reward learning: revaluation, revision, and revelation. Cogn Affect Behav Neurosci. 2014;14:473–93.
Article
PubMed
PubMed Central
Google Scholar
Craig W. Appetites and aversions as constituents of instincts. Biol Bull. 1918;34(2):91–107.
Article
Google Scholar
Sherrington C. The integrative action of the nervous system. New Haven: Yale University Press; 1906.
Google Scholar
Konorski J. Integrative activity of the brain. Chicago: University of Chicago Press; 1967.
Google Scholar
Baldo BA, Kelley AE. Discrete neurochemical coding of distinguishable motivational processes: insights from nucleus accumbens control of feeding. Psychopharmacology. 2007;191(3):439–59.
Article
CAS
PubMed
Google Scholar
Cools R. Role of dopamine in the motivational and cognitive control of behavior. Neuroscientist. 2008;14(4):381–95.
Article
CAS
PubMed
Google Scholar
Blackburn JR. The role of dopamine in preparatory and consummatory defensive behaviours. Ph.D. Thesis, University of British Columbia; 1989.
Nicola SM. The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J Neurosci. 2010;30(49):16585–600.
Article
CAS
PubMed
PubMed Central
Google Scholar
Ikemoto S, Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Rev. 1999;31:6–41.
Article
CAS
PubMed
Google Scholar
Berridge KC, Robinson TE. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev. 1998;28(3):309–69.
Article
CAS
PubMed
Google Scholar
Robbins TW, Everitt BJ. Functions of dopamine in the dorsal and ventral striatum. Semin Neurosci. 1992;4:119–27.
Article
Google Scholar
Williams DR, Williams H. Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement. J Exp Anal Behav. 1969;12:511–20.
Article
CAS
PubMed
PubMed Central
Google Scholar
Breland K, Breland M. The misbehavior of organisms. Am Psychol. 1961;16:681–4.
Article
Google Scholar
Dayan P, Niv Y, Seymour B, Daw ND. The misbehavior of value and the discipline of the will. Neural Netw. 2006;19(8):1153–60.
Article
PubMed
Google Scholar
Colwill RM, Rescorla RA. Associations between the discriminative stimulus and the reinforcer in instrumental learning. J Exp Psychol Anim Behav Process. 1988;14(2):155–64.
Article
Google Scholar
Estes WK. Discriminative conditioning. I: a discriminative property of conditioned anticipation. J Exp Psychol. 1943;32:150–5.
Article
Google Scholar
Holland PC. Relations between pavlovian-instrumental transfer and reinforcer devaluation. J Exp Psychol Anim Behav Process. 2004;30(2):104–17.
Article
PubMed
Google Scholar
Lovibond PF. Facilitation of instrumental behavior by a pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process. 1983;9:225–47.
Article
CAS
PubMed
Google Scholar
Rescorla RA, Wagner AR. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning II: current research and theory. New York: Appleton-Century-Crofts Ltd; 1972. p. 64–99.
Google Scholar
Sutton R, Barto AG. Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev. 1981;88(2):135–70.
Article
CAS
PubMed
Google Scholar
Sutton RS, Barto AG. Time-derivative models of pavlovian reinforcement. In: Gabriel M, Moore J, editors. Learning and computational neuroscience: foundations of adaptive networks. Cambridge: MIT Press; 1990. p. 497–537.
Google Scholar
Dayan P, Kakade S, Montague PR. Learning and selective attention. Nat Neurosci. 2000;3:1218–23.
Article
CAS
PubMed
Google Scholar
Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci. 1996;16(5):1936–47.
CAS
PubMed
Google Scholar
Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–9.
Article
CAS
PubMed
Google Scholar
O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–4.
Article
PubMed
CAS
Google Scholar
Calabresi P, Picconi B, Tozzi A, Di Filippo M. Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends Neurosci. 2007;30(5):211–9.
Article
CAS
PubMed
Google Scholar
Chen BT, Hopf FW, Bonci A. Synaptic plasticity in the mesolimbic system. Ann N Y Acad Sci. 2010;1187(1):129–39.
Article
CAS
PubMed
PubMed Central
Google Scholar
Reynolds JNJ, Hyland BI, Wickens JR. A cellular mechanism of reward-related learning. Nature. 2001;413(6851):67–70.
Article
CAS
PubMed
Google Scholar
Reynolds JNJ, Wickens JR. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 2002;15:507–21.
Article
PubMed
Google Scholar
Shen W, Flajolet M, Greengard P, Surmeier DJ. Dichotomous dopaminergic control of striatal synaptic plasticity. Science. 2008;321:848–51.
Article
CAS
PubMed
PubMed Central
Google Scholar
Mogenson GJ, Jones DL, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol. 1980;14:69–97.
Article
CAS
PubMed
Google Scholar
Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002;26(3):321–52.
Article
PubMed
Google Scholar
Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J Neurosci. 2001;21(23):9471–7.
PubMed
Google Scholar
Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PEM, Akil H. A selective role for dopamine in stimulus-reward learning. Nature. 2011;469(7328):53–7.
Article
CAS
PubMed
Google Scholar
Parkinson JA, Dalley J, Cardinal R, Bamford A, Fehnert B, Lachenal G, Rudarakanchana N, Halkerston K, Robbins T, Everitt B. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav Brain Res. 2002;137(1):149–63.
Article
CAS
PubMed
Google Scholar
Saunders BT, Robinson TE. The role of dopamine in the accumbens core in the expression of pavlovian-conditioned responses. Eur J Neurosci. 2012;36(4):2521–32.
Article
PubMed
PubMed Central
Google Scholar
Berridge KC. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology. 2007;191:391–431.
Article
CAS
PubMed
Google Scholar
Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5(6):483–94.
Article
CAS
PubMed
Google Scholar
McClure SM, Daw ND, Montague PR. A computational substrate for incentive salience. Trends Neurosci. 2003;26(8):423–8.
Article
CAS
PubMed
Google Scholar
Dickinson A, Smith J, Mirenowicz J. Dissociation of pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci. 2000;114(3):468–83.
Article
CAS
PubMed
Google Scholar
Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating pavlovian influences on instrumental behaviour. Eur J Neurosci. 2001;13(10):1984–92.
Article
CAS
PubMed
Google Scholar
Lex A, Hauber W. Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. Learn Mem. 2008;15:483–91.
Article
PubMed
PubMed Central
Google Scholar
Stuber GD, Hnasko TS, Britt JP, Edwards RH, Bonci A. Dopaminergic terminals in the nucleus accumbens but not the dorsal striatum corelease glutamate. J Neurosci. 2010;30(24):8229–33.
Article
CAS
PubMed
PubMed Central
Google Scholar
Tecuapetla F, Patel JC, Xenias H, English D, Tadros I, Shah F, Berlin J, Deisseroth K, Rice ME, Tepper JM, et al. Glutamatergic signaling by mesolimbic dopamine neurons in the nucleus accumbens. J Neurosci. 2010;30(20):7105–10.
Article
CAS
PubMed
PubMed Central
Google Scholar
Zhang S, Qi J, Li X, Wang HL, Britt JP, Hoffman AF, Bonci A, Lupica CR, Morales M. Dopaminergic and glutamatergic microdomains in a subset of rodent mesoaccumbens axons. Nat Neurosci. 2015;18(3):386–92.
Article
CAS
PubMed
PubMed Central
Google Scholar
Moss J, Ungless MA, Bolam JP. Dopaminergic axons in different divisions of the adult rat striatal complex do not express vesicular glutamate transporters. Eur J Neurosci. 2011;33(7):1205–11.
Article
PubMed
Google Scholar
Gläscher J, Hampton AN, O’Doherty JP. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2009;19(2):483–95.
Article
PubMed
Google Scholar
Gottfried JA, O’Doherty J, Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 2003;301(5636):1104–7.
Article
CAS
PubMed
Google Scholar
Hatfield T, Han JS, Conley M, Gallagher M, Holland PC. Neurotoxic lesions of basolateral, but not central, amygdala interfere with pavlovian second-order conditioning and reinforcer devaluation effects. J Neurosci. 1996;16(16):5256–65.
CAS
PubMed
Google Scholar
Holland PC, Gallagher M. Amygdala circuitry in attentional and representational processes. Trends Cogn Sci. 1999;3(2):65–73.
Article
PubMed
Google Scholar
Schoenbaum G, Chiba AA, Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci. 1998;1(2):155–9.
Article
CAS
PubMed
Google Scholar
Schoenbaum G, Chiba AA, Gallagher M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci. 1999;19(5):1876–84.
CAS
PubMed
Google Scholar
Valentin VV, Dickinson A, O’Doherty JP. Determining the neural substrates of goal-directed learning in the human brain. J Neurosci. 2007;27(15):4019–26.
Article
CAS
PubMed
Google Scholar
Dickinson A, Balleine B. Actions and responses: the dual psychology of behaviour. In: Eilan N, McCarthy RA, Brewer B, editors. Spatial representation: problems in philosophy and psychology. Oxford: Blackwell; 1993. p. 277–93.
Google Scholar
Zahm DS, Brog JS. On the significance of subterritories in the "accumbens" part of the rat ventral striatum. Neuroscience. 1992;50(4):751–67.
Article
CAS
PubMed
Google Scholar
Humphries MD, Prescott TJ. The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog Neurobiol. 2010;90(4):385–417.
Article
PubMed
Google Scholar
Kelley AE. Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev. 2004;27:765–76.
Article
PubMed
Google Scholar
Voorn P, Vanderschuren LJMJ, Groenewegen HJ, Robbins TW, Pennartz CMA. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 2004;27(8):468–74.
Article
CAS
PubMed
Google Scholar
Mogenson G, Swanson L, Wu M. Neural projections from nucleus accumbens to globus pallidus, substantia innominata, and lateral preoptic-lateral hypothalamic area: an anatomical and electrophysiological investigation in the rat. J Neurosci. 1983;3(1):189–202.
CAS
PubMed
Google Scholar
Faure A, Reynolds SM, Richard JM, Berridge KC. Mesolimbic dopamine in desire and dread: enabling motivation to be generated by localized glutamate disruptions in nucleus accumbens. J Neurosci. 2008;28(28):7184–92.
Article
CAS
PubMed
PubMed Central
Google Scholar
Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine. J Neurosci. 1999;19(6):2401–11.
CAS
PubMed
Google Scholar
Parkinson JA, Willoughby PJ, Robbins TW, Everitt BJ. Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. Behav Neurosci. 2000;114(1):42–63.
Article
CAS
PubMed
Google Scholar
Corbit LH, Balleine BW. The general and outcome-specific forms of pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J Neurosci. 2011;31(33):11786–94.
Article
CAS
PubMed
PubMed Central
Google Scholar
Bassareo V, Di Chiara G. Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments. Neuroscience. 1999;89(3):637–41.
Article
CAS
PubMed
Google Scholar
Loriaux AL, Roitman JD, Roitman MF. Nucleus accumbens shell, but not core, tracks motivational value of salt. J Neurophysiol. 2011;106(3):1537–44.
Article
CAS
PubMed
PubMed Central
Google Scholar
Shiflett MW, Balleine BW. At the limbic-motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation. Eur J Neurosci. 2010;32(10):1735–43.
Article
PubMed
PubMed Central
Google Scholar
Saddoris MP, Cacciapaglia F, Wightman RM, Carelli RM. Differential dopamine release dynamics in the nucleus accumbens core and shell reveal complementary signals for error prediction and incentive motivation. J Neurosci. 2015;35(33):11572–82.
Article
CAS
PubMed
PubMed Central
Google Scholar
West EA, Carelli RM. Nucleus accumbens core and shell differentially encode reward-associated cues after reinforcer devaluation. J Neurosci. 2016;36(4):1128–39.
Article
CAS
PubMed
PubMed Central
Google Scholar
Valjent E, Bertran-Gonzalez J, Hervé D, Fisone G, Girault JA. Looking BAC at striatal signalling: cell-specific analysis in new transgenic mice. Trends Neurosci. 2009;32(10):538–47.
Article
CAS
PubMed
Google Scholar
Tritsch NX, Sabatini BL. Dopaminergic modulation of synaptic transmission in cortex and striatum. Neuron. 2012;76:33–50.
Article
CAS
PubMed
PubMed Central
Google Scholar
Beaulieu JM, Gainetdinov RR. The physiology, signaling, and pharmacology of dopamine receptors. Pharmacol Rev. 2011;63:182–217.
Article
CAS
PubMed
Google Scholar
Missale C, Nash SR, Robinson SW, Jaber M, Caron MG. Dopamine receptors: from structure to function. Physiol Rev. 1998;78(1):189–225.
CAS
PubMed
Google Scholar
Vallone D, Picetti R, Borrelli E. Structure and function of dopamine receptors. Neurosci Biobehav Rev. 2000;24:125–32.
Article
CAS
PubMed
Google Scholar
Richfield EK, Penney JB, Young AB. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience. 1989;30(3):767–77.
Article
CAS
PubMed
Google Scholar
Dreyer JK, Herrik KF, Berg RW, Hounsgaard JD. Influence of phasic and tonic dopamine release on receptor activation. J Neurosci. 2010;30(42):14273–83.
Article
CAS
PubMed
Google Scholar
Gerfen CR, Surmeier DJ. Modulation of striatal projection systems by dopamine. Annu Rev Neurosci. 2011;34:441–66.
Article
CAS
PubMed
PubMed Central
Google Scholar
Surmeier DJ, Ding J, Day M, Wang Z, Shen W. D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci. 2007;30(5):228–35.
Article
CAS
PubMed
Google Scholar
Albin RL, Young AB, Penney JB. The functional anatomy of basal ganglia disorders. Trends Neurosci. 1989;12(10):366–75.
Article
CAS
PubMed
Google Scholar
DeLong MR. Primate models of movement disorders of basal ganglia origin. Trends Neurosci. 1990;13:281–5.
Article
CAS
PubMed
Google Scholar
Gerfen CR, Engber TM, Mahan LC, Susel Z, Chase TN, Monsma FJ, Sibley DR. D1 and d2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science. 1990;250:1429–32.
Article
CAS
PubMed
Google Scholar
Kravitz AV, Freeze BS, Parker PRL, Kay K, Thwin MT, Deisseroth K, Kreitzer AC. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature. 2010;466:622–6.
Article
CAS
PubMed
PubMed Central
Google Scholar
Frank MJ. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. J Cogn Neurosci. 2005;17(1):51–72.
Article
PubMed
Google Scholar
Carlezone WA Jr, Thomas MJ. Biological substrates of reward and aversion: a nucleus accumbens activity hypothesis. Neuropharmacology. 2009;56:122–32.
Article
CAS
Google Scholar
Grueter BA, Robison AJ, Neve RL, Nestler EJ, Malenka RC. \(\Delta\)FosB differentially modulates nucleus accumbens direct and indirect pathway function. Proc Natl Acad Sci USA. 2013;110(5):1923–8.
Article
CAS
PubMed
PubMed Central
Google Scholar
Hikida T, Yawata S, Yamaguchi T, Danjo T, Sasaoka T, Wang Y, Nakanishi S. Pathway-specific modulation of nucleus accumbens in reward and aversive behavior via selective transmitter receptors. Proc Natl Acad Sci USA. 2013;110(1):342–7.
Article
CAS
PubMed
Google Scholar
Kupchik YM, Brown RM, Heinsbroek JA, Lobo MK, Schwartz DJ, Kalivas PW. Coding the direct/indirect pathways by D1 and D2 receptors is not valid for accumbens projections. Nat Neurosci. 2015;18:1230–2.
Article
CAS
PubMed
PubMed Central
Google Scholar
Smith RJ, Lobo MK, Spencer S, Kalivas PW. Cocaine-induced adaptations in D1 and D2 accumbens projection neurons (a dichotomy not necessarily synonymous with direct and indirect pathways). Curr Opin Neurobiol. 2013;23:546–52.
Article
CAS
PubMed
PubMed Central
Google Scholar
Nicola SM, Surmeier DJ, Malenka RC. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu Rev Neurosci. 2000;23:185–215.
Article
CAS
PubMed
Google Scholar
Lu XY, Ghasemzadeh MB, Kalivas P. Expression of D1 receptor, D2 receptor, substance P and enkephalin messenger RNAs in the neurons projecting from the nucleus accumbens. Neuroscience. 1997;82(3):767–80.
Article
Google Scholar
Aizman O, Brismar H, Uhlén P, Zettergren E, Levey AI, Forssberg H, Greengard P, Aperia A. Anatomical and physiological evidence for D1 and D2 dopamine receptor colocalization in neostriatal neurons. Nat Neurosci. 2000;3(3):226–30.
Article
CAS
PubMed
Google Scholar
Bertran-Gonzalez J, Bosch C, Maroteaux M, Matamales M, Hervé D, Valjent E, Girault JA. Opposing patterns of signaling activation in dopamine D1 and D2 receptor-expressing striatal neurons in response to cocaine and haloperidol. J Neurosci. 2008;28(22):5671–85.
Article
CAS
PubMed
Google Scholar
Hasbi A, Fan T, Alijaniaram M, Nguyen T, Perreault ML, O’Dowd BF, George SR. Calcium signaling cascade links dopamine D1–D2 receptor heteromer to striatal BDNF production and neuronal growth. Proc Natl Acad Sci USA. 2009;106(50):21377–82.
Article
CAS
PubMed
PubMed Central
Google Scholar
Rashid AJ, So CH, Kong MM, Furtak T, El-Ghundi M, Cheng R, O’Dowd BF, George SR. D1–D2 dopamine receptor heterooligomers with unique pharmacology are coupled to rapid activation of Gq/11 in the striatum. Proc Natl Acad Sci USA. 2007;104(2):654–9.
Article
CAS
PubMed
Google Scholar
Frederick A, Yano H, Trifilieff P, Vishwasrao H, Biezonski D, Mészáros J, Urizar E, Sibley D, Kellendonk C, Sonntag K, et al. Evidence against dopamine D1/D2 receptor heteromers. Mol Psychiatry. 2015;20:1373–85.
Article
CAS
PubMed
PubMed Central
Google Scholar
Dalley JW, Lääne K, Theobald DE, Armstrong HC, Corlett PR, Chudasama Y, Robbins TW. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc Natl Acad Sci USA. 2005;102(17):6189–94.
Article
CAS
PubMed
PubMed Central
Google Scholar
Eyny YS, Horvitz JC. Opposing roles of D1 and D2 receptors in appetitive conditioning. J Neurosci. 2003;23(5):1584–7.
CAS
PubMed
Google Scholar
Beninger RJ, Miller R. Dopamine D1-like receptors and reward-related incentive learning. Neurosci Biobehav Rev. 1998;22(2):335–45.
Article
CAS
PubMed
Google Scholar
Parker JG, Zweifel LS, Clark JJ, Evans SB, Phillips PE, Palmiter RD. Absence of NMDA receptors in dopamine neurons attenuates dopamine release but not conditioned approach during Pavlovian conditioning. Proc Natl Acad Sci USA. 2010;107(30):13491–6.
Article
CAS
PubMed
PubMed Central
Google Scholar
Smith-Roe SL, Kelley AE. Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. J Neurosci. 2000;20(20):7737–42.
CAS
PubMed
Google Scholar
Bernal SY, Dostova I, Kest A, Abayev Y, Kandova E, Touzani K, Sclafani A, Bodnar RJ. Role of dopamine D1 and D2 receptors in the nucleus accumbens shell on the acquisition and expression of fructose-conditioned flavor-flavor preferences in rats. Behav Brain Res. 2008;190(1):59–66.
Article
CAS
PubMed
PubMed Central
Google Scholar
Fraser KM, Haight JL, Gardner EL, Flagel SB. Examining the role of dopamine D2 and D3 receptors in Pavlovian conditioned approach behaviors. Behav Brain Res. 2016;305:87–99.
Article
CAS
PubMed
Google Scholar
Lopez JC, Karlsson RM, O’Donnell P. Dopamine D2 modulation of sign and goal tracking in rats. Neuropsychopharmacology. 2015;40:2096–102.
Article
PubMed
PubMed Central
Google Scholar
Ranaldi R, Beninger RJ. Dopamine D1 and D2 antagonists attenuate amphetamine-produced enhancement of responding for conditioned reward in rats. Psychopharmacology. 1993;113(1):110–8.
Article
CAS
PubMed
Google Scholar
Wolterink G, Phillips G, Cador M, Donselaar-Wolterink I, Robbins T, Everitt B. Relative roles of ventral striatal D1 and D2 dopamine receptors in responding with conditioned reinforcement. Psychopharmacology. 1993;110(3):355–64.
Article
CAS
PubMed
Google Scholar
Sombers LA, Beyene M, Carelli RM, Wightman RM. Synaptic overflow of dopamine in the nucleus accumbens arises from neuronal activity in the ventral tegmental area. J Neurosci. 2009;29(6):1735–42.
Article
CAS
PubMed
PubMed Central
Google Scholar
Cachope R, Cheer JF. Local control of striatal dopamine release. Front Behav Neurosci. 2014;8:1–7.
Article
CAS
Google Scholar
Rice ME, Patel JC, Cragg SJ. Dopamine release in the basal ganglia. Neuroscience. 2011;198:112–37.
Article
CAS
PubMed
PubMed Central
Google Scholar
Threlfell S, Lalic T, Platt NJ, Jennings KA, Deisseroth K, Cragg SJ. Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron. 2012;75:58–64.
Article
CAS
PubMed
Google Scholar
Grace AA. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience. 1991;41(1):1–24.
Article
CAS
PubMed
Google Scholar
Floresco SB, West AR, Ash B, Moore H, Grace AA. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat Neurosci. 2003;6(9):968–73.
Article
CAS
PubMed
Google Scholar
Grace AA, Floresco SB, Goto Y, Lodge DJ. Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 2007;30(5):220–7.
Article
CAS
PubMed
Google Scholar
Tritsch NX, Ding JB, Sabatini BL. Dopaminergic neurons inhibit striatal output through non-canonical release of GABA. Nature. 2012;490(7419):262–6.
Article
CAS
PubMed
PubMed Central
Google Scholar
Tritsch NX, Granger AJ, Sabatini BL. Mechanisms and functions of GABA co-release. Nat Rev Neurosci. 2016;17:139–45.
Article
CAS
PubMed
Google Scholar
Suri RE, Schultz W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience. 1999;91(3):871–90.
Article
CAS
PubMed
Google Scholar
Balleine BW, Delgado MR, Hikosaka O. The role of the dorsal striatum in reward and decision-making. J Neurosci. 2007;27(31):8161–5.
Article
CAS
PubMed
Google Scholar
Packard MG, Knowlton BJ. Learning and memory functions of the basal ganglia. Annu Rev Neurosci. 2002;25(1):563–93.
Article
CAS
PubMed
Google Scholar
Yin HH, Knowlton BJ, Balleine BW. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci. 2004;19(1):181–9.
Article
PubMed
Google Scholar
Yin HH, Knowlton BJ, Balleine BW. Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behav Brain Res. 2006;166(2):189–96.
Article
PubMed
Google Scholar
Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev Neurosci. 2006;7(6):464–76.
Article
CAS
PubMed
Google Scholar
Tricomi E, Balleine BW, O’Doherty JP. A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci. 2009;29(11):2225–32.
Article
PubMed
PubMed Central
Google Scholar
Balleine BW, O’Doherty JP. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology. 2010;35:48–69.
Article
PubMed
Google Scholar
Doll BB, Simon DA, Daw ND. The ubiquity of model-based reinforcement learning. Curr Opin Neurobiol. 2012;22(6):1075–81.
Article
CAS
PubMed
PubMed Central
Google Scholar
Lee SW, Shimojo S, O’Doherty JP. Neural computations underlying arbitration between model-based and model-free learning. Neuron. 2014;81(3):687–99.
Article
CAS
PubMed
PubMed Central
Google Scholar
Killcross S, Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex. 2003;13(4):400–8.
Article
PubMed
Google Scholar
Cohen MX, Frank MJ. Neurocomputational models of basal ganglia function in learning, memory and choice. Behav Brain Res. 2009;199:141–56.
Article
PubMed
Google Scholar
Collins AGE, Frank MJ. Opponent actor learning (opal): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol Rev. 2014;121(3):337–66.
Article
PubMed
Google Scholar
Frank MJ, Loughry B, O’Reilly RC. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn Affect Behav Neurosci. 2001;1:137–60.
Article
CAS
PubMed
Google Scholar
Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci. 2012;15(6):816–9.
Article
CAS
PubMed
PubMed Central
Google Scholar
Cui G, Jun SB, Jin X, Pham MD, Vogel SS, Lovinger DM, Costa RM. Concurrent activation of striatal direct and indirect pathways during action initiation. Nature. 2013;494:238–42.
Article
CAS
PubMed
PubMed Central
Google Scholar
Mink JW. The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol. 1996;50(4):381–425.
Article
CAS
PubMed
Google Scholar
Nelson AB, Kreitzer AC. Reassessing models of basal ganglia function and dysfunction. Annu Rev Neurosci. 2014;37:117–35.
Article
CAS
PubMed
PubMed Central
Google Scholar
Calabresi P, Picconi B, Tozzi A, Ghiglieri V, Di Filippo M. Direct and indirect pathways of basal ganglia: a critical reappraisal. Nat Neurosci. 2014;17(8):1022–9.
Article
CAS
PubMed
Google Scholar
Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based influences on humans’ choices and striatal prediction errors. Neuron. 2011;69:1204–15.
Article
CAS
PubMed
PubMed Central
Google Scholar
Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O. A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol. 2010;104:1068–76.
Article
PubMed
PubMed Central
Google Scholar
Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O. Dopamine neurons can represent context-dependent prediction error. Neuron. 2004;41(2):269–80.
Article
CAS
PubMed
Google Scholar
Schultz W. Updating dopamine reward signals. Curr Opin Neurobiol. 2013;23:229–38.
Article
CAS
PubMed
PubMed Central
Google Scholar
Horvitz JC. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience. 2000;96:651–6.
Article
CAS
PubMed
Google Scholar
Kakade S, Dayan P. Dopamine: generalization and bonuses. Neural Netw. 2002;15:549–59.
Article
PubMed
Google Scholar
Balleine BW. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav. 2005;86(5):717–30.
Article
CAS
PubMed
Google Scholar
Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci. 2005;22(2):513–23.
Article
PubMed
Google Scholar
Cagniard B, Beeler JA, Britt JP, McGehee DS, Marinelli M, Zhuang X. Dopamine scales performance in the absence of new learning. Neuron. 2006;51(5):541–7.
Article
CAS
PubMed
Google Scholar
Niv Y, Joel D, Dayan P. A normative perspective on motivation. Trends Cogn Sci. 2006;10(8):375–81.
Article
PubMed
Google Scholar
Masterson FA, Crawford M. The defense motivation system: a theory of avoidance behavior. Behav Brain Sci. 1982;5(04):661–75.
Article
Google Scholar
Gray JA. The psychology of fear and stress. Cambridge: Cambridge University Press; 1987.
Google Scholar
Mowrer OH. On the dual nature of learning: a reinterpretation of "conditioning" and "problem-solving". Harv Educ Rev. 1947;17:102–50.
Google Scholar
Mowrer OH. Two-factor learning theory reconsidered, with special reference to secondary reinforcement and the concept of habit. Psychol Rev. 1956;63(2):114–28.
Article
CAS
PubMed
Google Scholar
Canteras NS, Graeff FG. Executive and modulatory neural circuits of defensive reactions: implications for panic disorder. Neurosci Biobehav Rev. 2014;46:352–64.
Article
PubMed
Google Scholar
Gross CT, Canteras NS. The many paths to fear. Nat Rev Neurosci. 2012;13(9):651–8.
Article
CAS
PubMed
Google Scholar
Bandler R, Shipley MT. Columnar organization in the midbrain periaqueductal gray: modules for emotional expression? Trends Neurosci. 1994;17(9):379–89.
Article
CAS
PubMed
Google Scholar
Bolles RC, Fanselow MS. A perceptual-defensive-recuperative model of fear and pain. Behav Brain Sci. 1980;3:291–323.
Article
Google Scholar
Fanselow MS. Neural organization of the defensive behavior system responsible for fear. Psychon Bull Rev. 1994;1(4):429–38.
Article
CAS
PubMed
Google Scholar
Fanselow MS, Lester LS. A functional behavioristic approach to aversive motivated behavior: Predatory imminence as a determinant of the topography of defensive behavior. In: Bolles RC, Beecher MD, editors. Evolution and learning. Hillsdale: Erlbaum; 1988. p. 185–211.
Google Scholar
Gray JA. The neuropsychology of anxiety: an enquiry into the functions of the septo-hippocampal system. Oxford: Oxford University Press; 1982.
Google Scholar
Cabib S, Puglisi-Allegra S. Opposite responses of mesolimbic dopamine system to controllable and uncontrollable aversive experiences. J Neurosci. 1994;14(5):3333–40.
CAS
PubMed
Google Scholar
Cabib S, Puglisi-Allegra S. Stress, depression and the mesolimbic dopamine system. Psychopharmacology. 1996;128:331–42.
Article
CAS
PubMed
Google Scholar
Cabib S, Puglisi-Allegra S. The mesoaccumbens dopamine in coping with stress. Neurosci Biobehav Rev. 2012;36(1):79–89.
Article
CAS
PubMed
Google Scholar
Redgrave P, Prescott TJ, Gurney K. The basal ganglia: a vertebrate solution to the selection problem? Neuroscience. 1999;89(4):1009–23.
Article
CAS
PubMed
Google Scholar
Blanchard DC, Blanchard RJ. Ethoexperimental approaches to the biology of emotion. Annu Rev Psychol. 1988;39:43–68.
Article
CAS
PubMed
Google Scholar
Swanson LW. Cerebral hemisphere regulation of motivated behavior. Brain Res. 2000;886(1):113–64.
Article
CAS
PubMed
Google Scholar
Joseph MH, Datla K, Young AMJ. The interpretation of the measurement of nucleus accumbens dopamine by in vivo analysis: the kick, the craving or the cognition? Neurosci Biobehav Rev. 2003;27:527–41.
Article
CAS
PubMed
Google Scholar
Young AMJ. Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J Neurosci Methods. 2004;138(1):57–63.
Article
CAS
PubMed
Google Scholar
Abercrombie ED, Keefe KA, DiFrischia DS, Zigmond MJ. Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. J Neurochem. 1989;52(5):1655–8.
Article
CAS
PubMed
Google Scholar
Inglis FM, Moghaddam B. Dopaminergic innervation of the amygdala is highly responsive to stress. J Neurochem. 1999;72(3):1088–94.
Article
CAS
PubMed
Google Scholar
Young AMJ, Rees KR. Dopamine release in the amygdaloid complex of the rat, studied by brain microdialysis. Neurosci Lett. 1998;249(1):49–52.
Article
CAS
PubMed
Google Scholar
Budygin EA, Park J, Bass CE, Grinevich VP, Bonin KD, Wightman RM. Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience. 2012;201:331–7.
Article
CAS
PubMed
Google Scholar
Park J, Bucher ES, Budygin EA, Wightman RM. Norepinephrine and dopamine transmission in 2 limbic regions differentially respond to acute noxious stimulation. Pain. 2015;156(2):318–27.
Article
CAS
PubMed
PubMed Central
Google Scholar
Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012;482:85–90.
Article
CAS
PubMed
PubMed Central
Google Scholar
Brischoux F, Chakraborty S, Brierley DI, Ungless MA. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA. 2009;106(12):4894–9.
Article
CAS
PubMed
PubMed Central
Google Scholar
Lammel S, Hetzel A, Häckel O, Jones I, Liss B, Roeper J. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron. 2008;57:760–73.
Article
CAS
PubMed
Google Scholar
Lammel S, Ion DI, Roeper J, Malenka RC. Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron. 2011;70(5):855–62.
Article
CAS
PubMed
PubMed Central
Google Scholar
Mantz J, Thierry A, Glowinski J. Effect of noxious tail pinch on the discharge rate of mesocortical and mesolimbic dopamine neurons: selective activation of the mesocortical system. Brain Res. 1989;476(2):377–81.
Article
CAS
PubMed
Google Scholar
Lerner TN, Shilyansky C, Davidson TJ, Evans KE, Beier KT, Zalocusky KA, Crow AK, Malenka RC, Luo L, Tomer R, et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell. 2015;162(3):635–47.
Article
CAS
PubMed
PubMed Central
Google Scholar
Maldonado-Irizarry CS, Swanson CJ, Kelley AE. Glutamate receptors in the nucleus accumbens shell control feeding behavior via the lateral hypothalamus. J Neurosci. 1995;15(10):6779–88.
CAS
PubMed
Google Scholar
Reynolds SM, Berridge KC. Fear and feeding in the nucleus accumbens shell: rostrocaudal segregation of gaba-elicited defensive behavior versus eating behavior. J Neurosci. 2001;21(9):3261–70.
CAS
PubMed
Google Scholar
Richard JM, Berridge KC. Nucleus accumbens dopamine/glutamate interaction switches modes to generate desire versus dread: D1 alone for appetitive eating but D1 and D2 together for fear. J Neurosci. 2011;31(36):12866–79.
Article
CAS
PubMed
PubMed Central
Google Scholar
Reynolds SM, Berridge KC. Emotional environments retune the valence of appetitive versus fearful functions in nucleus accumbens. Nat Neurosci. 2008;11:423–5.
Article
CAS
PubMed
PubMed Central
Google Scholar
Sweidan S, Edinger H, Siegel A. The role of D1 and D2 receptors in dopamine agonist-induced modulation of affective defense behavior in the cat. Pharmacol Biochem Behav. 1990;36(3):491–9.
Article
CAS
PubMed
Google Scholar
Sweidan S, Edinger H, Siegel A. D2 dopamine receptor-mediated mechanisms in the medial preoptic-anterior hypothalamus regulate affective defense behavior in the cat. Brain Res. 1991;549(1):127–37.
Article
CAS
PubMed
Google Scholar
Willner P. Animal models of depression: an overview. Pharmacol Ther. 1990;45(3):425–55.
Article
CAS
PubMed
Google Scholar
Steru L, Chermat R, Thierry B, Simon P. The tail suspension test: a new method for screening antidepressants in mice. Psychopharmacology. 1985;85(3):367–70.
Article
CAS
PubMed
Google Scholar
Porsolt RD, Le Pichon M, Jalfre M. Depression: a new animal model sensitive to antidepressant treatments. Nature. 1977;266(5604):730–2.
Article
CAS
PubMed
Google Scholar
Maier SF, Seligman ME. Learned helplessness: theory and evidence. J Exp Psychol Gen. 1976;105(1):3–46.
Article
Google Scholar
Puglisi-Allegra S, Imperato A, Angelucci L, Cabib S. Acute stress induces time-dependent responses in dopamine mesolimbic system. Brain Res. 1991;554:217–22.
Article
CAS
PubMed
Google Scholar
Imperato A, Angelucci L, Casolini P, Zocchi A, Puglisi-Allegra S. Repeated stressful experiences differently affect limbic dopamine release during and following stress. Brain Res. 1992;577:194–9.
Article
CAS
PubMed
Google Scholar
Imperato A, Cabib S, Puglisi-Allegra S. Repeated stressful experiences differently affect the time-dependent responses of the mesolimbic dopamine system to the stressor. Brain Res. 1993;601:333–6.
Article
CAS
PubMed
Google Scholar
Pascucci T, Ventura R, Latagliata EC, Cabib S, Puglisi-Allegra S. The medial prefrontal cortex determines the accumbens dopamine response to stress through the opposing influences of norepinephrine and dopamine. Cereb Cortex. 2007;17(12):2796–804.
Article
PubMed
Google Scholar
Leknes S, Tracey I. A common neurobiology for pain and pleasure. Nat Rev Neurosci. 2008;9(4):314–20.
Article
CAS
PubMed
Google Scholar
Wood PB. Role of central dopamine in pain and analgesia. Expert Rev Neurother. 2008;8(5):781–97.
Article
CAS
PubMed
Google Scholar
Schwartz N, Temkin P, Jurado S, Lim BK, Heifets BD, Polepalli JS, Malenka RC. Decreased motivation during chronic pain requires long-term depression in the nucleus accumbens. Science. 2014;345(6196):535–42.
Article
CAS
PubMed
PubMed Central
Google Scholar
Ren W, Centeno MV, Berger S, Wu Y, Na X, Liu X, Kondapalli J, Apkarian AV, Martina M, Surmeier DJ. The indirect pathway of the nucleus accumbens shell amplifies neuropathic pain. Nature Neurosci. 2016;19:220–2.
Article
CAS
PubMed
Google Scholar
Farrar AM, Segovia KN, Randall PA, Nunes EJ, Collins LE, Stopper CM, Port RG, Hockemeyer J, Müller CE, Correa M, Salamone JD. Nucleus accumbens and effort-related functions: behavioral and neural markers of the interactions between adenosine A2A and dopamine D2 receptors. Neuroscience. 2010;166(4):1056–67.
Article
CAS
PubMed
Google Scholar
Santerre JL, Nunes EJ, Kovner R, Leser CE, Randall PA, Collins-Praino LE, Cruz LL, Correa M, Baqi Y, Müller CE, et al. The novel adenosine A2A antagonist prodrug MSX-4 is effective in animal models related to motivational and motor functions. Pharmacol Biochem Behav. 2012;102(4):477–87.
Article
CAS
PubMed
Google Scholar
Wadenberg MG, Hicks PB. The conditioned avoidance response test re-evaluated: is it a sensitive test for the detection of potentially atypical antipsychotics? Neurosci Biobehav Rev. 1999;23:851–62.
Article
CAS
PubMed
Google Scholar
Deakin JFW, Graeff FG. 5-HT and mechanisms of defence. J psychopharmacol. 1991;5:305–15.
Article
CAS
PubMed
Google Scholar
Graeff FG, Guimarães FS, De Andrade TG, Deakin JF. Role of 5-HT in stress, anxiety, and depression. Pharmacol Biochem Behav. 1996;54(1):129–41.
Article
CAS
PubMed
Google Scholar
Dayan P, Huys QJM. Serotonin in affective control. Annu Rev Neurosci. 2009;32:95–126.
Article
CAS
PubMed
Google Scholar
Dayan P. Instrumental vigour in punishment and reward. Eur J Neurosci. 2012;35(7):1152–68.
Article
PubMed
Google Scholar
Grossberg S. Some normal and abnormal behavioral syndromes due to transmitter gating of opponent systems. Biol Psychiatry. 1984;19:1075–118.
CAS
PubMed
Google Scholar
Solomon RL, Corbit JD. An opponent-process theory of motivation: I. temporal dynamics of affect. Psychol Rev. 1974;81(2):119–45.
Article
CAS
PubMed
Google Scholar
Daw ND, Kakade S, Dayan P. Opponent interactions between serotonin and dopamine. Neural Netw. 2002;15:603–16.
Article
PubMed
Google Scholar
Deakin JFW. Roles of serotonergic systems in escape, avoidance and other behaviours. In: Cooper SJ, editor. Theory in psychopharmacology. vol. 2. 2nd edn., New York: Academic Press; 1983. pp. 149–193.
Google Scholar
García J, Fernández F. A comprehensive survey on safe reinforcement learning. J Mach Learn Res. 2015;16:1437–80.
Google Scholar
Huys QJ, Daw ND, Dayan P. Depression: a decision-theoretic analysis. Annu Rev Neurosci. 2015;38:1–23.
Article
CAS
PubMed
Google Scholar
Gray JA, McNaughton N. The neuropsychology of anxiety: an enquiry into the function of the septo-hippocampal system, vol. 33. Oxford: Oxford University Press; 2003.
Book
Google Scholar
Blanchard RJ, Yudko EB, Rodgers RJ, Blanchard DC. Defense system psychopharmacology: an ethological approach to the pharmacology of fear and anxiety. Behav Brain Res. 1993;58(1):155–65.
Article
CAS
PubMed
Google Scholar
Lister RG. Ethologically-based animal models of anxiety disorders. Pharmacol Ther. 1990;46(3):321–40.
Article
CAS
PubMed
Google Scholar
Bach DR. Anxiety-like behavioural inhibition is normative under environmental threat-reward correlations. PLoS Comput Biol. 2015;11(12):1004646.
Article
CAS
Google Scholar
Fanselow MS. The postshock activity burst. Anim Learn Behav. 1982;10(4):448–54.
Article
Google Scholar
Jenkins H, Moore BR. The form of the auto-shaped response with food or water reinforcers. J Exp Anal Behav. 1973;20(2):163–81.
Article
CAS
PubMed
PubMed Central
Google Scholar
Abraham AD, Neve KA, Lattal KM. Dopamine and extinction: a convergence of theory with fear and reward circuitry. Neurobiol Learn Mem. 2014;108:65–77.
Article
PubMed
Google Scholar
Levita L, Dalley JW, Robbins TW. Nucleus accumbens dopamine and learned fear revisited: a review and some new findings. Behav Brain Res. 2002;137:115–27.
Article
CAS
PubMed
Google Scholar
Pezze MA, Feldon J. Mesolimbic dopaminergic pathways in fear conditioning. Prog Neurobiol. 2004;74:301–20.
Article
CAS
PubMed
Google Scholar
Frank MJ, Surmeier DJ. Do substantia nigra dopaminergic neurons differentiate between reward and punishment? J Mol Cell Biol. 2009;1:15–6.
Article
CAS
PubMed
Google Scholar
Zweifel LS, Fadok JP, Argilli E, Garelick MG, Jones GL, Dickerson TMK, Allen JM, Mizumori SJY, Bonci A, Palmiter RD. Activation of dopamine neurons is critical for aversive conditioning and prevention of generalized anxiety. Nat Neurosci. 2011;14(5):620–6.
Article
CAS
PubMed
PubMed Central
Google Scholar
Badrinarayan A, Wescott SA, Vander Weele CM, Saunders BT, Couturier BE, Maren S, Aragona BJ. Aversive stimuli differentially modulate real-time dopamine transmission dynamics within the nucleus accumbens core and shell. J Neurosci. 2012;32(45):15779–90.
Article
CAS
PubMed
PubMed Central
Google Scholar
Oleson EB, Gentry RN, Chioma VC, Cheer JF. Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J Neurosci. 2012;32(42):14804–8.
Article
CAS
PubMed
PubMed Central
Google Scholar
Fadok JP, Dickerson TMK, Palmiter RD. Dopamine is necessary for cue-dependent fear conditioning. J Neurosci. 2009;29(36):11089–97.
Article
CAS
PubMed
PubMed Central
Google Scholar
Ikegami M, Uemura T, Kishioka A, Sakimura K, Mishina M. Striatal dopamine D1 receptor is essential for contextual fear conditioning. Sci Rep. 2014;4:3976.
PubMed
PubMed Central
Google Scholar
Blackburn JR, Phillips AG. Enhancement of freezing behaviour by metaclopromide: implications for neuroleptic-induced avoidance deficits. Pharmacol Biochem Behav. 1990;35(3):685–91.
Article
CAS
PubMed
Google Scholar
de Souza Caetano KA, de Oliveira AR, Brandão ML. Dopamine D2 receptors modulate the expression of contextual conditioned fear: role of the ventral tegmental area and the basolateral amygdala. Behav Pharmacol. 2013;24(4):264–74.
Article
PubMed
CAS
Google Scholar
Davis M, Falls WA, Campeau S, Kim M. Fear-potentiated startle: a neural and pharmacological analysis. Behav Brain Res. 1993;58:175–98.
Article
CAS
PubMed
Google Scholar
de Oliveira AR, Reimer AE, Brandão ML. Dopamine D2 receptor mechanisms in the expression of conditioned fear. Pharmacol Biochem Behav. 2006;84(1):102–11.
Article
PubMed
CAS
Google Scholar
Li SSY, McNally GP. A role of nucleus accumbens dopamine receptors in the nucleus accumbens core, but not shell, in fear prediction error. Behav Neurosci. 2015;129(4):450–6.
Article
PubMed
Google Scholar
Pavlov IP. Conditioned reflexes. Oxford: Oxford University Press; 1927.
Google Scholar
Rescorla RA. Pavlovian conditioned inhibition. Psychol Bull. 1969;72(2):77–94.
Article
Google Scholar
Christianson JP, Fernando ABP, Kazama AM, Jovanovic T, Ostroff LE, Sangha S. Inhibition of fear by learned safety signals: a mini-symposium review. J Neurosci. 2012;32(41):14118–24.
Article
CAS
PubMed
PubMed Central
Google Scholar
Kong E, Monje FJ, Hirsch J, Pollak DD. Learning not to fear: neural correlates of learned safety. Neuropsychopharmacology. 2014;39:515–27.
Article
PubMed
Google Scholar
Fernando ABP, Urcelay GP, Mar AC, Dickinson A, Robbins TW. Safety signals as instrumental reinforcers during free-operant avoidance. Learn Mem. 2014;21:488–97.
Article
PubMed
PubMed Central
Google Scholar
Dickinson A, Pearce J. Inhibitory interactions between appetitive and aversive stimuli. Psychol Bull. 1977;84:690–711.
Article
Google Scholar
Dickinson A, Dearing MF. Appetitive-aversive interactions and inhibitory processes. In: Dickinson A, Boakes RA, editors. Mechanisms of learning and motivation. Hillsdale: Erlbaum; 1979. p. 203–31.
Google Scholar
Rogan MT, Leon KS, Perez DL, Kandel ER. Distinct neural signatures for safety and danger in the amygdala and striatum of the mouse. Neuron. 2005;46:309–20.
Article
CAS
PubMed
Google Scholar
Genud-Gabai R, Klavir O, Paz R. Safety signals in the primate amygdala. J Neurosci. 2013;33(46):17986–94.
Article
CAS
PubMed
PubMed Central
Google Scholar
Sangha S, Chadick JZ, Janak PH. Safety encoding in the basal amygdala. J Neurosci. 2013;33(9):3744–51.
Article
CAS
PubMed
Google Scholar
Pollak DD, Rogan MT, Egner T, Perez DL, Yanagihara TK, Hirsch J. A translational bridge between mouse and human models of learned safety. Ann Med. 2010;42(2):127–34.
Article
Google Scholar
Fernando ABP, Urcelay GP, Mar AC, Dickenson TA, Robbins TW. The role of nucleus accumbens shell in the mediation of the reinforcing properties of a safety signal in free-operant avoidance: dopamine-dependent inhibitory effects of d-amphetamine. Neuropsychopharmacology. 2014;39:1420–30.
Article
CAS
PubMed
PubMed Central
Google Scholar
Bouton ME. Learning and behavior: a contemporary synthesis. Sunderland: Sinauer Associates Inc; 2007.
Google Scholar
Beninger RJ. The role of dopamine in locomotor activity and learning. Brain Res Rev. 1983;6:173–96.
Article
CAS
Google Scholar
Dinsmoor JA. Punishment: I. the avoidance hypothesis. Psychol Rev. 1954;61:34–46.
Article
CAS
PubMed
Google Scholar
Dinsmoor JA. Stimuli inevitably generated by behavior that avoids electric shock are inherently reinforcing. J Exp Anal Behav. 2001;75:311–33.
Article
CAS
PubMed
PubMed Central
Google Scholar
Konorski J. Conditioned reflexes and neuron organization. Cambridge: Cambridge University Press; 1948.
Google Scholar
Miller NE. Studies of fear as an acquirable drive: I. fear as motivation and fear-reduction as reinforcement in the learning of new responses. J Exp Psychol. 1948;38:89–101.
Article
CAS
PubMed
Google Scholar
Tolman EC. Purposive behavior in animals and men. New York: Century; 1932.
Google Scholar
Blanchard RJ, Fukunaga KK, Blanchard DC. Environmental control of defensive reactions to footshock. Bull Psychon Soc. 1976;8(2):129–30.
Article
Google Scholar
Seymour B, Singer T, Dolan R. The neurobiology of punishment. Nat Rev Neurosci. 2007;8(4):300–11.
Article
CAS
PubMed
Google Scholar
Morse WH, Mead RN, Kelleher RT. Modulation of elicited behavior by a fixed-interval schedule of electric shock presentation. Science. 1967;157(3785):215–7.
Article
CAS
PubMed
Google Scholar
Overmier JB, Seligman ME. Effects of inescapable shock upon subsequent escape and avoidance responding. J Comp Physiol Psychol. 1967;63(1):28–33.
Article
CAS
PubMed
Google Scholar
Seligman ME, Maier SF. Failure to escape traumatic shock. J Exp Psychol. 1967;74:1–9.
Article
CAS
PubMed
Google Scholar
Fernando A, Urcelay G, Mar A, Dickinson A, Robbins T. Free-operant avoidance behavior by rats after reinforcer revaluation using opioid agonists and d-amphetamine. J Neurosci. 2014;34(18):6286–93.
Article
CAS
PubMed
PubMed Central
Google Scholar
Hendersen RW, Graham J. Avoidance of heat by rats: effects of thermal context on rapidity of extinction. Learn Motiv. 1979;10(3):351–63.
Article
Google Scholar
Declercq M, De Houwer J. On the role of us expectancies in avoidance behavior. Psychon Bull Rev. 2008;15(1):99–102.
Article
PubMed
Google Scholar
Gillan CM, Morein-Zamir S, Urcelay GP, Sule A, Voon V, Apergis-Schoute AM, Fineberg NA, Sahakian BJ, Robbins TW. Enhanced avoidance habits in obsessive-compulsive disorder. Biol Psychiatry. 2014;75(8):631–8.
Article
PubMed
PubMed Central
Google Scholar
Maia TV, Frank MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci. 2011;14(2):154–62.
Article
CAS
PubMed
PubMed Central
Google Scholar
Fibiger HC, Phillips AG, Zis AP. Deficits in instrumental responding after 6-hydroxydopamine lesions of the nigro-striatal dopaminergic projection. Pharmacol Biochem Behav. 1974;2:87–96.
Article
CAS
PubMed
Google Scholar
Koob GF, Simon H, Herman JP, Le Moal M. Neuroleptic-like disruption of the conditioned avoidance response requires destruction of both mesolimbic and nigrostriatal dopamine systems. Brain Res. 1984;303:319–29.
Article
CAS
PubMed
Google Scholar
Darvas M, Fadok JP, Palmiter RD. Requirement of dopamine signaling in the amygdala and striatum for learning and maintenance of a conditioned avoidance response. Learn Mem. 2011;18(3):136–43.
Article
CAS
PubMed
PubMed Central
Google Scholar
Beninger RJ, Mason ST, Phillips AG, Fibiger HC. The use of extinction to investigate the nature of neuroleptic-induced avoidance deficits. Psychopharmacology. 1980;69:11–8.
Article
CAS
PubMed
Google Scholar
Beninger RJ. The role of serotonin and dopamine in learning to avoid aversive stimuli. In: Archer T, Nilsson LG, editors. Aversion, avoidance and anxiety: perspectives on aversively motivated behavior. Hillsdale: Lawrence Erlbaum Associates; 1989. p. 265–84.
Google Scholar
Wadenberg MG, Ericson E, Magnusson O, Ahlenius S. Suppression of conditioned avoidance behavior by the local application of (-)sulpiride into the ventral, but not the dorsal, striatum of the rat. Biol Psychiatry. 1990;28:297–307.
Article
CAS
PubMed
Google Scholar
Tye KM, Mirzabekov JJ, Warden MR, Ferenczi EA, Tsai HC, Finkelstein J, Kim SY, Adhikari A, Thompson KR, Andalman AS, Gunaydin L, Witten I, Deisseroth K. Dopamine neurons modulate neural encoding and expression of depression-related behaviour. Nature. 2013;493:537–41.
Article
CAS
PubMed
Google Scholar
Anstrom KK, Woodward DJ. Restraint increases dopaminergic burst firing in awake rats. Neuropsychopharmacology. 2005;10(10):1832–40.
Article
CAS
Google Scholar
Chaudhury D, Walsh JJ, Friedman AK, Juarez B, Ku SM, Koo JW, Ferguson D, Tsai HC, Pomeranz L, Christoffel DJ, et al. Rapid regulation of depression-related behaviours by control of midbrain dopamine neurons. Nature. 2013;493:532–6.
Article
CAS
PubMed
Google Scholar
Friedman AK, Walsh JJ, Juarez B, Ku SM, Chaudhury D, Wang J, Li X, Dietz DM, Pan N, Vialou VF, et al. Enhancing depression mechanisms in midbrain dopamine neurons achieves homeostatic resilience. Science. 2014;344(6181):313–9.
Article
CAS
PubMed
PubMed Central
Google Scholar
Hollon NG, Burgeno LM, Phillips PE. Stress effects on the neural substrates of motivated behavior. Nat Neurosci. 2015;18(10):1405–12.
Article
CAS
PubMed
Google Scholar
Bland ST, Twining C, Watkins LR, Maier SF. Stressor controllability modulates stress-induced serotonin but not dopamine efflux in the nucleus accumbens shell. Synapse. 2003;49:206–8.
Article
CAS
PubMed
Google Scholar
Bland ST, Hargrave D, Pepin JL, Amat J, Watkins LR, Maier SF. Stressor controllability modulates stress-induced dopamine and serotonin efflux and morphine-induced serotonin efflux in the medial prefrontal cortex. Neuropsychopharmacology. 2003;28:1589–96.
Article
CAS
PubMed
Google Scholar
Amat J, Matus-Amat P, Watkins LR, Maier SF. Escapable and inescapable stress differentially alter extracellular levels of 5-HT in the basolateral amygdala of the rat. Brain Res. 1998;812:113–20.
Article
CAS
PubMed
Google Scholar
Amat J, Matus-Amat P, Watkins LR, Maier SF. Escapable and inescapable stress differentially and selectively alter extracellular levels of 5-HT in the ventral hippocampus and dorsal periaqueductal gray of the rat. Brain Res. 1998;797:12–22.
Article
CAS
PubMed
Google Scholar
Amat J, Baratta MV, Paul E, Bland ST, Watkins LR, Maier SF. Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat Neurosci. 2005;8:365–71.
Article
CAS
PubMed
Google Scholar
Amo R, Fredes F, Kinoshita M, Aoki R, Aizawa H, Agetsuma M, Aoki T, Shiraki T, Kakinuma H, Matsuda M, et al. The habenulo-raphe serotonergic circuit encodes an aversive expectation value essential for adaptive active avoidance of danger. Neuron. 2014;84(5):1034–48.
Article
CAS
PubMed
Google Scholar
Li J, Daw ND. Signals in human striatum are appropriate for policy update rather than value prediction. J Neurosci. 2011;31(14):5504–11.
Article
CAS
PubMed
PubMed Central
Google Scholar
Kishida KT, Saez I, Lohrenz T, Witcher MR, Laxton AW, Tatter SB, White JP, Ellis TL, Phillips PE, Montague PR. Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward. Proc Natl Acad Sci USA. 2016;113(1):200–5.
Article
CAS
PubMed
Google Scholar
D’Ardenne K, Lohrenz T, Bartley KA, Montague PR. Computational heterogeneity in the human mesencephalic dopamine system. Cogn Affect Behav Neurosci. 2013;13(4):747–56.
Article
PubMed
PubMed Central
Google Scholar
Lohrenz T, McCabe K, Camerer CF, Montague PR. Neural signature of fictive learning signals in a sequential investment task. Proc Natl Acad Sci USA. 2007;104(22):9493–8.
Article
CAS
PubMed