Comparison of probabilistic choice models in humans
- Taiki Takahashi^{1}Email author,
- Hidemi Oono^{2} and
- Mark HB Radford^{2}
DOI: 10.1186/1744-9081-3-20
© Takahashi et al; licensee BioMed Central Ltd. 2007
Received: 11 November 2006
Accepted: 20 April 2007
Published: 20 April 2007
Abstract
Background
Probabilistic choice has been attracting attention in psychopharmacology and neuroeconomics. Several parametric models have been proposed for probabilistic choice; entropy model, Prelec's probability weight function, and hyperbola-like probability discounting functions.
Methods
In order to examine (i) fitness of the probabilistic models to behavioral data, (ii) relationships between the parameters and psychological processes, e.g., aversion to possible non-gain in each probabilistic choice and aversion to unpredictability, we estimated the parameters and AICc (Akaike Information Criterion with small sample correction) of the probabilistic choice models by assessing the points of subjective equality at seven probability values (95%–5%). We examined both fitness of the models parametrized by utilizing AICc, and the relationships between the model parameters and equation-free parameter of aversion to possible non-gain.
Results
Our results have shown that (i) the goodness of fitness for group data was [Entropy model>Prelec's function>General hyperbola>Simple hyperbola]; while Prelec's function best fitted individual data, (ii) aversion to possible non-gain and aversion to unpredictability are distinct psychological processes.
Conclusion
Entropy and Prelec models can be utilized in psychopharmacological and neuroeconomic studies of risky decision-making.
Background
Studies in psychopharmacology, neuroscience, and behavioral economics have revealed that humans and non-human animals discount the value of probabilistic rewards as the receipt becomes more uncertain ("probability discounting", [1–5]). Because pathological gambling and drug misuse are associated with low degree of aversion to uncertainty in probabilistic choice, it is of psychopharmacological interest to examine probabilistic choice models. In neoclassical economic theory, the expected utility theory [6] has been allowed to express subject' risk-attitude (risk-aversion/seeking). In Kahneman and Tversky's prospect theory [7], an extension of expected utility theory models, a subjective value of a probabilistic outcome is expressed as Af(p), where A is a value of a certain reward and f(p) is a some function of p, corresponding to subjective probability weight. However, psychological processes underlying discounting uncertain rewards are yet to be investigated. Several important findings on probabilistic choice are observed in behavioral economic, behavioral ecological and psychopharmacological studies. For instance, (i) behavioral economists have demonstrated that in probabilistic choice, small probabilities are overweighted; while large probabilities are underweighted [7–10], (ii) a theoretical framework for intertemporal choice is useful for analyzing probabilistic choice, by replacing delay in intertemporal choice with an average waiting time until winning, i.e. "odds-against", O = (1- probability)/probability [11–13], (iii) people have aversion to loss of information about outcomes of decision under uncertainty (e.g. ambiguity aversion, [14]). Based on these findings, several types of parametric models of probabilistic choice have been proposed in behavioral economics, ecology, psychopharmacology, and neuroeconomics as introduced below. However, to date, comparison of explanatory power of the probabilistic models, and the relationships between parameters in the probabilistic choice models and psychological processes have not extensively investigated.
Entropy model
where A is the value of a certain reward when p = 1, 0 <a < 1 indicates a psychophysical effect on small probability estimation (importantly, smaller a values correspond to strong overestimation of small probability, because 0 <p < 1) and T is a degree of aversion to lack of information [15]. A recent neuroimaging study employed a psychologically similar probabilistic choice model and reported that during anticipation periods in probabilistic choice, degrees of unpredictability were correlated with the activations in an area extending posterior to and bilateral from the ventral striatum to the subthalamic nucleus as well as mediodorsal thalamic nucleus, midbrain, and bilateral anterior insula [16]. The entropy-based model of probabilistic choice has an advantage that each parameter has specific psychological correlates (i.e. psychophysics of probability estimation and aversion to lack of information).
Prelec's probability weight function
where α and β are free parameters and ln indicates a natural log (= log_{ e }). An important characteristic of Prelec's weight function is that this captures human bias in probabilistic choice (i.e. overweighting of small probabilities and underweighting of large probabilities). Notably, when β is assumed to be 1, V is equal to a statistical expected value = Ap at p = 1/e ≈ 0.37, which is reportedly consistent with several behavioral data [8–10]. A recent neuroeconomic study (employing Prelec's weight function with the assumption of β = 1, following Prelec's proposal) observed that α is related to the activity of the anterior cingulate in the brain, and the authors interpreted the activity of the anterior cingulated might reflect subject's risk attitude in probabilistic choice [17]. However, it is still unknown whether the parameters in Prelec's weight function are actually related to subject's risk aversion (i.e., aversion to possible non-gain in each probabilistic choice and delay until winning, [11]). To examine this question is a part of the objectives of the present study.
Hyperbolic models
Psychopharmacological studies have employed these hyperbolic models to describe subjects' probabilistic choice [1–4, 19]. However, recent studies have cast a doubt on the hypothetical equivalence of a decrease in probability to an increase in delay; specifically, several studies reported that parameters in delay and probability discounting functions are not so strongly correlated as originally supposed [13, 20], and some studies have reported that neurobiological/psychopharmacological manipulation distinctly impacts delay and probability discounting [3, 21]. Additionally, a recent study by Bickel's group has proposed an analytical strategy combining delay and probability [22].
In the present study, the goodness of fitness of each equation to the behavioral data in probabilistic choice was quantified with AICc (Akaike Information Criterion with small sample correction, a second-order AIC, [23]) and compared at both group and individual levels. Furthermore, to avoid equation type-dependent systematic errors, we also computed each subject's AUC (i.e., area under the normalized indifference curve) [24] in order to quantify the subject's degree of risk aversion (aversion to possible non-gain in each probabilistic choice). Finally, we computed correlation coefficients between the parameters in probabilistic choice models and individuals' degrees of discounting probabilistic rewards, in order to elucidate a psychological correlate of the parameters of the probabilistic choice models.
Methods
Participants
Twenty-one (9 male and 12 female) volunteer students from a major national university in Japan participated in the experiment. The average age was 22.05 (SD = 2.2) years. They were recruited from several psychology classes. They participated in a probabilistic choice task, for their parameters of probabilistic choice models to be estimated.
Procedure
We used exactly the same experimental procedure as Ohmura et al's [20] in the probabilistic choice task. Firstly, participants were seated individually in a quiet room, and face the experimenter across a table. After that, participants received the simple instruction that the monetary reward in this experiment was hypothetical, but the experimenter wanted you to think as though it were real money. Then the participants were asked to choose between the card describing money delivered certainly and the card describing money delivered with a certain degree of probability. The left card viewed from participants indicated the amounts of money that could be received certainly, and the right card indicated 100,000 yen that could be received with a certain probability.
For the probabilistic choice task, monetary rewards and the probability were printed on 3 × 5 index cards. The 27 monetary reward amounts were 100,000 yen (about $1,000), 99,000 yen, 96,000 yen, 92,000 yen, 85,000 yen, 80,000 yen, 75,000 yen, 70,000 yen, 65,000 yen, 60,000 yen, 55,000 yen, 50,000 yen, 45,000 yen, 40,000 yen, 35,000 yen, 30,000 yen, 25,000 yen, 20,000 yen, 15,000 yen, 10,000 yen, 8,000 yen, 6,000 yen, 4,000 yen, 2,000 yen, 1,000 yen, 500 yen, and 100 yen. The seven probabilities of receipt were 95%, 90%, 70%, 50%, 30%, 10%, and 5%.
The experimenter turned the 27 100% cards sequentially. The card started with 100,000 yen, down to 100 yen, and back to 100,000 yen. For each card, the participant pointed the certain (100%) or probabilistic reward. The experimenter wrote down the last 100% reward chosen in the descending order, and the first 100% reward chosen in the ascending order, and the average of them was used as the point of subjective equality (hereafter indifference point) in the following analysis. This procedure was repeated at each of the seven probabilities (for more details, [20]).
Data analysis
We employed both equation type-dependent and -independent parameters. For estimating equation type-dependent parameters (i.e. a and T in the entropy model, α and β in Prelec's weight function, k and s in the general hyperbolic function, and k in the simple hyperbolic function). We fitted the four types of the model equations (i.e. Equation 1–4) to the data (R statistical language, non-linear modeling package) and the fitness of each equation was estimated with AICc (Akaike Information Criterion with small sample correction) values, which is the most standard criterion for the fitness of mathematical model to observed data with a small sample size [23]. It should be noted that the comparison between the R-square values of equations with different numbers of free parameters are statistically irrelevant (note that an increase in the numbers of free parameters in a fitting equation always yield a larger R-square value[25]). Therefore, we compared AICc values for the equations (note that smaller AICc values correspond to better fitting). Furthermore, calculating R-square values demonstrated that, the models with two parameters (i.e. Entropy, Prelec, and general hyperbola) had larger R-squre values than the simple hyperbola (which is a statistically trivial result), but no statistically significant difference was observed between the models with two parameters. Because utilizing R-square in non-linear curve fitting is statistically problematic [23], we do not present R-square values below.
For an equation type-independent parameter of probabilistic choice, we adopted AUC, which is defined as the normalized area under the linkage of indifference points (i.e., subjective values) at each odds against (= (1- probabilit y)/probability [11]). The rationale for employing AUC is that (i) AUC indicates subject's aversion to possible non-gain (or possible temporal delay until winning [11], (ii) AUC does not depend on the type of fitting functions, and (iii) studies in psychopharmacology often utilize AUC as an equation-free parameter for probabilistic choice [3, 24].
In order to examine the relationship between the parameters of the probabilistic choice models and subject's degree of aversion to possible non-gain (risk aversion), we utilized Pearson's correlation analyses between the parameters of the model equations and AUC, because Kolmogorov-Smirnov tests revealed no significant deviation from Gaussian distribution in all parameter distributions (p > .05). It is to be noted that a significant correlation in the Pearson's analysis indicates that the parameter is related to psychological processes of aversion to possible non-gain (or possible waiting time until winning).
For the analysis of group data, we utilized group median data [26]. The reason is that a (linearly averaged) group mean may not be a valid statistical summary of group data when the relationship between probability values and subjective values is non-linear.
All statistical procedures were conducted with R statistical language (The R Project for Statistical Computing). Data are expressed as Mean and/or Median ± Standard Error of Mean (SEM). Significance level was set at 5% throughout (for pairwise multiple comparisons, Bonferoni's correction was utilized).
Results
Fitness of probabilistic choice models for group data
Parameters and AICc (Akaike Information Criterion with small sample correction) of probabilistic choice models for group data
Entropy model | Prelec's function | General hyperbola | Simple hyperbola | ||||
---|---|---|---|---|---|---|---|
AICc | 137.96 | 138.61 | 139.46 | 143.05 | |||
Parameter | a | T | α | β | k | s | k |
0.59437 | 0.21222 | 0.75456 | 1.02024 | 3.21403 | 0.55207 | 1.195 |
Fitness of probabilistic choice models for individual data
AICc (Akaike Information Criterion with small sample correction) and parameters of probabilistic choice models for individual data
Entropy model | Prelec's function | General hyperbola | Simple hyperbola | ||||
---|---|---|---|---|---|---|---|
AIC_{c} | 133.31 ± 1.79 | 129.38 ± 1.80 | 130.82 ± 1.80 | 139.91 ± 1.80 | |||
Parameter | a | T | α | β | k | s | k |
0.691 ± 0.090 | 0.197 ± 0.045 | 0.774 ± 0.068 | 1.063 ± 0.075 | 10.910 ± 3.23 | 0.6348 ± 0.110 | 1.59 ± 0.33 |
Correlation between probabilistic model parameters and AUC for individual data
Discussion
To our knowledge, this study is the first to systematically examine the fitness of probabilistic choice models by employing AICc as an indicator of the fitness. Previous studies have focused on the comparison of R-square values; however, this is problematic and statistically irrelevant, because the model equations have different numbers of free parameters and non-linear [18, 23, 25].
Comparison of probabilistic choice models at the group level
The order of fitness of the model is [entropy model > Prelec's weight function > general hyperbolic function > simple hyperbolic function]. Although the differences are not dramatically large in a quantitative sense, it may be recommended, in future psychopharmacological studies on probabilistic choice, to utilize the entropy model for examining group differences, in order to elucidate the distinctions in neuropharmacological mechanisms between healthy controls and addicts. Moreover, it should be noted that β was close to 1, which confirms that people's probability estimation is the most accurate at the probability value of about 0.37, as Prelec originally proposed and observed in behavioral data [8]. However, this does not necessarily allow us to assume β = 1 a priori.
Comparison of probabilistic choice models at the individual level
Prelec's weight function may be the best to parametrize each individual's probabilistic choice, when the objective of the study is to solely reveal individual differences in probabilistic choice. Several authors in neuroeconomics have assumed β = 1 and estimate α in order to assess individual differences in risk aversion. However, the problem of Prelec's function is that the parameters α and β do not have clear interpretation, in terms of psychological functioning. We therefore examined the relationships between parameters in each probabilistic choice model and AUC (an index of risk-aversion [11]), in order to better understand the psychological interpretation of parameters in the probabilistic choice models.
Relationships between probabilistic model parameters and AUC
Our correlational analysis demonstrated that a in the entropy model, β in Prelec's weight function, and general hyperbolic s, and simple hyperbolic k were associated with each subject's risk aversion (assessed with AUC, see Fig. 2). It is important to note that T in the entropy model did not relate to risk aversion, confirming that aversion to lack of information and risk are distinct psychological processes [14]. Psychological interpretation for this distinction is that unpredictability-aversive subjects do not necessarily avoid uncertain rewards with very small probabilities (i.e., predictable non-gains), due to predictability of the outcomes. Our result is, as far as we know, the first to dissociate these two attitudes toward uncertainty in a parametric manner. Furthermore, the finding that not α but β in Prelec's weight function was associated with risk aversion is also important. This finding implies that when the objective of the study is to assess each subject's risk aversion, β should not be assumed to be 1. It is also confirmed that simple hyperbolic k, rather than general hyperbolic k is related to subject's risk aversion. Moreover, as can be seen from Fig. 2, simple hyperbolic parameters at large values are not good predictors of AUC. This is also an important point of cautions for psychopharmacological study on probabilistic choice employing hyperbolic function parameters.
Limitations and future directions
We now discuss limitations of the present study. Because the present study employed hypothetical money, although discounting behaviors of real and hypothetical rewards have been shown to correlate [27, 28], it is not completely evident that the degrees of discounting real monetary gains were exactly reflected. However, probabilistic choice tasks with real money have other limitations; e.g., it is virtually impossible or difficult to utilize large real rewards and real monetary loss, and paying real money for randomly selected participants introduces another probabilistic factor. Nevertheless, real gambling tasks such as the IOWA gambling task [29] may also help understand neuropsychological functioning underlying probabilistic choice. Furthermore, it is also of interest to examine the relationship between ambiguity aversion in decision under uncertainty with unknown probability [14] and subjects' degree of unpredictability aversion, to elucidate substance abusers' risk-taking, and psychopharmacological treatment for impulsive behavior.
Conclusion
Our present study demonstrated that (i) the entropy model best fit group level data of probabilistic choice, (ii) Prelec's weight function best fit individual data, and (iii) aversions to risk and lack of information are distinct psychological processes. Further studies are required in order to answer whether psychiatrics such as drug-dependent subjects and pathological gamblers actually differ in parameters in the entropy model and Prelec's weight function.
Declarations
Acknowledgements
The research reported in this paper was supported by a grant from the Grant- in-Aid for Scientific Research ("21 st century center of excellence" grant) from the Ministry of Education, Culture, Sports, Science and Technology of Japan, and Yamaguchi endocrinological disorder research grant.
Authors’ Affiliations
References
- Anderson IM, Richell RA, Bradshaw CM: The effect of acute tryptophan depletion on probabilistic choice. J Psychopharmacol. 2003, 17: 3-7. 10.1177/0269881103017001687.View ArticlePubMedGoogle Scholar
- Mitchell SH: Measures of impulsivity in cigarette smokers and non-smokers. Psychopharmacology(Berl). 1999, 146: 455-464. 10.1007/PL00005491.View ArticleGoogle Scholar
- Ohmura Y, Takahashi T, Kitamura N: Discounting delayed and probabilistic monetary gains and losses by smokers of cigarettes. Psychopharmacology (Berl). 2005, 182: 508-515. 10.1007/s00213-005-0110-8.View ArticleGoogle Scholar
- Reynolds B, Richards JB, Horn K, Karraker K: Delay discounting and probability discounting as related to cigarette smoking status in adults. Behav Processes. 2004, 65: 35-42. 10.1016/S0376-6357(03)00109-8.View ArticlePubMedGoogle Scholar
- Richards JB, Zhang L, Mitchell SH, de Wit H: Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. J Exp Anal Behav. 1999, 71: 121-143. 10.1901/jeab.1999.71-121.PubMed CentralView ArticlePubMedGoogle Scholar
- von Neumann J, Morgenstern O: Theory of Games and Economic Behavior. 1947, Princeton, NJ Princeton Univ. PressGoogle Scholar
- Kahneman D, Tversky A: Prospect Theory: An Analysis of Decision under Risk. Econometrica. 1979, 47: 263-292. 10.2307/1914185.View ArticleGoogle Scholar
- Prelec D: The probability weighting function. Econometrica. 1998, 66: 497-527. 10.2307/2998573.View ArticleGoogle Scholar
- Wu G, Gonzalez R: Curvature of the probability weighting function. Manage Sci. 1996, 42: 1676-1690.View ArticleGoogle Scholar
- Wu G, Gonzalez R: Nonlinear decision weights in choice under uncertainty. Manage Sci. 1999, 45: 74-85.View ArticleGoogle Scholar
- Rachlin H, Raineri A, Cross D: Subjective probability and delay. J Exp Anal Behav. 1991, 55: 233-244. 10.1901/jeab.1991.55-233.PubMed CentralView ArticlePubMedGoogle Scholar
- Green L, Myerson J, Ostaszewski P: Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes. J Exp Psychol Learn Mem Cogn. 1999, 25: 418-427. 10.1037/0278-7393.25.2.418.View ArticlePubMedGoogle Scholar
- Myerson J, Green L, Hanson JS, Holt DD, Estle SJ: Discounting delayed and probabilistic rewards: Processes and traits. J Econ Psychol. 2003, 24: 619-635. 10.1016/S0167-4870(03)00005-9.View ArticleGoogle Scholar
- Camerer CF, Weber M: Recent developments in modeling preferences: Uncertainty and ambiguity. J Risk Uncertain. 1992, 5: 325-370. 10.1007/BF00122575.View ArticleGoogle Scholar
- Takahashi T: A mathematical framework for probabilistic choice based on information theory and psychophysics. Med Hypotheses. 2006, 67: 183-186. 10.1016/j.mehy.2005.12.045.View ArticlePubMedGoogle Scholar
- Preuschoff K, Bossaerts P, Quartz S: Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures. Neuron. 2006, 51: 381-390. 10.1016/j.neuron.2006.06.024.View ArticlePubMedGoogle Scholar
- Paulus MP, Frank LR: Anterior cingulate activity modulates nonlinear decision weight function of uncertain prospects. Neuroimage. 2006, 30: 668-677. 10.1016/j.neuroimage.2005.09.061.View ArticlePubMedGoogle Scholar
- Green L, Fry AF, Myerson J: Discounting of delayed rewards: A lifespan comparison. Psychol Sci. 1994, 5: 33-36. 10.1111/j.1467-9280.1994.tb00610.x.View ArticleGoogle Scholar
- Mobini S, Chiang TJ, Ho MY, Bradshaw CM, Szabadi E: Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl). 2002, 152: 390-397. 10.1007/s002130000542.View ArticleGoogle Scholar
- Ohmura Y, Takahashi T, Kitamura N, Wehr P: Three-month stability of delay and probability discounting measures. Exp Clin Psychopharmacol. 2006, 14: 318-328. 10.1037/1064-1297.14.3.318.View ArticlePubMedGoogle Scholar
- Adriani W, Laviola G: Delay aversion but preference for large and rare rewards in two choice tasks: Implications for the measurement of self-control parameters. BMC Neurosci. 2006, 7: 52-10.1186/1471-2202-7-52.PubMed CentralView ArticlePubMedGoogle Scholar
- Yi R, de la Piedad X, Bickel WK: The combined effects of delay and probability in discounting. Behav Process. 2006, 73: 149-155. 10.1016/j.beproc.2006.05.001.View ArticleGoogle Scholar
- Burnham KP, Anderson DR: Model selection and multimodel inference: A practical information-theoretic approach. 2002, New York, Springer-Verlag, 2Google Scholar
- Myerson J, Green L, Warusawitharana M: Area under the curve as a measure of discounting. J Exp Anal Behav. 2001, 76: 235-243. 10.1901/jeab.2001.76-235.PubMed CentralView ArticlePubMedGoogle Scholar
- Green L, Myerson J, Lichtman D, Rosen S, Fry A: Temporal discounting in choice between delayed rewards: The role of age and income. Psychol Aging. 1996, 11: 79-84. 10.1037/0882-7974.11.1.79.View ArticlePubMedGoogle Scholar
- Bickel WK, Odum AL, Madden GJ: Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers. Psychopharmacology (Berl). 1999, 146: 447-454. 10.1007/PL00005490.View ArticleGoogle Scholar
- Madden GJ, Raiff BR, Lagorio CH, Begotka AM, Mueller AM, Hehli DJ, Wegener AA: Delay discounting of potentially real and hypothetical rewards: II. Between- and within-subject comparisons. Exp Clin Psychopharmacol. 2004, 12: 251-261. 10.1037/1064-1297.12.4.251.View ArticlePubMedGoogle Scholar
- Johnson MW, Bickel WK: Within-subject comparison of real and hypothetical money rewards in delay discounting. J Exp Anal Behav. 2002, 77: 129-146. 10.1901/jeab.2002.77-129.PubMed CentralView ArticlePubMedGoogle Scholar
- Bechara A, Damasio H, Tranel D, Damasio AR: Deciding advantageously before knowing the advantageous strategy. Science. 1997, 275: 1293-1295. 10.1126/science.275.5304.1293.View ArticlePubMedGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.