- Open Access
Why are psychiatric imaging methods clinically unreliable? Conclusions and practical guidelines for authors, editors and reviewers
© Borgwardt et al.; licensee BioMed Central Ltd. 2012
- Received: 8 May 2012
- Accepted: 24 August 2012
- Published: 1 September 2012
No reliable anatomical or functional alterations have been confirmed in psychiatric neuroimaging; however it can become reliable with translational impact on clinical practice when considering crucial methodological issues. We provide guidelines to authors, editors and reviewers in the implementation/evaluation of neuroimaging studies to bend neuroimaging to be more than basic neuroscience.
- Magnetic resonance imaging
- Voxel-based morphometry
More than three decades after Johnstone’s first computerised axial tomography of the brain of individuals with schizophrenia , no consistent or reliable anatomical or functional alterations have been univocally associated with any mental disorder and no neurobiological alterations have been ultimately confirmed in psychiatric neuroimaging.
A number of methodological problems may underlie the inconsistencies across studies and the difficulty of identifying reliable results. Heterogeneity in psychiatric neuroimaging originates from multiple differences across studies: in conceptual issues underlying psychiatric diagnoses and psychopathology [2, 3], the inclusion criteria for and the clinical characteristics of psychiatric samples ; the use of different paradigms and designs , and the use of different forms of image acquisition and image analysis .
The latter point is critically addressed by the recent study of Ioannidis . He stated that “the excess significance may be due to unpublished negative results, or it may be due to negative results having been turned into positive results through selective exploratory analyses”. Because of multiple comparisons across different brain regions, reporting of regions of interest (ROIs) can be guided by post-hoc significance of the results, with the whole brain results remaining unpublished . Additionally, when there are many ROI analyses that can be performed, only one of them, the one with the best results, may be presented . These practices limit the correct localization of the potential brain abnormalities, which should be based on a whole-brain analysis of the differences between patients and controls. To make an analogy, it’s as if an attorney decides to investigate only an arbitrary subgroup of the suspects of a crime, and not to report any proof, which may involve individuals which he wants to keep untarnished.
As Ioannidis acknowledged these concerns do mainly refer to morphometry studies and not directly extend to automated whole-brain voxel-based studies or functional imaging studies. In particular, voxel-based meta-analyses have the potential to overcome the limited sample size of individual studies revealing structural differences at specific brain coordinates rather than differences in volumes of pre-specified ROIs. A recently developed meta-analytic method, Signed Differential Mapping [7, 8], considers null findings as well and thus attenuates the disproportionate influence of single study data sets. However, even meta-analyses of voxel-based studies are grounded on the available published results, which often do not report null findings. In this regard, it must be noted that no meta-analytic method can detect an abnormality if this is deliberately not reported in the individual studies, e.g. by repeating the analysis with different parameters until the finding disappears. This may be the case of abnormalities in regions not thought to be related to the disorder, which may be “felt” to be false positives or artifacts  by the authors of the studies and by the peer-reviewers.
- Johnstone EC, Crow TJ, Frith CD, Husband J, Kreel L: Cerebral ventricular size and cognitive impairment in chronic schizophrenia. Lancet. 1976, 7992: 924-926.View ArticleGoogle Scholar
- Fusar-Poli P, Broome MR: Conceptual issues in psychiatric neuroimaging. Curr Opin Psychiatry. 2006, 19: 608-612. 10.1097/01.yco.0000245750.98749.1b.View ArticlePubMedGoogle Scholar
- Fusar-Poli P, Broome M, Barale F, Stanghellini G: Why is psychiatric imaging clinically unreliable? Epistemological perspectives in clinical neuroscience. Psychother Psychosom. 2009, 78: 320-321. 10.1159/000229771.View ArticlePubMedGoogle Scholar
- Fusar-Poli P, Allen P, McGuire P: Neuroimaging studies of the early stages of psychosis: a critical review. Eur Psychiatry. 2008, 23: 237-244. 10.1016/j.eurpsy.2008.03.008.View ArticlePubMedGoogle Scholar
- Fusar-Poli P, Bhattacharyya S, Allen P, Crippa JA, Borgwardt S, Martin-Santos R, Seal M, O’Carroll C, Atakan Z, Zuardi AW, McGuire P: Effect of image analysis software on neurofunctional activation during processing of emotional human faces. J Clin Neurosci. 2010, 17: 311-314. 10.1016/j.jocn.2009.06.027.View ArticlePubMedGoogle Scholar
- Ioannidis JP: Excess significance bias in the literature on brain volume abnormalities. Arch Gen Psychiatry. 2011, 68: 773-780. 10.1001/archgenpsychiatry.2011.28.View ArticlePubMedGoogle Scholar
- Radua J, van den Heuvel OA, Surguladze S, Mataix-Cols D: Meta-analytical comparison of voxel-based morphometry studies in obsessivecompulsive disorder vs. other anxiety disorders. Arch Gen Psychiatry. 2010, 67: 701-711. 10.1001/archgenpsychiatry.2010.70.View ArticlePubMedGoogle Scholar
- Radua J, Mataix-Cols D, Phillips ML, El-Hage W, Kronhaus DM, Cardoner N, Surguladze S: A new meta-analytic method for neuroimaging studies that combines reported peak coordinates and statistical parametric maps. Eur Psychiatry. 2011, published online June 7Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.