Does this paper say what it means?

This is another neuroimaging study using another methodology: Positron Emission Tomography (PET). It is a secondary and post hoc analysis from a project: the neuroimaging data has been “salami sliced” from the main paper and has been promoted as a positive result. If you read the abstract, it looks like that is the case.

m_yoi160043f1

Importance Biomarkers that predict suicidal behavior, especially highly lethal behavior, are urgently needed. In cross-sectional studies, individuals with depression who attempt suicide have lower midbrain serotonin transporter binding potential compared with those who do not attempt suicide, and higher serotonin1A binding potential in the raphe nuclei (RN) is associated with greater lethality of past suicide attempts and suicidal intent and ideation.

Objectives To determine whether serotonin transporter binding potential in the lower midbrain predicts future suicide attempts and whether higher RN serotonin1A binding potential predicts future suicidal ideation and intent and lethality of future suicide attempts.

Design, Setting, and Participants In this prospective 2-year observational study, a well-characterized cohort of 100 patients presenting for treatment of a major depressive episode of at least moderate severity underwent positron emission tomography using carbon 11–labeled N-(2-(1-(4-(2-methoxyphenyl)-1-piperazinyl)ethyl))-N-(2-pyridyl)-cyclohexanecarboxamide ([11C]WAY-100635), a serotonin1A antagonist; a subset of 50 patients also underwent imaging with carbon 11–labeled 3-amino-4-(2-dimethylaminomethyl-phenylsulfanyl)- benzonitrile ([11C]DASB), a serotonin transporter radioligand. Imaging was performed at Columbia University Medical Center from May 3, 1999, to March 11, 2008. Follow-up was completed on May 28, 2010, and data were analyzed from August 1, 2013, to March 1, 2016.

Exposures Patients were treated naturalistically in the community and followed up for 2 years with documentation of suicidal behavior, its lethality, and suicidal ideation and intent.

Main Outcomes and Measures Suicide attempt or suicide.

Results Of the 100 patients undergoing follow-up for more than 2 years (39 men; 61 women; mean [SD] age, 40.2 [11.2] years), 15 made suicide attempts, including 2 who died by suicide. Higher RN serotonin1A binding potential predicted more suicidal ideation at 3 (b?=?0.02; t?=?3.45; P?=?.001) and 12 (b?=?0.02; t?=?3.63; P?=?.001) months and greater lethality of subsequent suicidal behavior (b?=?0.08; t?=?2.89; P?=?.01). Exploratory analyses suggest that the serotonin1A binding potential of the insula (t?=?2.41; P?=?.04), anterior cingulate (t?=?2.27; P?=?.04), and dorsolateral prefrontal cortex (t?=?2.44; P?=?.03) were also predictive of lethality. Contrary to our hypotheses, suicidal intent was not predicted by serotonin1A binding potential in any brain region (F1,10?=?0.83; P?=?.38), and midbrain serotonin transporter binding potential did not predict future attempts (log-rank ?21?=?0.4; P?=?.54), possibly owing to low power.

Conclusions and Relevance Greater RN serotonin1A binding potential predicted higher suicidal ideation and more lethal suicidal behavior during a 2-year period. This effect may be mediated through less serotonin neuron firing and release, which affects mood and suicidal ideation and thereby decision making.

That looked interesting. But, when I read the results, I found something that confirmed a concern from the first table. To get a positive correlation, they removed the people who had no lethal suicide attempts. At that point, they gained one significant result.

After removing 3 patients with zero-lethality follow-up attempts in an exploratory analysis, we found a positive correlation between serotonin1A BPF and future lethality (b?=?0.09; t9?=?2.38; P?=?.04) across all ROIs (interaction with ROI: F11,11?=?0.99; P?=?.45). The ROI-specific post hoc analyses without these 3 patients showed significant associations between future lethality and serotonin1A BPF in the RN, anterior cingulate, dorsolateral PFC, and insula. After Benjamini-Hochberg correction, these regions showed a strengthened but nonsignificant association

The Benjami-Hochberg correction assumes independence between tests, which is not the wisest assumption when considering neuroimaging. There may be no significant results in this table. If one uses a Bonferonni correction (p == 0.05/N tests), with the same assumptions, one would eyeball the p value is around 0.10 if one adds in the “zero suicide lethality” group for the raphe nucleus and about 0.03 if one removes them.

m_yoi160043t2

And if one looks at the graph of Raphe nucleus serotonin binding at each time point there is marked scatter: one wonders if the variation as measured has significance.

yoi160043f2

PET scans cost around two to four thousand dollars in NZ. They result in about 2 -10 mSV radiation, or three to five years normal exposure. The sampling frame is skewed. The correlations are poor.

This paper should not change practice. It is, however, an example of how to not do a post hoc analysis.

2 thoughts on “Does this paper say what it means?

  1. Most of what is called soft science is pseudo science. See the Feynman discussion on utube

  2. One of the interesting things about being around the Medical sphere, after a deep background in classic mathematics, is how overly dependent Medicine has become on Statistics. While at the same time being completely unable to grasp that you can torture numbers all day, but if you don’t have extremely tight controls on your inputs, all of your data is worthless.

    There’s actually an understanding around Mathematics, as a field, that no one truly working in Statistics does anything outside of abstract concepts, Game Theory or Sports Analysis. The reason for this is actually applying Statistical Analysis would render most of it as “no relevancy/worthless”.

    Further, there’s almost never any consideration of the error factors intrinsic in the testing regime itself. In a lot of cases, it would obliterate most significance of the resulting mathematics work. Which explains a lot of the problems with the “drug pipeline” lately, as the Research Journals are chalk full of worthless papers.

Comments are closed.