The problems of meta analysis.

Among those who suffer with psychosis, many have but a partial or minimal response to any medication they can tolerate. This has led to trials of multiple adjuvant medications. This meta analysis looks at classes of medications, and notes that the recommendations for augmentation do not follow the evidence from the pooled data.

Altogether, 1 of the 42 included augmentation strategies was explicitly recommended by the authors, and 20 were at least partially recommended. There were 3 strategies (ie, nonsteroidal anti-inflammatory drugs, lithium, and lamotrigine) that were not recommended, despite having significant effect sizes favoring the combination treatment, because of methodological limitations. Also, there were 12 strategies that were at least partially recommended by authors of the meta-analyses despite their lack of significant differences between combination treatments and controls (ie, selective serotonin reuptake inhibitors, serotonin-noradrenaline reuptake inhibitors, tricyclic antidepressants, norepinephrine reuptake inhibitors, dopamine and norepinephrine reuptake inhibitors, monoamine-oxydase-B inhibitors, acetylcholinesterase inhibitors, and n-methyl-d-aspartate antagonists combined with any antipsychotic drug, as well as a second antipsychotic drug, topiramate, lamotrigine, or glycine combined with clozapine)

The authors continue in their discussion to discuss the quality scoring and the need for more rigour in this. I find this highly problematic. Scoring for quality is very subjective: there are disagreements in every meta analytic team on this, and the scales are still being developed. I also think pooling the raw data does not remove the quality issue: the risk of bias is baked into the study design not the participant.

Across 37 pharmacologic combination strategies with any antipsychotic drug, 14 outperformed controls for the primary outcome of total psychopathology, mostly with large to medium effect sizes. Conversely, none of the 5 combinations with clozapine outperformed controls regarding total psychopathology. The recommendation to clinicians by the authors of each meta-analysis favoring using the combination treatment was correlated with the effect size produced by each meta-analysis. However, when all this meta-analytic literature was compared regarding the quality of its meta-analyzed content, the effect sizes were inversely correlated with the study quality, reducing confidence in these affirmative recommendations.

Given our results of an absent high-quality efficacy signal in study-based meta-analyses for the entire studied population with schizophrenia, patient-based meta-analyses should be performed to identify moderators and subgroups of responders. Furthermore, patient-level meta-analyses could also disentangle the heterogeneity introduced by poor-quality trials from the true variation in treatment responses around a “mean” effect by allowing for such powered moderator and subgroup analyses. The move toward developing a platform that allows for the exchange of individual patient data for patient-level meta-analyses will require standardizing methods, data elements and database structures, and funding or incentives to build a data repository, but the initial examples already exist (http://yoda.yale.edu/).

The authors reject their findings on the basis that the papers are not of high enough quality. I think they protest too much. No paper is perfect: all papers have problems. It is better to say that there may be a place for augmentation, particularly with substances that are safe enough to be sold in supermarkets.

Leave a Reply