I am showing in the graph a secondary analysis from a paper that pooled the cognitive tests in the placebo arm of 12 cognitive remediation RCTs in psychosis. Those with worse pre randomization testing improved: those with better testing had no differences. There is a question as to if this shows reversion to the mean.

In the discussion, the authors note:

These results have implications for study design. First, researchers will need to weigh the value of additional assessments during clinical trials against the potential for an additional learning effect in patients receiving placebo and active treatment, although this effect is small. Second, important information may be gained from a prebaseline assessment. Patients whose cognitive performance worsened from screening to baseline had the largest placebo effect, and those with the largest improvements from screening to baseline had no placebo effect at all. These results, consistent with the principle of regression to the mean, suggest that, for schizophrenia cognition studies, the concept of eliminating “placebo responders” who would then go on to have large placebo effects during a trial is not a viable approach. However, a screening assessment may produce information that can facilitate study design, such as stratification of patients based on potential placebo effects. It may also allow statistical approaches that adjust for screening values (along with the customary baseline values) in the primary efficacy model as a method for reducing the noise associated with large fluctuations in patients who may have had exceptionally good or bad days at the baseline visit. Preliminary analyses using this method suggest that it may increase precision of assessing treatment effects in cognitive treatment trials.40 Other individual difference characteristics found in our analyses to be associated with placebo group improvements, including more severe depression/anxiety and more motivation, could be combined with prebaseline changes to equalize factors that appear to increase the placebo response.

We have to recall that “placebo” includes the same assessments and expectation of hope as the active treatment group had. These are not without some efficacy. Moreover, with any testing there is a practice effect. The pooled data suggests thee effects are small, but real, in this population.

The risk is that this leads to more elaborate analyses, applied without considering if the assumptions inherent in the statistical model fit the data. It may be better, as the authors discuss, to find better ways of measuring cognitive deficits and recovery.