Goyal, M., Singh, S., Sibinga, E. M., Gould, N. F., Rowland-Seymour, A., Sharma, R., ... & Ranasinghe, P. D. (2014). Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA internal medicine, 174(3), 357-368.
Goyal et al. (2014) conducted a systematic conducted a systematic review and meta-analysis of meditation interventions. They limited their research to studies involving (a) adults, (b) interventions that primarily focused on meditation, (c) RCTs with an active control, and (d) participants with a “clinical condition,” which they defined “broadly to include mental health/psychiatric conditions (eg, anxiety or stress) and physical conditions (eg, lower back pain, heart disease, or advanced age).” They divided active controls into nonspecific and specific active controls. A nonspecific active control means that the control group participated in an activity that matched the treatment in terms of time and attention [meaning attention devoted to participants] but was not a known therapy; a specific active control is a therapy that has been shown to be efficacious.
Goyal et al. (2014) noted that most of the meditation studies were not methodologically sound. Many of these studies failed to control for placebo effects (e.g., waitlists and treatment as usual). And many of these studies were observational studies, meaning that participants could select which group to be part of. “Observational studies have a high risk of bias owing to problems such as self-selection of interventions (people who believe in the benefits of meditation or who have prior experience with meditation are more likely to enroll in a meditation program and report that they benefited from one) and use of outcome measures that can be easily biased by participants’ beliefs in the benefits of meditation.”
Goyal et al. (2014) found that meditation interventions cause “small improvements in anxiety, depression, and pain with moderate evidence and small improvements in stress/distress and the mental health component of health-related quality of life with low evidence when compared with nonspecific active controls. Mantra meditation programs did not improve any of the outcomes examined.” The researchers noted that the effects that meditation had on anxiety and depression “are comparable with what would be expected from the use of an antidepressant in a primary care population but without the associated toxicities.”
Goyal et al. (2014) found no evidence that meditation interventions have an effect on “positive mood, attention, sleep, and weight” and no evidence that they have an effect on “health-related behaviors affected by stress, including substance use and sleep.” And they found that whenever meditation interventions were compared to specific active controls (i.e., a known therapy), the meditation interventions were not found to be superior.
James Coyne discusses the findings of this study, writing that the “advantages of mindfulness training disappear in a fairly matched cage fight with a treatment of comparable frequency and intensity.” This study further illustrates that “[t]he domination of the MBSR literature by nonrandomized trials and randomized trials with inadequate control groups represents one contribution to an exaggeration of the efficacy of MBSR.”
Coyne points out that not only do most MBSR studies have noncomparable control groups but they use self-report questionnaires to measure growth. The placebo effect, he reminds us, is huge and adduces a study done among asthma patients. The patients were placed into one of four groups: the first received an actual inhaler to use, the second received a placebo inhaler, the third received sham acupuncture, and the fourth was waitlisted. After the intervention period, an objective measure showed that only the first group experienced physiological improvement, but according to a self-report measure, people in the first three groups reported similarly high levels of improved functioning.
* * * * *
Kreplin, U., Farias, M., & Brazil, I. A. (2018). The limited prosocial effects of meditation: A systematic review and meta-analysis. Scientific reports, 8(1), 2403.
Kreplin, Farias, & Brazil (2018). These researchers conducted a systematic review and meta-analysis on RCTs which assessed the prosocial effects of mindfulness, specifically compassion, empathy, aggression, connectedness, and prejudice. The researchers found that meditation interventions had an effect on compassion and empathy but that “compassion levels only increased under two conditions: when the teacher in the meditation intervention was a co-author in the published study; and when the study employed a passive (waiting list) control group but not an active one.”
Kreplin, Farias, & Brazil (2018) noted that the fact that compassion only increased when there was a passive control corroborated the findings of Goyal et al. (2014). They consider their second finding — that compassion only increased when the teacher was a co-author in the published study — to be novel. “At best this [finding] shows that a motivated meditation teacher will impact to a greater extent one’s students; at worse, it suggests that experimenter biases are introduced which affect the outcomes of the studies.” They found that authors ran interventions in 48 percent of the studies, and in these studies “unintentional experimenter biases could have been introduced by researchers/teachers with a personal interest in the intervention (e.g., by giving preferential treatment or being particularly enthusiastic to participants in the experimental group).”
They found just one study that assessed “beliefs about the efficacy and relevance of the intervention (meditation versus analytical training) and found that the meditation group had substantially higher expectations of a positive effect for the intervention, even though participants were not explicitly told that they were engaging in meditation.”
The authors discuss solutions to experimenter bias and expectation effects. First, measure participants’ expectations (e.g., the Credibility and Expectancy Questionnaire). Second, introducing blinding studies in these RCTs. They continue:
The authors write that many studies are also compromised by confirmation bias:
* * * * *
Coyne shows the flaws of a meta-analysis conducted by the Benson-Henry Institute. Summary: the meta-analysis includes several very flawed research studies. Also: "The larger issue is that we should not rely on promoters of MSBR products to provide unbiased estimates of their efficacy. This issue recalls very similar problems in the evaluation of Triple P Parenting Programs. Evaluations in which promoters were involved produce markedly more positive results than from independent evaluations. Exposure by my colleagues and me led to over 50 corrections and corrigendum to articles that previously had no conflicts of interest. But the process did not occur without fierce resistance from those whose livelihood was being challenged."
Coyne points out that not only do most MBSR studies have noncomparable control groups but they use self-report questionnaires to measure growth. The placebo effect, he reminds us, is huge and adduces a study done among asthma patients. The patients were placed into one of four groups: the first received an actual inhaler to use, the second received a placebo inhaler, the third received sham acupuncture, and the fourth was waitlisted. After the intervention period, an objective measure showed that only the first group experienced physiological improvement, but according to a self-report measure, people in the first three groups reported similarly high levels of improved functioning.
* * * * *
Kreplin, U., Farias, M., & Brazil, I. A. (2018). The limited prosocial effects of meditation: A systematic review and meta-analysis. Scientific reports, 8(1), 2403.
Kreplin, Farias, & Brazil (2018). These researchers conducted a systematic review and meta-analysis on RCTs which assessed the prosocial effects of mindfulness, specifically compassion, empathy, aggression, connectedness, and prejudice. The researchers found that meditation interventions had an effect on compassion and empathy but that “compassion levels only increased under two conditions: when the teacher in the meditation intervention was a co-author in the published study; and when the study employed a passive (waiting list) control group but not an active one.”
Kreplin, Farias, & Brazil (2018) noted that the fact that compassion only increased when there was a passive control corroborated the findings of Goyal et al. (2014). They consider their second finding — that compassion only increased when the teacher was a co-author in the published study — to be novel. “At best this [finding] shows that a motivated meditation teacher will impact to a greater extent one’s students; at worse, it suggests that experimenter biases are introduced which affect the outcomes of the studies.” They found that authors ran interventions in 48 percent of the studies, and in these studies “unintentional experimenter biases could have been introduced by researchers/teachers with a personal interest in the intervention (e.g., by giving preferential treatment or being particularly enthusiastic to participants in the experimental group).”
They found just one study that assessed “beliefs about the efficacy and relevance of the intervention (meditation versus analytical training) and found that the meditation group had substantially higher expectations of a positive effect for the intervention, even though participants were not explicitly told that they were engaging in meditation.”
The authors discuss solutions to experimenter bias and expectation effects. First, measure participants’ expectations (e.g., the Credibility and Expectancy Questionnaire). Second, introducing blinding studies in these RCTs. They continue:
Then, the challenge is to find suitable interventions that can function as active control conditions. An interesting solution was developed by Smith, who developed a 71-page manual describing the rationale and benefits of a made-up meditation technique. He gave the manual to a research assistant, who was unaware that it was a placebo, and who then proceeded to give a lecture to participants in the control group about the merits of the technique (very much like in the experimental group that used Transcendental Meditation). When it came to the actual placebo technique, participants were instructed to sit quietly for 20 minutes twice per day in a dark room, and to think of anything they wanted…
Having a meditation teacher who knows nothing about the hypotheses of the study and has no part in designing, analysing and writing the results, would also reduce the likelihood of methodological biases.
The authors write that many studies are also compromised by confirmation bias:
Kaptchuk has summarised a number of potential interpretative biases that have become widespread in science reporting, including a confirmation bias, where one tends to evaluate evidence that supports one’s beliefs more favourably than evidence that challenges it.Kreplin writes:
Confirmation bias was particularly prevalent in the form of an over-reporting of marginally significant results. When using statistical testing, a p-value of 0.05 and below typically indicates that the results are statistically significant in psychological research. But it has become common practice to report results as ‘trends’ or as ‘marginally significant’ if they are close to, but don’t quite reach the desired 0.05 cut-off. The problem is that there is little consensus in psychology as to what might constitute ‘marginal significance’, which in our review ranged from p-values of 0.06 to 0.14 – hardly even marginal..
Being liberal with statistical methods that were designed to have clear cut-offs increases the chance of finding an effect when there is none. A further problem with the use of ‘marginal significance’ is reporting it free from bias. For instance, in one study the authors reported a marginally significant difference (p = 0.069) in favour of the meditation intervention relative to the control group. However, on the following page, when the authors reported a different set of results that did not favour the meditation group, they claimed the exact same p-level as non-significant. When the results confirmed their hypothesis, it was ‘significant’ – but only in that case. In fact, the majority of studies in our review discussed the marginally significant as equal to statistically significant.
* * * * *
Coyne shows the flaws of a meta-analysis conducted by the Benson-Henry Institute. Summary: the meta-analysis includes several very flawed research studies. Also: "The larger issue is that we should not rely on promoters of MSBR products to provide unbiased estimates of their efficacy. This issue recalls very similar problems in the evaluation of Triple P Parenting Programs. Evaluations in which promoters were involved produce markedly more positive results than from independent evaluations. Exposure by my colleagues and me led to over 50 corrections and corrigendum to articles that previously had no conflicts of interest. But the process did not occur without fierce resistance from those whose livelihood was being challenged."
* * * * *
Over-reporting positive results. Reporting of Positive Results in Randomized Controlled Trials of Mindfulness-Based Mental Health Interventions: "108 (87%) of 124 published trials reported ≥1 positive outcome in the abstract, and 109 (88%) concluded that mindfulness-based therapy was effective, 1.6 times greater than the expected number of positive trials based on effect size d = 0.55 (expected number positive trials = 65.7). Of 21 trial registrations, 13 (62%) remained unpublished 30 months post-trial completion. No trial registrations adequately specified a single primary outcome measure with time of assessment. None of 36 systematic reviews and meta-analyses concluded that effect estimates were overestimated due to reporting biases."
* * * * *
Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A., ... & Fox, K. C. (2018). Mind the hype: A critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science, 13(1), 36-61.
* * * * *
Individual Studies (Adults)
Sundquist, J., Lilja, Å., Palmér, K., Memon, A. A., Wang, X., Johansson, L. M., & Sundquist, K. (2015). Mindfulness group therapy in primary care patients with depression, anxiety and stress and adjustment disorders: randomised controlled trial. The British Journal of Psychiatry, 206(2), 128-135. Study claiming that mindfulness is effective as CBT at treating depression. Coyne debunks study.
Meditation vs. Sham Meditation: No difference at improving critical thinking skills. Noone, C., & Hogan, M. J. (2018). A randomised active-controlled trial to examine the effects of an online mindfulness intervention on executive control, critical thinking and key thinking dispositions in a university student sample. BMC psychology, 6(1), 13.
No comments:
Post a Comment