About half of studies of some types of brain stimulation cannot be reproduced. So, how do we know if these work? Credit: www.shutterstock.com
Interest
in electrical brain
stimulation has skyrocketed in recent years, both in the popular
media and scientific
literature.
Scientists
and clinicians are using the non-invasive and cheap technique to treat various neurological and
psychiatric disorders, including depression, epilepsy and addiction.
The US military is researching whether it improves
learning and attention. And those who train elite athletes can see its
potential to enhance performance.
But
our research
shows the evidence to back electrical brain stimulation varies in
quality, and the results are commonly not reproduced in other studies. Our
survey also unearthed the lengths to which some researchers go to to
present their findings in the best light.
What
is electrical brain stimulation?
The
type of electrical brain stimulation we studied is transcranial direct-current
stimulation. This is when a small electric current is applied to the brain for
20 to 30 minutes. Electrodes are placed on the patient's head, and some of the
current passes through the skull to the brain.
It
is thought this alters brain function mainly by inducing persistent changes in
the excitability of neurones.
This
isn't to be confused with electroconvulsive
therapy, which uses currents hundreds of times larger. This induces
a seizure.
What
we did
We
used an online survey to ask researchers if they could reproduce published
findings related to electrical brain stimulation. We invited all researchers
who served as corresponding authors on a published scientific paper on
electrical brain stimulation in humans to do so.
In
all, 976 researchers from all over the world were invited to answer the
question of whether they could reproduce published electrical brain stimulation
effects.
We
also asked whether researchers used, but didn't report, questionable research
practices in their own research – such as fiddling with statistics
to make them look more favourable and selectively reporting results. And we
asked if they thought other researchers used these questionable techniques, and
whether they should be reported in publications.
To
check what researchers actually do, we audited a random selection of 100
publications featuring research on electrical brain stimulation. We looked to
see if they admitted to the dodgy practices in their publications.
What
we found
For
the two most popular types of electrical brain stimulation (anodal and cathodal
stimulation), only 45 to 50% of researchers routinely reproduced published
findings.
Some
researchers were aware of others who handpicked which experimental conditions
(36%) and which results (41%) to publish. They also knew researchers who
manipulated results by excluding data based on a gut feeling (20%) and fiddling
with the statistics (43%).
As
expected, fewer researchers admitted to personally using these types of shady
research practices. Still, 25% admitted to adjusting statistical analysis to
optimise results – namely p-hacking, when researchers manipulate the statistics
to make results appear more statistically significant than they might otherwise
be.
Our
research also revealed the difference between whether these questionable types
of practices should be reported in research papers, and whether they are.
Although 92% of respondents said all researchers should admit to the
questionable practices in their publications, we found only two such admissions
(2%) in our audit of published studies.
So,
what do we make of this?
Meta-analyses,
which are studies that pool results from several other studies, indicate
electrical brain stimulation is effective in major depression. But it
isn't in fibromyalgia (where people experience widespread pain without a known
cause), food craving and overeating, Parkinson's disease, and speech problems
after a stroke.
Unfortunately,
a general finding is that electrical brain stimulation studies are often of low
quality and that, when present, therapeutic effects are often small. So, before
you decide to strap electrodes to your head, speak to an informed health
professional.
Poor
reproducibility and bad science are not unique
to electrical brain stimulation research. Nor are these problems new. But
public funds are being wasted on poorly conducted research that cannot be
reproduced, which means the results are questionable. Such poor research is
tarnishing the genuine efforts of researchers to improve human brain function.
The
main reason researchers engage in questionable researcher practices is the
continual pressure to publish scientific papers to gain funding or to progress
scientific careers. If results are statistically significant, researchers are more likely to
be published. So, researchers may consciously, or unconsciously,
resort to questionable or fraudulent research practices.
What
can we do about it?
Awareness
of bad science is on the rise – and recommendations and guidelines are
emerging to deal with this. But there needs to be more education and
true incentives for scientists to conduct better, reproducible science.
If
not, some scientists will continue to do as they have always done. Incentives
to improve the culture of research include promoting researchers who do more open science, and funding projects that adhere
to open science practices as well as those that attempt to replicate studies.
The
responsibility to improve the quality of our science lies with research
institutions and universities, funding agencies,
scientific publishers and individual researchers.
Our
goal of clinically useful brain
stimulation techniques is a worthy one. But our progress is limited by findings
of often variable and small effects currently reported, as well as the poor
quality of some of the studies that claim any effects at all.
https://medicalxpress.com/news/2017-04-brain-isnt.html
No comments:
Post a Comment