How To Spot A Scientific Study That Isn’t Terrible

John Oliver has weighed in on junk science, and it’s a must-watch. But behind the comedy, there are two serious questions: One, just how dangerous is a junk study? And two, how can you sort the scientific gold from the utter trash?

Junk science is more than just the fuel for dumb Facebook posts. In the wrong hands, it can kill. In 2004, Andrew Wakefield released a study that claimed autism was caused by vaccines. How could such a thing be possible? It wasn’t. Wakefield’s data was collected using at best questionable and at worst outright unethical methods, because lawyers saw a chance to sue vaccine manufacturers.

But the damage was done: In 2016, measles, a disease declared eradicated in the U.S. in 2000, is back with a vengeance. And all because people didn’t know how to read a study. So what should you be looking for?

Good Studies Don’t Make Grand Claims

When approaching any scientific study, especially ones you want to hear, it pays to take a step back and ask if this passes a gut check. We all want to hear that you can eat ice cream with abandon, that we don’t have to exercise, and that people who disagree with us politically are dimwits who can’t screw in a lightbulb, but that doesn’t mean any of those conclusions are accurate. This is doubly true if the study is being used by somebody with an agenda to plug; they’re unlikely to look closely, as long as the “science” tells them what they want to hear.

Wakefield is just the most obvious example. Anti-GMO advocates are still citing a study that claimed rats eating GMO corn got tumors, but conveniently forget to mention the study was withdrawn for a number of reasons, not the least of which was the researchers used a strain of rat bred to grow tumors in the first place.

Good Studies Aren’t Self-Reported