Eh… yes and no. I’ve got an engineering degree, I’ve learned how to design studies and do science properly, and I still struggle when a study is on topics I’m less familiar with. I can’t imagine most people going through these. They’re not accessible.
And if you’re just reading the abstract and conclusion, or worse a science article, you’ve got to hope they’ve interpreted things properly. Which articles are particularly bad at because they need to sound like news.
But then they still need to trust the journalist. And considering how much crap science gets published even in supposedly high quality journals, and how little quality peer review happens, even the journalists don’t have a scientific basis for much of science reporting.
Yes, that’s a huge issue. Another issue is that the reward for doing peer reviews is far too low, and publishing negative peer reviews comes with the risk of making an enemy in the same field, who might do your next peer review. So you only call out egregiously bad science or just rubber stamp every peer review, because there’s nothing in it for you to publish a negative peer review.
I’ve read meta studies that said that huge amounts of published scientific studies cannot be reproduced. I can’t remember the exact number, but it was >30%.
So if the published science itself is already full of garbage, how is a journalist (who is themselves not a scientist or at least not a scientist in the specific field) know what study is good and what is garbage? And even then, how many people read science journalism compared to boulevard media?
John Bohannon comes to mind, with his purposeful bogus study that claimed that eating chocolate can help with weight loss. He used overfitting and p-hacking to create a study that was purposely garbage and got it published. His goal was to show how easy it is to publish a sensationalist-but-garbage paper. This went so well that every trashy boulevard paper but also many major newspapers ran it, often as a title page news story.
In an interview he said that he got hundreds of calls, all on the level of “Which brand of chocolate helps best?”, and only a single serious inquiry doubting his methods.
He published his own debunk shortly after publishing the original story, it it got pretty much no media attention at all.
He basically couldn’t even recall his own bogus study, and to this day many people worldwide still believe that chocolate can help with weight loss.
Eh… yes and no. I’ve got an engineering degree, I’ve learned how to design studies and do science properly, and I still struggle when a study is on topics I’m less familiar with. I can’t imagine most people going through these. They’re not accessible.
And if you’re just reading the abstract and conclusion, or worse a science article, you’ve got to hope they’ve interpreted things properly. Which articles are particularly bad at because they need to sound like news.
Or they need a competent journalist to translate the findings without being sensational.
But then they still need to trust the journalist. And considering how much crap science gets published even in supposedly high quality journals, and how little quality peer review happens, even the journalists don’t have a scientific basis for much of science reporting.
Part of the problem is the “publish or die” mentality.
Personally, I think the Journal of Negative Results needs more love.
Yes, that’s a huge issue. Another issue is that the reward for doing peer reviews is far too low, and publishing negative peer reviews comes with the risk of making an enemy in the same field, who might do your next peer review. So you only call out egregiously bad science or just rubber stamp every peer review, because there’s nothing in it for you to publish a negative peer review.
I’ve read meta studies that said that huge amounts of published scientific studies cannot be reproduced. I can’t remember the exact number, but it was >30%.
So if the published science itself is already full of garbage, how is a journalist (who is themselves not a scientist or at least not a scientist in the specific field) know what study is good and what is garbage? And even then, how many people read science journalism compared to boulevard media?
John Bohannon comes to mind, with his purposeful bogus study that claimed that eating chocolate can help with weight loss. He used overfitting and p-hacking to create a study that was purposely garbage and got it published. His goal was to show how easy it is to publish a sensationalist-but-garbage paper. This went so well that every trashy boulevard paper but also many major newspapers ran it, often as a title page news story.
In an interview he said that he got hundreds of calls, all on the level of “Which brand of chocolate helps best?”, and only a single serious inquiry doubting his methods.
He published his own debunk shortly after publishing the original story, it it got pretty much no media attention at all.
He basically couldn’t even recall his own bogus study, and to this day many people worldwide still believe that chocolate can help with weight loss.