Health research is based on trust. Health professionals and journal editors reading the results of a clinical trial assume that the trial happened and that the results were honestly reported. But about 20% of the time, said Ben Mol, professor of obstetrics and gynaecology at Monash Health, they would be wrong. As I’ve been concerned about research fraud for 40 years, I wasn’t that surprised as many would be by this figure, but it led me to think that the time may have come to stop assuming that research actually happened and is honestly reported, and assume that the research is fraudulent until there is some evidence to support it having happened and been honestly reported. The Cochrane Collaboration, which purveys “trusted information,” has now taken a step in that direction.
[…]
We have now reached a point where those doing systematic reviews must start by assuming that a study is fraudulent until they can have some evidence to the contrary.
|
|||||
Samizdata quote of the dayJuly 27th, 2021 |
 24 comments to Samizdata quote of the day |
   |
||||
All content on this website (including text, photographs, audio files, and any other original works), unless otherwise noted, is licensed under a Creative Commons License. |
|||||
Is this blog in decline? It would be nice to see more on here than a daily quote. Most of us will have encountered the quote of the day already via our customary reading and subscriptions.
Then look elsewhere. We all have lives beyond this blog, its a hobby not a job.
Going on someone’s blog and complaining about the content marks you out as a knob. If you don’t like what you read don’t visit, if you think that you could do better start your own blog.
“Most of us will have encountered the quote of the day already via our customary reading and subscriptions.”
I will admit – horrors – that I don’t read BMJ every day. SQoTD is a valuable resource for me as the authorship here – peace be upon them all – have a much much wider reading list and possibly greater inclination to post about it than I do.
Keep up the good work.
Graham Asher, I couldn’t disagree more. I wish that I had the time to dedicate to reading sources of information as far and wide across the web as possible. Sadly, the real world gets in the way far too often. Fortunately, there is on this site, a home for me, a minarchist libertarian, where I can read the thought-provoking content and bask in a community of people whom I’ve never met yet know share a similar mindset to myself.
I have been reading Samizdata most days for the last ten years or so and, although I rarely comment, I fully appreciate the work of Perry et al. Thank you all ever so much.
Graham Asher, I agree with those who say that if you think you can do better, then start your own blog.
And, no, I had not heard of the topic under discussion, which I feel is vitally important and needs to be widely circulated. After all, we live in an age where “science” is being increasingly used as a justification for taking away our liberties.
@Pedant
What you said.
Samizdata is one of my few daily reads.
I have a day job, Graham Asher (July 27, 2021 at 10:34 am), and it sometimes has emergencies and urgent tasks well beyond its (in any case, not quite conventional) working hours – else there would be more posts from me.
One thing: if there happens to be a succession of SQotDs following one after another at a time of light other posting, it does make it a bit harder to follow comments in the right side-bar. If posting a SQotD while a prior one still has active commenting then, to help both that quote’s discussion and also your own quote’s discussion, consider varying the title a bit: if meaningful titles are elusive and/or you cannot think of even a sentence-worth of valuable commentary to include in your post to justify them, then Samizdata Quote of Tuesday, Samizdata Quote of Friday, Samizdata Quote of the Morning and suchlike could indicate who is discussing what when Samizdata Quote of the Day is already half the side-bar. Just a thought offered FWIW.
As regards the OP, the linked article notes certain countries (the one’s you’d expect) as disproportionately producing the fake and zombie studies. I can well believe it of those places, but while they may be obvious and easy to spot, certain Western disciplines had a bad reputation even back in the days when the word ‘woke’ was not around to (with reason) characterise them. For once, I find myself not thinking the high Egyptian and Iranian figures betoken a problem it’s irrational to seek hard for in westerners.
Samizdata has been a stop on my daily blog patrol since the (very) early oughties. I obviously feel compelled to return for some reason. There may not be something new every day, but when there is, I find it worth my while to read, and as Perry points out, Samizdatistas have lives outside the blog.
If he didn’t like it, he wouldn’t be asking for more, so it’s not all bad.
Sometimes saying “I can do better than that!” works. I know an author who wrote a very good mystery that way. Whether she did better than all is a matter of judgement, but she did better than many.
Ellen (July 27, 2021 at 5:11 pm), Baroness Orczy wrote ‘The Scarlet Pimpernel’ because a woman she knew (and didn’t think much of) got a book published, leading the Baroness to think, “If she can do it, I can do it – and do it better.” The manuscript was rejected by publisher after publisher (the foreign-born Baroness’ command of English and of plausibility were neither of them strongpoints). Eventually, it was seen by a publisher who had a particular method of judging books. He sent them to his uneducated granny who lived in a cottage in the New Forest – if she liked it, he published it, whatever his own opinions. And the rest, as they say, is history – quite profitable history for the Baroness at the time.
Getting back to the original topic, Richard Smith, of Cochrane, has a good track record in evidence-based medicine. It’s currently the “most read” article here on BMJ Blogs : https://blogs.bmj.com
Dare we do as Richard Smith suggests and assume this about some Covid research?
Could it really be that some Covid researchers are untrustworthy and might have venal interests at heart?
Why assume research is fraudulent?
Over the last 40 years medicine has advanced greatly in its ability to solve health problems.
Life expectancy has risen by many years.
For these reasons I assume that most medical research is not fraudulent.
There is already the aptly named Reproducibility Initiative which undertakes validation studies of significant papers, I believe their results are available on line.
http://validation.scienceexchange.com/#/reproducibility-initiative
I assume that most medical research is not fraudulent.
Results for a surprisingly large number of medical research papers cannot be replicated or reproduced which does not speak well for the quality or honesty of the work.
Peer review of Isaac Newton’s alchemy would not have found fault with it, peer reviews of Isaac Newton’s physics would not have understood it.
Nowadays, most ‘research’ I read about appears to have been built around finding an ‘answer’ to a proposition that was the basis of the grant application and which fits a narrative, e.g. (I exaggerate) a link between poverty and voting for Brexit.
“For these reasons I assume that most medical research is not fraudulent.”
First of all, Smith doesn’t say that most research is fraudulent. He says that Ben Moi’s threshold of 20% is the “point where those doing systematic reviews must start by assuming that a study is fraudulent until they can have some evidence to the contrary”. Which seems a little pessimistic to me, but not unfair. If one-in-five of everything that crossed your desk was a complete waste of time, you’d probably start checking it all more carefully too.
But secondly, even if it were 20% which isn’t fraudulent, that’s still a lot of good research. Major advancements could still be made on that basis.
The Lamborghini Miura was the result of Mr Lambo buying a Ferrari and thinking that it was a piece of crap. A lot of businesses in the US were set up after someone received lousy service and decided that they could do better. Over to you Mr. Asher.
Well, thanks for all your replies to my criticism. Some of you exhibit somewhat excessive irritability and sensitivity. It was only a mild criticism, and yes, I do usually like what I find here, and I have a day job too, and I sympathise with the difficulty of keeping a blog like this going. It is not a valid response to any criticism of any original work to say, “if you think you can do better, have a go”, unless you think that criticism of books, plays, and paintings should be abolished. And even less so to say that I am a knob; a term of abuse from someone hiding behind a pseudonym.
No, I couldn’t do better, but I believe that criticism has its place. I think it might be a good idea to remove the Quote of the Day feature, but it doesn’t really matter. I keep up with quite a lot of blogs, and some of the best have very few posts – one a month or fewer.
To reiterate: I am on your side. I am a free market, small government libertarian and was once (long ago) even a “crazed Ayn Rand worshipper”. More power to your elbows.
Sorry “Pete” – but it is quite clear for some years that a lot of scientific (including medical) studies are fraudulent. If you do not know that – then it is time you did.
For many years we have been told that the corruption of the humanities and “social sciences” did not matter – because the West really depended on the “hard subjects” such as a medicine.
The idea that the physical sciences could remain as some sort of island of sanity in a sea of madness was never very convincing – and it has totally fallen apart now.
The corruption of the culture has hit everything – from the study of history, to the study (and practice) of medicine.
Medicine used to be about curing individual patients – but that is not the philosophy of the World Health Organisation, or of “SAGE” in Britain, or of Dr Fauci and his associates in the United States.
Their agenda is about “Public Health” – about “Social Justice”. Individual patients are an annoying distraction to these “experts”.
Where did the cultural corruption start? It started in philosophy.
Which of the fashionable philosophers taught in the universities over the last century or so believed in objective moral truth, the ability of individuals to discover it and choose to follow it, and in the rights of the individual AGAINST the collective?
If you answered “none of them” – you would be close to the bitter truth.
And, eventually, the corruption of philosophy spreads to just about everything else.
There are exceptions – for example in early 20 century Oxford there were good teachers of philosophy, and other things that relate directly the life of the mind in relation to objective moral truth, they may not have been the majority – but they were there.
Such people as Tolkien and C.S. Lewis, and such philosophers as Harold Prichard and Sir William David Ross (Major Ross).
Sadly I suspect that the only job such men would get now is cleaning the toilets.
This was a good article up until: “Science really needs global governance” …
Agreed.