Excellent summary. I always say the most important part of a paper is the methods section, cuz like many things, it’s garbage in/garbage out.
Like you, I care much more about ARR and NNT, and basically don’t care about RRR.
The only thing I would add, when trying to decide whether X deserves being incorporated into practice, is to make a distinction btw statistically significant effect and clinically relevant effect.
James, the comparator is the section I linger over too, because it is where a trial quietly states what it takes ordinary care to be. A weak control arm does more than flatter the intervention. It redraws the moral baseline of the paper, so the patient in the comparator group becomes the study's stand-in for the patient we would have treated anyway, only treated worse.
That is what makes the methods more than housekeeping. Reading them is how you ask whether the trial respected the clinical world it is requesting permission to change. And when a new intervention beats a diminished version of care, the result stops being about the intervention. It becomes a measure of how far the baseline had to be lowered to let it win.
Wonderful summary. I appreciate how you covered all the bases but stressed key points that aren't emphasized enough in guides like this (e.g., pay attention to comparators, and review the supplemental data).
Your focus was on RCTs where the outcome variable is event/no event. I routinely see spin for less simple outcomes. If the experimental group "improves", the improvement may not be clinically meaningful. Apart from that, improvement is often expressed in aggregate terms (e.g., group means) that can mask individual and subgroup differences.
Thank you and I agree - the group means issue is so important. I love the graphs that shows individual participant responses or percent reaching a certain thresholds for studies of continuous variable outcomes, like weight loss, BP change, pain improvement, etc.
I second that!!
Excellent summary. I always say the most important part of a paper is the methods section, cuz like many things, it’s garbage in/garbage out.
Like you, I care much more about ARR and NNT, and basically don’t care about RRR.
The only thing I would add, when trying to decide whether X deserves being incorporated into practice, is to make a distinction btw statistically significant effect and clinically relevant effect.
James, the comparator is the section I linger over too, because it is where a trial quietly states what it takes ordinary care to be. A weak control arm does more than flatter the intervention. It redraws the moral baseline of the paper, so the patient in the comparator group becomes the study's stand-in for the patient we would have treated anyway, only treated worse.
That is what makes the methods more than housekeeping. Reading them is how you ask whether the trial respected the clinical world it is requesting permission to change. And when a new intervention beats a diminished version of care, the result stops being about the intervention. It becomes a measure of how far the baseline had to be lowered to let it win.
Thank you
Wonderful summary. I appreciate how you covered all the bases but stressed key points that aren't emphasized enough in guides like this (e.g., pay attention to comparators, and review the supplemental data).
Your focus was on RCTs where the outcome variable is event/no event. I routinely see spin for less simple outcomes. If the experimental group "improves", the improvement may not be clinically meaningful. Apart from that, improvement is often expressed in aggregate terms (e.g., group means) that can mask individual and subgroup differences.
Thank you and I agree - the group means issue is so important. I love the graphs that shows individual participant responses or percent reaching a certain thresholds for studies of continuous variable outcomes, like weight loss, BP change, pain improvement, etc.
Nice summary of effective literature review.
I’ve come to love your Sunday articles. Great read. Wish you’d been my attending in residency! Thanks for writing, and please keep going!
Wow! Thank you!