This forum is about wrong numbers in science, politics and the media. It respects good science and good English.
I thought the readers here might be interested in an article presently appearing on the BBC News' health pages about another study raising the alarm about the supposed dangers of hormone replacement therapy for post-menopausal women, a subject which has provided fertile ground for junk science researchers as has been remarked upon by our host on several occasions over the years.
The claims of the study are that HRT can shrink the frontal lobe of the brain (by an average of 2.37 cubic centimetres) and shrink the hippocampus (by an average of 0.10 cubic centimetres!). Going to the website of the journal which published the study, one can find a summary here:
What is interesting is that the first result (2.37 cubic centimetre reduction in frontal lobe) is based on a test at a claimed level of significance of p = 0.004 and the second is at a significance level of p = 0.05 (the standard for epidemiological junk science). More absurdly the article states "differences in total brain volume approached significance (p = 0.07)". The latter statement is a rather weaselly way of trying to suggest that a result which is clearly not significant is significant (if it only approaches significance, then it obviously does not reach significance). But why is p = 0.05 significant and yet p = 0.07 is not?
Seems then that they performed their tests under initially demanding levels of significance, and if they could not reject the null hypothesis, then they reduced the level of significance until they reached a level at which they may claim significance (which as we know, was the method of the EPA's infamous study into environmental tobacco smoke). This might then suggest that these methods are actually now available as subroutines of statistical analysis software packages.
The BBC claims that the brain scans were carried out on 1,400 women (that would be extraordinarily difficult). However, this is a trojan number--the number of women who participated in the HRT study. The paper summary merely says that "Brain MRI scans were obtained in a subset of 1,403 women [who participated in the HRT study]". No confidence intervals are given, so we cannot estimate the sample size, although it is probably considerably smaller than the 1,400 number, because MRI scans are expensive and time consuming. The actual number might be available in the full text, but I do not have a subscription, nor institutional access at the moment.
If you are not confused by the authors' wording, they actually admit that there is no statistically significant result suggesting an overall change in brain volume, thus the BBC headline summarizing the study as "HRT 'can shrink women's brains'" is a lie. The study's results, if we take them at face value, suggest a change in distribution of brain matter, but the distributions for those who took the HRT may be closer to normal for pre-menopausal women and the changes may be the result of the menopause and not the HRT (note that there were no brain scans performed before the HRT study).
"The claims of the study are that HRT can shrink the frontal lobe of the brain (by an average of 2.37 cubic centimetres) and shrink the hippocampus (by an average of 0.10 cubic centimetres!)."
Right, I will try this on my dear lady, and see if she accepts that in future she should defer more to my judgement. She has been on HRT for some years.
If I don't report back, the search parties should scour intensive care units.