February 18th, 2007
Today’s Independent on Sunday reports on a forthcoming paper to be published in the European Journal of Obstetrics and Gynaecology and Reproductive Biology. The paper by Italian and Dutch researchers is entitled ‘Women’s perceptions of sexuality around the menopause: outcomes of a European telephone survey’.
The Indy runs with the headline ‘Why British women go off sex (unlike the French and Germans)’ and claims “Britain’s middle-aged women don’t think an active sex life is important, but on the Continent they consider it essential”.
We can expect to see other news coverage tomorrow in the British press bemoaning the fate of British women and tut-tutting over their lack of effort in the bedroom department compared with their European sisters.
There are unfortunately many errors with the research covered in the Indy that should have been picked up on before going to press, but when you’ve only a press release and a pressing deadline to go on asking questions about research rigour tends to go out the window. As a result the angle of the piece suggests that genuine significant statistical differences were found between European women, which isn’t true at all.
I obtained a copy of the paper today and here are the main problems with it. If you are able to access the paper you may also want to read it though and see if you agree with my observations.
The paper has not yet been published
Nothing sinister in this but papers that are awaiting publication are usually only released by a journal to the press if the research is so groundbreaking the public need to hear about it now. With respect this paper does not fit this category.
Did the study breach ethical guidelines?
Ethical approval is required for any research on a health-related topic (including sexual functioning). In a cross-country study like this one even if one country does not have such stringent ethical controls as another it is important that ethical approval is sought from the countries where the research will be taking place in order to protect the well-being of participants.
This study involved a telephone survey of women across the UK, France, Germany, Italy, The Netherlands and Switzerland. In the UK at least if you want to ask people health-related questions our ethical approval system is increasingly likely to refuse research that involves ‘cold calling’ people to ask them sensitive questions. So it would be useful to know in the case of this current study whether any ethical approvals were sought to complete the research with UK women; who granted ethical approval; and if ethical approval was not given why the journal accepted the paper for publication? The same applies for the other countries where women were sampled.
Who funded the research?
Usually within research there is a mention somewhere in the paper about who funded the study. Declaring this information makes the reader aware whether the funding of the research may have affected its outcomes. If research is funded by a drug company or organisation with something to gain from a particular publication, or if the researchers stand to gain financially from publishing their data then this represents a conflict of interest and means the research could be flawed. That’s why journals ask you to declare on submission any conflict of interest you have – including who funded your work – and this is usually made transparent within a paper. In this case no such information is given which means we cannot work out who paid for this research or why it was conducted.
One of the authors of this research work for Organon, a pharmaceutical company who make prescription medicines for gynecology, fertility, neuroscience and anesthesia (according to their website). It is not clear whether the company funded this research or not, but it would be helpful for such information to be disclosed.
What was the point of the study anyway?
The paper states “The aim of the present survey was to generate broad findings on women’s perceptions and experience of sexuality around the menopause in six European countries”, which tell us what they wanted to do. It doesn’t tell the reader why they wanted to do it though. Why was this issue considered important? What was the purpose of the study – and the objectives guiding the research? Why were these six countries chosen, and what was the clinical need for carrying out the study? What would completing the study add to our existing knowledge in this area? None of these questions are answered anywhere within the paper.
Did they select the right method to study this topic?
The method selected for this study was a telephone structured interview (also described in the paper as a survey). There is no explanation for the reader about why a telephone interview was considered the most appropriate for female participants across six countries, who completed the telephone interviews, and how they were recorded. No mention is made of the language or languages used within the structured telephone interviews. If the interviews were completed in one language only this would have excluded large numbers of women who did not speak that language. If the structured interviews were translated into six different languages checks need to be made on the accuracy of translated surveys to ensure all participants were being asked the same questions – and answering appropriately.
Usually in a survey you check these issues of validity and reliability but in this paper no mention is made of how these factors were monitored.
Within the paper it claims ‘those who reported having sex five times or more per month had a higher satisfaction score (7.4 on a visual analogue scale)’. Now that all may be gobbledegook to you, so let me explain. A visual analogue scale is a line scale participants are shown. In this case it would be something like
‘how satisfied are you with your sex life?’
1 Not satisfied at all_____________________________10 very satisfied
You would put a cross on the part of the line that best matched your answer.
Okay, now you tell me how you make that work when the survey method is a telephone survey? The participant can’t see the visual analogue scale because they’re at the other end of the phone line?
Of course you could ask someone to say out of ten how satisfied they are (which would be an acceptable measure), but this seems to be the case of using the wrong measure for the method. And again I’m wondering how the journal and its reviewers missed this?
How were the women picked for this research?
Usually in an academic paper you will find a table or chart that indicates the recruitment of people into the research and describes participants characteristics. This means you can see how many people were approached overall to be in a study, how many were unsuitable for the research or refused to participate (and why), and of those who participated what were they like? In this research we’re told that around 300 women aged 50-60 were selected from each of the six countries for telephone interview but we were not told any more about them. In order to understand the data we need to know what was the ethnicity, sexuality, relationship status and social class of the women selected for study? All of these factors will have a bearing on results and it is very worrying that this data is simply absent from the research paper.
Without this information we cannot be sure whether the women used in the research were representative of all women within the six countries studied – and whether it is safe to generalise from their answers to a wider population.
The paper explains the women selected for the study were menopausal and experiencing at least one menopausal symptom (e.g. hot flushes or sleeplessness).
This means the women selected for the study were a specific group. They had a telephone in their home, were prepared to talk to an unknown person over the phone about their sex lives and menopausal symptoms, and also had to have menopausal symptoms to be eligible to be in the study.
Did the way they asked questions affect their results?
The survey/structured telephone interview used in the study is not reproduced within the paper so we are unable to tell what specific questions were asked of participants. However the paper does indicate women were asked to say ‘are you currently experiencing of have you experienced any of the following in the past year?’
No periods, hot flushes, sleeplessness, irritability, mood swings, reduced sex drive, headaches or migraines, depression, involuntary loss of urine, vaginal pain or dryness, very occasional periods.
Now if you ask people if they’ve experienced negative symptoms a fair number of them are going to say yes. The more negative things you ask them about the more negatively primed they become (so the more likely they are to say they’ve experienced negative symptoms). Women could also have experienced increased sex drive, more energy, greater confidence, more powerful orgasms, a sense of freedom. However in this research it was assumed that the menopause was a. medical and b. negative so women were only asked about these factors.
Moreover the paper shows that sex was presented to participants in the context of being in a relationship with a partner or husband. That made it difficult for those who were single, divorced or widowed to answer. It also constructed ‘sex’ as intercourse. So women were not asked about masturbation on their own, oral sex, or other pleasurable activities. It could be that women were enjoying a great sex life on their own or enjoying non-penetrative activities, but this simply wasn’t included in the study itself and therefore is missing in the data.
The order in which questions were asked might have affected answers
It is not clear from the paper what order questions were asked, but it seems after the questions about negative health problems women were asked to talk about their sex lives. In particular they were asked to agree or disagree with statements like
“It is very important to me that I have a satisfying sex life
Sex is an important part of my relationship with my partner or husband
It is very important to me that I maintain an active sex life”
Now these questions automatically exclude single women who may have been recorded as disagreeing with a statement that perhaps just did not apply to them. There are also issues of phrasing within questions since what does an ‘active’ sex life mean? The term will certainly be interpreted in different ways by different women.
Participants had already been primed to talk about health problems that would affect the way they described their sex lives. If you’ve just been reminded how bad your health is you’re hardly going to feel able or willing to disclose you’ve the best sex life in Europe.
Was the data analysis carried out correctly?
The paper states “Data were mainly expressed descriptively as percentages or mean scores. Spearman correlation analysis was used for symptom correlation and linear regression analysis was used for calculation of the regression coefficient and corresponding p-values”. Which sounds very impressive until you notice that no further mention of Spearman correlations, p-values or regression analysis appears in the paper. Claims are made of differences between women in different countries – but no statistical analysis appears to have been conducted on them – only a list of raw data, mean scores or percentages are given.
It is incorrect to make claims of cross-country differences without backing this up with statistical analysis (which quite clearly is required here). The results section is a list of percentages and bar charts, which is okay as a start but worrying that a journal accepted the paper with no additional statistical analysis to back it up.
You need the statistical analysis to explain, for example whether women with more health symptoms reported more problematic sex drives, or whether women who reported wanting an active sex life also believed in the importance of a satisfying sex life.
Findings appear to have been manipulated
Given the participants in this study were all women reporting negative menopausal symptoms one might expect they were all having a terrible time of it in their sex lives. But the curious part of this research is that 2/3 of respondents reported their sex drives were the same as (or perhaps better) than prior to the menopause. The paper does not report this though, instead focusing on the 1/3 of women who said their sex drive was reduced. The authors also draw attention to the 53% of participants who stated they were currently less interested in sex, again negatively priming the focus of the paper and ignoring the fact that the other half of respondents (give or take a few %) clearly were not less interested in sex.
Moreover the researchers consistently claim women ‘experienced’ low sexual desire or other problems. But a survey can’t really tell you this. What they measured was women reported low sexual desire and other problems – which is not quite the same as ‘experiencing’ them.
Sex research often constructs the way we see sex in very subtle ways. In this case the paper clearly sets out to show that the menopause is negative and has negative effects on women’s sex lives, although the data does not fully support this.
Why do we need to be concerned?
As the research is now being launched to the general public they too are going to be led to believe the menopause does bad things to women’s sex drives – and that some women – particularly those in the UK – are more prone to having a bad time of it than others.
All this is worrying since this is a paper that, were I sent it to review for a journal I’d reject for publication for the reasons outlined above. If the paper gets more press coverage it is no doubt going to encourage more worries amongst women and spawn countless women’s magazine articles comparing how women in different countries have ‘better’ or ‘worse’ sex lives. It will further play into the hands of drug companies eager to medicalise the menopause and women’s lack of desire. And it will make women feel they need to compete against each other in some kind of European sex competition.
The questions to be asked here are why this paper was accepted for publication, why was it press released, and why won’t journalists ask the right questions about research before giving publicity to a study that perhaps is not as accurate as it first seems?Tweet