April 1st, 2005
Perhaps not surprisingly, lots of people were upset by the attention generated by Bliss magazine’s latest ‘sex survey’. Groups including the Family Planning Association and Youth concern, alongside experienced sex researchers, argued the ‘study’ was biased and flawed.
But did this deter Bliss’s editor? Not one bit. In an interview with Press Gazette, they displayed a worrying amount of ignorance around the method of surveys and sex research generally, again calling into question the appropriateness of the Bliss ‘sex survey’.
The Press Gazette interview criticised existing, reputable research for not including under 16s, missing the point that there is research on this age group, but it can only be carried out by highly trained researchers operating under full ethical approval. Just because you write a magazine for teens, doesn’t automatically provide you with the skills to ask them sensitive personal questions, nor report that data accurately. Nor does it give you the right to run research – particularly if it could exploit your readers or only serves the purpose of promoting your publication.
The editor said ‘it’s very hard to get data on teenagers and we are the only ones asking questions in this age group’. Presumably then, they hadn’t noticed at least three other teen magazines having their ‘surveys’ covered in the press recently. Wasn’t aware that most charities dealing with teens regularly survey their clients. Nor been aware of the multitude of studies on teenagers published in an array of medical and social science journals.
The editor continues to say ‘we had 2,000 girls spread across the UK, which we thought was a pretty representative sample’. Statistics and sampling aren’t a matter of guesswork, any reputable researcher would know, not think that their sample was representative, and furthermore be able to specify the nature of their sampling techniques.
They then state ‘a survey done online and through a magazine is very reliable’. Again, as someone who’s set herself up as an expert on survey design, she may have thought twice before using the term ‘reliable’. A well-designed and thoroughly piloted online survey is likely to generate reliable data. One done through a magazine without rigorous controls isn’t going to do this. And of course magazine surveys only represent subsets of their readers, so cannot be representative of our population as a whole. If it were the case that online magazine surveys were so good, all the current health and social research would immediately cease and would simply run through magazines. You have to ask yourself why that’s not happening.
The comments made by the Bliss editor imply that they were doing a ground breaking and fantastic teen sex survey. If so, why weren’t they doing what all other reputable researchers do? Using existing measures, consulting with expert researchers, and drawing on reliable evidence on this topic (of which there is plenty).
Good researchers don’t reinvent the wheel. In order to generate comparable data, they frequently use existing questionnaires in their research. If the Bliss survey and other magazine ones like it were so good, they’d be being published and quoted in academic journals so all health and social researchers/practitioners could use them. They never are.
Most scary is the editor’s comment ‘the people in this age group wouldn’t be telling us the truth if we were stopping them in the street [to do a survey]’. You’re damned right they wouldn’t. Which is why sex research is never conducted in this manner.
Only those who know little about a method would make such comments. So can we therefore trust their belief that they’re the only people conducting useful research on teens? I don’t think so.Tweet