There’s been a substantial amount of media attention today for a survey by BBC Radio One’s Newsbeat and the Portman Clinic. Suggesting porn ‘use’ among men is endemic and in many cases problematic.

It’s led to a slew of scary headlines including:

Men view two hours of porn a week – The Sun
Young men worried about the amount of porn they watch – Mirror
Men ‘worried’ about heavy online porn use

What did this survey cover?
This survey heard from 1057 18-24 year old women and men (no information available about how many males and females made up the final sample). They completed an online survey via TNS Market Research Company between March 18-21 2011. It asked about their porn consumption and attitudes to porn and relationships.

What were the main findings?
8/10 men and 1/3 women had looked at porn online
The most popular place to access porn was free websites
The ‘average’ man in study (no figures given for this) looks at porn for 2 hours a week, the average woman around 15 minutes
4% of male respondents ‘used’ adult sites for more than 10 hours per week – these were reported as having a ‘problematic and potentially compulsive’ condition
¼ men said they were worried about the amount of time spent looking at porn
¼ men said they were worried about the content of porn
61% of respondents (gender unspecified) said porn could make you less interested in sex with a partner

Should we be concerned about these findings? Not until we’ve looked more closely at this survey.

Problems with the survey
It is not clear exactly how participants were recruited/heard about this research. Market Research Company TNS administered the online survey but it’s unclear who responded, the characteristics of those who didn’t respond, and how representative of the wider population respondents were. The survey was only run over a three day period which is not unusual in commercial online surveys, but doesn’t always allow for generating reliable data – particularly about sensitive issues.

The full list of survey questions have not been made publicly available. Without these you can’t work out what people were asked. Whether questions were leading, potentially distressing, counteracted/contradicted each other, were confusing or didn’t match the final data reported.

I’ve spoken to nine journalists from different broadcast outlets about this survey over the past two days. All have called me for comment about the survey. All have taken it at face value. None had seen the original survey questions. None had asked to see them either.

This isn’t something unique to this Radio One survey however. Most commercial and academic surveys don’t make their original survey questions available when press releasing their work. It remains a problem as you cannot make any judgements about how useful a survey is without seeing what people were asked. This survey may have been amazingly well designed, carefully piloted, developed and run. Or it may not have been so stringently managed. The problem is without making both the methodology behind the research and the survey questions public it is impossible to know whether to trust in the data or not.

One question was reported in the Radio One report of the study. It asked ‘How have you ever looked at porn?’ It then gave respondents the choice of magazines, DVDs, TV, Free and Pay websites, mobile phones and file sharing.

The problem with questions phrased like this is they may look sensible at first glance, but actually are highly problematic and potentially meaningless.

If we take ‘ever looked at’ first, what does that mean? Ever looked at as in sat down, watched a lot and masturbated yourself stupid? Or ever looked at as in was shown by some mates for a laugh, or ever looked at as in accidentally found when looking for something else online or on TV?

‘Ever looked at’ seems to be taken here to mean regular porn use, or at least some porn use. Whereas it doesn’t tell us anything about the context people are viewing porn – or the amount. I may have ‘ever looked at’ porn once ten years ago and never again. But this survey would put me in the same category as someone whose ‘ever looked at’ definition included looking at porn several times a week over months or years.

The lack of clarity in the question also doesn’t explain whether people are looking at porn alone, with a partner, when single or a relationship. Nor address whether their porn ‘use’ is consistent over time or varies depending on their relationship status or other life events.

Also what does ‘porn’ mean in this survey question? Is it an erotic scene in a movie shown on mainstream TV? Tuning into Babestation? Reading Nuts, Zoo or Cosmo? Visiting YouPorn? This survey asked respondents to state the format in which they access porn, but they are not asked to explain what they mean by porn. That means we’ve no idea what it is they’re looking at.

This is a classic problem in poor survey design. That terms used aren’t operationalised. Meaning neither the respondents or the researchers actually have any idea what it is they’re asking about, or what the data means when it comes back.

‘Heavy’ porn use was described in this survey report as 10 hours or more. But it is not clear how this was decided upon as a measure of ‘heavy’ use.

¼ male participants stated they were worried about the amount of time they spent looking at porn. But since we don’t know what they were asked we can’t tell if this was based on a leading question, or whether they would have said they were worried if not prompted. It doesn’t tell us what exactly they were worried about (i.e. did they think they were spending too much time in general online, or had a specific concern re porn use).

¼ respondents also said they were worried about what they were looking at, but we can’t tell from this whether this meant they’d already stopped looking at the problematic porn. ‘Worried’ by content doesn’t tell us whether they had issues over the wellbeing of actors in porn; a person’s sexual prowess in comparison to what they were viewing; or whether their gender, sexuality or relationship was being challenged by the things that they found a turn on. In short we may know a fair number of participants expressed concern, but we’ve no real clue about the source or magnitude of that worry.

Usually with media surveys of this kind the standard approach is back of the envelope question design, based around a predetermined angle that’s being picked to generate publicity. Radio One aren’t unique in this approach (although they do have form for running similarly questionable sex surveys in the past). It is likely this current survey is more about generating publicity for Radio One and Newsbeat than really addressing issues about our relationship with sexually explicit materials. It seems the main purpose of the survey was both to inform a report for Newsbeat and draw attention to said programme.

Unusually this research has teamed a media outlet (Radio One) with clinicians from an NHS trust. Specifically practitioners from the Portman and Tavistock NHS Trust who apparently designed the survey.

This raises more complex issues than just the usual shoddy media approach to surveys. If NHS staff are involved in research they ought to be grounding this within a critical appraisal of the published evidence in this area. They also ought to ensure they employ research tools correctly (in this case completing a quality survey, piloting and reviewing it – or better still basing it on an existing measure). And if they’re doing the work on NHS staff or patients or in their role as an NHS clinician then it ought to be subject to ethical approval.

Not all health research is subject to peer review, but usually if a survey is going to be made public – either through publication in a journal or through a report launch or similar – then you would expect a thorough peer review of the work before it hit the headlines. Peer review here could have helped spot potential problems in both survey design and delivery, and ensure the relevant background literature (of which there is a lot in this area) had been carefully consulted. It would also have guarded against making claims that go beyond the data.

This may have happened in the case of this survey but that information has not been made transparent. If that is the case it is a pity since it doesn’t put the research in context nor show it in a good light. If these checks and balances hadn’t been put in place again transparency might help us assess whether this work is reliable or not.

Given the example of the questions asked of participants along with conclusions drawn it does raise serious questions about the quality of the survey. Along with the ethics of using such a tool to draw clinical conclusions – as has happened around the diagnosing of those with ‘heavy’ porn use.

Journalists I spoke with about this survey felt the inclusion of medics in the survey meant it was more trustworthy. One said to me that because a doctor had designed the survey it had to be good. As someone who’s been teaching medics internationally how to evaluated, design and use surveys in health research for the past fifteen years; I can confidently report most are pretty dreadful at this craft. Indeed, as with any other social research skill it takes time to learn how to create a quality questionnaire. Without disrespecting the skills of the medics in this survey, you cannot conclude because a doctor helped put the survey together that it’s automatically accurate.

There are in fact many people actively studying pornography – some of whom can be found here. All of whom Radio One could have teamed up with had they genuinely wanted to explore our relationship with porn. This IS an important area that is worth studying and is currently hampered by a lot of poorly conducted academic studies, mostly based on undergraduate student samples. So we need to have more quality work conducted and there are plenty of people Radio One should have involved. The fact that some of the leading names in this area weren’t even mentioned or consulted points to either a lack of awareness of the subject area. Or deliberately avoiding to engage with those who may not be fitting with a predetermined porn/shock angle. Radio One should have been clearer about why did they pick this particular team from the Portman to collaborate with? Were there any agendas there? Certainly given the worries over medicalising our sexual behaviour one might argue it could be in a clinic’s interest to set up a figure of ¼ men being anxious about porn use. After all if you’ve identified a problem you are in a very good position to offer a cure.

Finally there’s no real context about why Radio One thought this was a major area worth studying. Given it’s a youth channel it could have easily picked up on numerous other social issues – poverty, employment, education, university fees. Or if they wanted to be sex campaigners to perhaps look at the issue around how sex education is haphazardly (and often poorly) delivered in schools. Are all young people really worried about online porn, or are there other things directly affecting their lives that may be more relevant (but perhaps less publicity generating?).

Despite all the percentages shown in this report and warnings about ‘heavy’ porn use there seems to be no take home message for people about what this means. If you suspect, after reading about/hearing Newsbeat’s survey, that you have a ‘problem’ – what should you do? If you have questions about porn, where could you ask them? In this case the audience are given some dire warnings but no referrals to sources of help or advice. Which again makes the overall purpose of this research unclear.

I may be being very disingenuous about the researchers from the Portman. But it worries me that their relationship with this research has not been fully declared. No doubt they acted with good intentions, and obviously drew upon their experiences as psychotherapists (one author has written about internet porn and psychoanalysis). But it is concerning that journalists covering this story didn’t think to ask more about who was doing the research, and what their motivations might have been.

Which brings us on to…

Problems with media coverage
I’ve already mentioned how the journalists I’ve talked to hadn’t seen the survey nor saw this as important. What was more worrying was the unquestioning pickup of this story. Journalists weren’t asking what was asked in the survey, of whom and why. What they were doing was accepting the figures from the survey – particularly the amount of time men were spending looking at porn, and how they were worried about porn use.

They were then using this as a jumping off point for other discussions. For example to talk about porn addiction, how pornography use could change men’s brains, what women felt about men who used porn. As a psychologist they wanted me to come and talk about neurological changes from porn use, and as a female psychologist they wanted an insight ‘as a woman’ to talk about how all women felt about their men’s porn use (more on this later).

What journalists didn’t want, was me talking about this survey from the perspective of someone who teaches survey design. They didn’t want the survey critiqued because their editor/producer had already fixed it as their angle. Indeed they’d already accepted it as ‘true’. Most worryingly those from the BBC seemed least keen to critique the research. One researcher calling from BBC Radio 5Live Drive Time confirmed they didn’t want to question the survey because it was done by Radio One – and as such they couldn’t challenge the output from a sister station.

It really does beg the question what is going on with the BBC College of Journalism that such problematic research can be put together to begin with – and how it can continually be endorsed even when flaws are being pointed out. (Sadly I’ve tried previously to help the BBC College of Journalism to think more critically about sex research but have got nowhere. I have alerted them to the current Radio One survey and uncritical coverage on twitter today. Let’s hope they decide to take this further) .

This case has been a useful example in seeing how journalists lack the understanding to critique research, but also the practical reasons they’re unable to do so. Moreover it gives us an ….

Insight into journalists views of sex/pornography
If you look at the press coverage of this survey (alongside reflecting on the discussions I had with journalists today) some very definite patterns of how journalists/the media see sex/relationships and porn.

The view from medialand is as follows:

Who looks at porn? Well, it’s men. They are all straight and the porn they are seeking out is also heterosexual. Women are constructed as having problems/concerns about pornography – but only in relation to their (male) partner’s use of it. ‘Pornography’ as a term is used to mean one genre from one format (the internet). Looking at mainstream porn in moderation is okay, but if you do it often then it becomes a problem. Quite often described in the medicalised language of addiction.

Men are naturally sexual and so can’t help liking porn, but if they do look at it they’ll become abusers or change their neurological makeup or sexual behaviour. Women don’t like porn, those who do are presented as being in a minority, probably deluded, or liking romantic/couples-based/equality-based/feminist porn. Porn within relationships is only permissible if it’s to spice things up (or encourage reluctant wives to get in the mood). LGBT folk aren’t even thought about.

Alongside this is a widespread acceptance that porn influences sexual behaviour (i.e. shaving pubic hair, trying anal sex, different positions etc). There is no insight from journalists or often the public that these issues all have – and continue to be – widely covered in mainstream media, not just porn.

If you’re starting from this as your standard position it makes thinking critically about pornography difficult. It means journalists will be tasked (or choose) to find evidence to stack up this world view. It also means it’s risky to find other ways to think about/explore porn for fear of being seen to endorse it.

Why is coverage of porn research so poor?
Coverage within the media tends to be poor because journalists are always on tight deadlines, lack skills to evaluate both research and critically reflect on their own assumptions about ‘normal’ sex. Finding experts who can talk clearly on the topic or searching through evidence is often difficult. And not helped when the standard way to approach this issue is in a false debate format where you have to find a pro and anti porn person in the mistaken belief there are clearly defined ‘sides’ in this area.

The pressure from editors to fit particular (and usually sex negative) agendas leaves little autonomy for journalists to tackle problems they may spot with research. Freelancers in particular may struggle as if they don’t write what the editor wants they won’t get a commission. Staffers may struggle with bullying if they don’t deliver an angle to deadline.

Journalists are not impartial. Many lack basic sex education or have particularly sex negative views. It is threatening for them to have this questioned – especially when on deadline. And if something seems intuitive then it won’t be questioned. Indeed if you try you’re more likely to be seen as a crank. Or unhelpful.

Part of our problem is there is a lack of funding to really investigate porn use in sensitive and open ways. Where quality research is undertaken in this area it’s often more nuanced and doesn’t lend itself well to scary headlines and battle of the sexes style debates. This means quick and dirty studies get completed and hit the headlines far more readily, and inform public opinion more often. That can be dangerous and misleading, particularly when behaviours that may not be a problem become medicalised.

What can be done about this issue?
Offer better training for journalists to understand surveys and other social methods. You might find these guidance papers I’ve written on the topic helpful:
Administering, analysing and reporting your questionnaire
Selecting, designing and developing your questionnaire

Reaching beyond the white middle classes

Make editors and broadcasters more accountable – challenging them when they run poor research or give bad science uncritical coverage.

Require media outlets that create surveys need to be completely transparent about the work they’ve done (as we equally should with academic researchers).

Encourage the public to engage in discussing and critiquing sex research from media outlets via social media. Today we saw two things happen. The media lost the news, while the public via social media (particularly on twitter) got to the bottom of a problematic story and had a much more interesting time doing it than if they’d relied on old media coverage alone.

Unless you’ve been living on another planet for the past week or so you can’t have escaped the news of Tiger Woods’ infidelities. The media have been thrilled to report on every new salacious revelation and fuel speculation over Mr Woods’ career and marriage. In some cases abetted by a minority of therapists and psychologists.

I can’t comment on Mr Woods. Here’s why. If I know a celebrity then I’d be breaching their confidentiality by talking about them to the media. And if I don’t know a celebrity personally all I’d be doing in discussing their personal life with the media is gossiping. That’s not me being prissy, it’s the basic ethical guidance set out for psychologists to follow.

In cases where celebrities run into trouble it’s logical journalists will want to cover the story. And the public may be interested either voyeuristically or because they’re also experiencing problems and hope they might learn how to deal with the situation by following what’s written about the celeb in question.

It’s not unreasonable for experts to play a role in such coverage. For example in this case it would be possible to talk about the wider issues of what cheating involves, why cheating occurs, the impact on children or wider family members, and how couples can deal with such problems within their relationships. Ethically you can discuss general issues. What you can’t do is talk specifically about the celebrity and speculate on their life.

Unfortunately many journalists aren’t satisfied with this. They want the ‘personal touch’ (a diagnosis of the celeb in question). And to help they’re happy to provide you with film clips and photographs to analyse or give you statements from the celeb or those close to them for you to diagnose. You’ll find with such celebrity cases that you’re not only pushed into talking directly about the person and their family, you’re often put in a context where the celebrity is ridiculed or blamed. Perhaps unsurprisingly many ethical practitioners avoid any discussions of celebrities within the media simply because there is little or no hope of discussing wider issues, but an almost certain guarantee you’d be asked to judge celebrities directly.

And as we’ve seen so many times, if you won’t do this then there are plenty of therapists, psychologists, medics and other self proclaimed ‘experts’ who will be more than willing to do so. Meaning journalists are confused over ethical standards but don’t need to respect them since they’re not consistently adhered to.

In the case of Tiger Woods I gather from journalist colleagues in the US a minority of therapists began badgering the media as soon as the story broke. So this wasn’t even a case of journalists deciding to focus on the ‘sex addict’ angle. It was being suggested to them by professionals not only willing to diagnose someone via media, but also speculate on their personal life. Both unethical practices, but both very popular in celebrity crisis stories.

Behind the scenes discussions I’ve observed between reputable therapists and sex educators have indicated dismay over the willingness for a minority of practitioners to promote the ‘sex addiction’ tag for Mr Woods.

Aside from the ethical issues over speculation and confidentiality outlined above, the main reason for this concern is down to the very diagnosis of ‘sex addiction’. While the idea of sex addiction (when it’s not being sniggered about) is becoming increasingly popular, it doesn’t mean to say there is an accepted evidence base for it. Indeed, while people can act in sexually compulsive, risky, thoughtless, cruel, controlling and abusive ways, that’s not the same as saying they are ‘addicted’ to sex.

Critics of the ‘sex addiction’ model argue it medicalises behaviour that may be unpleasant or upsetting to partners in a relationship, but does not mean what is happening is a sickness. They complain that making problematic sexual behaviour into a disease-based model actually permits people to take less responsibility or see sex as something they can’t or won’t control. Given that many advocates of the sex addiction movement are often working from right wing and faith based perspectives it’s perhaps unsurprising that they classify masturbation, watching porn or having open relationships under their ‘addiction umbrella’.

If you’re interested in reading around this area I’d recommend starting with Marty Klein, an uncompromising and often challenging sex therapist who nevertheless outlines very clearly why the sex addiction approach (and related industry) is problematic for our relationships. Klein does not excuse negative behaviour in relationships, but argues that by focusing on sex as addiction we may miss key problems and not provide adequate therapeutic interventions for people who need it (although we may well provide inadequate and untested therapeutic interventions for people who don’t). Picking up on some of the issues Klein raises, Leonore Tiefer in conversation with Susie Bright also identify why the addiction model is problematic and why people are so anxious about it. You can listen to this podcast via Susie’s blog ‘sex addiction the big con’.

Sex addiction is a field which does have some evidence behind it, although the quality of said evidence is questionable and there is a major problem with many of the diagnostic tools used to diagnose sex addiction. Which is how I discovered that I was a sex addict – as did several of my colleagues including Cory Silverberg and Susie Bright (who also reports in this blog on her experiences of debating one of the sex addiction advocates within the media).

More worryingly is many ‘sex addiction therapists’ are not necessarily therapists at all and the interventions they endorse are usually homespun ideas based on flimsy evidence and no independent evaluations. This is of particular concern given many of these practitioners target therapists, physicians, faith and youth leaders to ‘train’ them in the homespun therapies or how to spot a sex addict. Although describing themselves as qualified because of being sex addicts themselves, many practitioners have little or no formal qualifications and lack supervision in the work they do. They are careless over ethical standards (evidenced in their eagerness to speculate on celebrities in the media among other things) and either unaware of or unable to understand the wider evidence base about sex and relationships problems.

It is worth noting that most reputable professionals do not endorse the idea of sex addiction, although they are concerned about relationship problems. If, however, there are an aggressive group of practitioners promoting this view – and they’re as eager to work with the media as the media are to hear from them – then it is very difficult to challenge.

For example, yesterday a journalist who called and asked me to talk about Tiger Woods sex addiction responded to my point that in general we don’t recognise the concept in the UK by saying ‘ordinarily I’d agree with you but the therapist whose diagnosed Tiger has advised the government on sex addiction’.

That journalist wasn’t aware of the debates on sex addiction, nor the lack of an evidence base for it, nor the concerns over the ethical practice of those promoting it. But they were impressed that the therapist they’d heard about enhanced their expertise by claiming they advised the government on the issue. This would be the Bush administration which promoted abstinence based sex education, withheld funding for sex positive education, restricted the availability of over the counter emergency contraception (despite the evidence base saying this was a good idea), and was not known for its sex positive approach. Of course such an administration would also welcome a neo Conservative therapy that was so gloomy and judgemental about sexual behaviour – and advocated sexual control.

Journalists need to be very careful when covering stories like this. It’s not good enough to run with a story and not question why certain therapists seem so keen to court the media when a story breaks, to not be alert to professional standards and be suspicious of those who so enthusiastically wish to overstep them. Journalists shouldn’t just accept a diagnosis of ‘sex addiction’ but should instead ask questions about what the term means, who is promoting it, why it’s gaining popularity, and whether the wider scientific/health community even agree with it.

Your story then is not Mr-Woods-the-sex-addict, but questioning why some folk seem so keen to diagnose by media and what are we saying about our relationships in the 21st century that infidelity is now a medical condition requiring a 12 step solution.

Rather than accepting at face value a glossy website, smart suit, claims of being seen on chat shows or advising politicians, why not check if this reputation is shared among therapists professional colleagues (not just their hangers on).

It’s dangerous when psychologists and therapists use celebrity relationship problems as a means for promoting themselves, their products or their services. It misleads the public about the nature of problems and solutions available. It promotes untested ideas and misdirects journalists about the evidence in this area. It reduces the chances to offer solutions to those needing help, while it invites judgemental attitudes and blame – which may well put people off seeking therapy.

Since it’s unlikely Tiger Woods will be the last celebrity to ever be in this situation, what can (and should) reputable journalists do if such a story is to be covered?

Firstly, if you want to gossip or speculate about the celebrity don’t do this with the involvement of any therapist, health professional or psychologist. Where you do involve an expert only have them speak about the issue at hand – not the celebrity. Ask the professional to provide information about the issue (why it happens, how common the problem is, how people usually respond and what help is available). Focus on solution based discussions rather than inviting the expert to blame or be judgemental.

Be very cautious about therapists or other experts who contact you direct when a story breaks and offer their services to talk about a celebrity – indeed such a person is indicating they are not professional and it’s a sign you shouldn’t be talking to them – ever.

When celebrity relationships break down it’s easy for journalists to see it as copy, the public to see as entertainment, and therapists to view as a fantastic marketing opportunity for their counselling services.

But we must never forget this is about these stories are always about families falling apart and relationships under pressure. Intense media scrutiny at such a time can be devastating and harmful and as ethical practitioners our job should never be to add to any suffering a person is already experiencing.