June 5th, 2005
Yesterday I commented on Maxine Clayman’s Press Gazette article about problems between PR companies and journalists.
And today there’s a classic example of how a PR company has got one over on a national newspaper.
On the front cover of ‘The Observer’ – under the heading ‘The Sharpest Writing on a Sunday’ – was the beguiling question ‘Does Infidelity Matter?’
I eagerly turned to page three for the answer. The whole page was dominated by an article claiming ‘Now sex with other people doesn’t mean you’re unfaithful’. I was intrigued. Most other social research on this topic is fairly clear. The majority of people questioned usually have issues about infidelity and certainly see sex with other people as ‘cheating’. Was this feature based on exiting, rigorous, and far reaching research that would disprove findings from all other studies?
No. It was a PR based ‘survey’ to promote the film ‘Closer’ about to be launched on DVD.
And I started getting flashbacks.
A few months ago I was sent an email from the PR Company promoting the DVD for Closer. They said they’d commissioned a survey for the film launch of the movie but hadn’t got round to using it. There’s your first clue this Observer piece isn’t exactly based on a cutting edge report – it’s been stuck on a shelf for months. I was asked if I’d like to be the media spokesperson for the DVD.
I asked to see the research first. And I was shocked. It was 30 pages long and contained no statistical analysis whatsoever (unless you count listing pages of percentages, which I think most social scientists wouldn’t). The qualitative data collected was even worse – simply dumped into boxes with no attempt made at analysis. And many of the questions respondents were asked wouldn’t have got past an ethics committee. I told the PR Company whatever they’d paid for this report; I felt they’d wasted their money.
I also told them I wouldn’t be able to endorse the movie if I also had to endorse the research. ‘Why?’ they asked me. ‘Because I conduct survey research and teach other people to do so. I can hardly support a survey that’s as poor as this one’
‘What’s wrong with it?’ They wanted to know. So here’s what I told them.
There was no evidence for the ‘study’
The report opened with the dramatic claim “[t]his in-depth survey of people’s attitudes to infidelity shows that views on being unfaithful have shifted dramatically over the last 20 years”. Which may be true but the people behind the report hadn’t found any evidence to support the claim – nor cited evidence from the previous twenty years to show these attitude changes. Perhaps that’s because they weren’t social researchers who specialise in sex, but a consumer study company.
Any reputable social scientist working in this area wouldn’t just make bold claims, they’d have the data to back them up. And they’d be able to do some statistics. They’d also probably not use pejorative terms in their research like ‘ cheat’ to refer to people who’re unfaithful – since they’d know people won’t be honest about their activities if they feel they’re being judged. Most reputable modern sex surveys don’t even use the term ‘adultery’ or ‘infidelity’ for that very reason.
There’s no analysis to explain the data
The study made great claims without any analysis, rendering it meaningless – “there’s a huge rift between men and women. Almost half of men don’t believe in monogamy, but less than a third of women are non-believers”. Statistical analysis might reveal that ‘huge rift’ wasn’t as great as all that.
The participants were asked leading questions
The ‘survey’ included howlers like:
“Are human beings meant to be monogamous?”
“Leaving aside your own personal view of infidelity, do you think it is becoming more acceptable to be unfaithful to a partner than it was 20 yrs ago?”
“Do you think our current technology is making it easier to be unfaithful to a partner than it was 20 years ago?”
“Can you be described as being ‘unfaithful’ if you cheat on someone when you are not married?”
Infidelity wasn’t questioned
The crucial problem with the Closer ‘infidelity survey’ wasn’t so much that it was an exercise in poor ideas asked inappropriately and then not analysed. It was the built in assumption that participants would know what ‘infidelity’ meant and all would respond accordingly.
Towards the end of the ‘survey’ participants were asked to say what activities could count as infidelity:
Sending flirty text messages to another
Sending flirty emails to another
Making sexy phone calls to another
Sharing an intimate meal with another
A long lasting kiss with another
Going further than a kiss (i.e. sex)
Clearly these six options aren’t going to cover the scope of what people might see as cheating, and based on them of course participants aren’t going to be overly concerned. What the study really found was participants were less bothered by flirty emails or texts than a partner having sex with someone else.
What is infidelity? For some people it’s as simple as thinking about another person when you’re in a relationship, for other’s it has to involve intercourse. Some people can handle a partner being sexually unfaithful but go to pieces if they become emotionally involved with another. We simply don’t know exactly what people consider to be ‘cheating’ but we do know it’s varied. Shame the people who conducted this ‘infidelity survey’ didn’t read the existing evidence to figure out the same.
By assuming there is something out there called ‘infidelity’ that we all know about and agree upon, they created meaningless data since we can’t be sure what people really meant when they were giving their answers about ‘cheating’.
Even a novice social science student should know you don’t use hard to clarify terms in surveys for this very reason.
Anyway, after sharing my concerns with the PR Company I heard no more about this study until today. Obviously I couldn’t endorse a study that wasn’t telling us anything useful and at times appeared unethical. I had said I’d be happy to talk about the issues the film raised, but seemingly they weren’t interested. And even after being told the study was fundamentally flawed, they press released it anyway.
Not that it made any difference since oops, the journalists reporting the study couldn’t do the math either! The ‘infidelity survey’ questioned 500 participants – 202 men and 298 women. Perhaps the journalists who wrote The Observer feature were having a bad day when they reported these findings because in the paper today they stated over 1000 respondents took part. Or perhaps they just didn’t check the original report.
If they had bothered to read it they’d perhaps have noticed the problems I’ve listed above. And also seen the study actually found people were concerned about infidelity if it meant having sex with another person, they just weren’t very bothered about flirty text messages.
The Observer article announced new rules for dating and infidelity that certainly can’t be found in the ‘infidelity survey’. It also over hyped the issue – talking about ‘a new phenomenon taking hold in Britain: young couples who are much more relaxed about concepts of fidelity and monogamy than their parents generation’. Again, the evidence for this can’t be drawn from the ‘infidelity survey’, nor the other backup sources the article proudly lists – Sex and the City and Desperate Housewives. Oh, and ‘The Rules’ dating guide, which psychologists have proven damages relationships. So we had a non-story based on a non-study, but still it found it’s way to page three of a national broadsheet paper.
Does infidelity matter? Yes it does. It matters to people who’re having relationship problems, and it matters to social scientists trying to understand the issue in more depth. It clearly matters to journalists too, but in a very different way. It matters to them because they want to run a dramatic story without having to do any fact checking first, and it undoubtedly matters to PR companies wanting to promote a film on the back of a survey that would be given a ‘fail’ grade if presented as a first year undergraduate social science project.
Last week a journalist told me ‘you shouldn’t care about the media so much – journalists don’t want facts or accuracy, they just want good headlines’. I was hoping that wasn’t true, but today it seems chillingly accurate. A front-page headline and a page three spread. A nation being told that nobody cares about cheating based on a flawed study.
And for what?
To sell people DVDs.
And journalists are worried about how PRs pitch them stories? I’d have thought this shows they’ve got many more things to be concerned about.Tweet