Yesterday evening a group of science enthusiasts, bloggers and at least two science journalists crammed into the Penderel’s Oak pub, Holborn. We were all there as an alternative (and free) event to run alongside the very expensive World Conference of Science Journalists happening in London this week.

The ‘Troublemaker’s Fringe’ event was planned in response to poor science and health journalism, and the apparent lack of critical reflection organised in the official WCSJ event. All attendees at the WCSJ event were welcome at our gathering, although rather depressingly/predictably it seemed most weren’t interested. Maybe the way we pitched the evening offended their sensibilities?

One journalist, the Independent’s Science Editor Steve Connor certainly was put out by the event. He told all three presenters in no uncertain terms where we could shove it in his opinion piece “Lofty medics should stick to their day job” In fact we made Mr Connor so furious he was able to describe the event before it had even taken place. Mr Connor was invited to the event, but sadly didn’t turn up. I’m sure he’ll be glad to know the event was recorded so he can catch up at his leisure.

The presenters were myself, Vaughan from Mind Hacks (who is very organised and has already posted his slides from his excellent talk on media panics over communication technologies), and Ben Goldacre who you’ll all know from the famous Bad Science site.

My talk tackled Eight problems with science/health journalism and what we can do about it. I’ll be posting the slides later. Below is a list of the eight problems identified and links to further reading about the issues raised in my talk.

Problem #1 – Overreliance on surveys
The media is currently swamped with ‘surveys’ which are really a disguise for PR activity and free advertising. You can check places like response source or google news at any time with the word ‘survey’ (prefixed with any term you like – car, health, sex, food) and you’ll find a smorgasbord of dreadful surveys promoting all manner of nonsense. Along with the occasional kosher study. Except those are hard to spot and increasingly difficult to get covered as they’re competing alongside PR surveys which use shocking ‘findings’ and (allegedly) large samples to ensure coverage.

PR based surveys (or as Charlie Brooker describes them - PR-Reviewed Phindings) are cheap and easy content for journalists who are pressed for time and under pressure to fill copy. Unfortunately many journalists don’t understand social research and therefore believe the ONLY method out there is a survey, or that studies using smaller scale samples (qualitative research) or complex designs (RCTs, epidemiological studies) are either untrustworthy or too difficult to report on. Besides, PR companies make a very good job of writing clear press releases with all information set out for copying straight into your piece – and they pester the life out of journalists to ensure coverage. Something academic research doesn’t tend to do so well.

I’ve written various rants about crappy PR surveys on this blog previously (a fairly good summary with links to some atrocious examples of PR surveys can be found here). You may also be interested in this guide I wrote for the charity Media Wise ‘How to spot PR based research’. If you are interested in surveys for research I wrote a three part series on questionnaire design and use in the BMJ with colleagues a few years back. Unfortunately these aren’t open access, but if you want copies please email me.

One comment arising from the meeting last night (from someone who works in PR) was that academics could learn from PR in terms of how we deliver key messages. I completely agree with this, and for those of you interested in this area you might want to check out Gerard Hasting’s excellent Social Marketing: why should the devil have all the best tunes?

Problem #2 – The Fake Formula
This is the friend to the duff PR survey. It’s a false quasi mathematical ‘formula’ that suggests you can identify perfect days, biscuits, kisses and the like. All to promote a product. I mentioned an ongoing and well known problem formula from Cliff Arnall concerning the Most Depressing Day equation, which has combined poor mathematics with collaboration with mental health charities who ought to know better. Oh and threats to me from Mr Arnall to sue, so the least said there the better.

Aside from the bad science behind such formula there’s the problem of ‘experts’ fronting them. Which is particularly worrying when they are endorsing sexist celebrations of say, the perfect breast or bottom. Interestingly some of those who’re keen to participate in such activities view what they’re doing as science communication – as evidenced on a previous episode of Radio 4’s More or Less.

Problem #3 – Science/health stories are not always written by science/health journalists
Many of the stories concerning the health or social sciences are written about in the style pages of newspapers or mainstream magazines. By people with little or no training in how to understand social research – or critically appraise evidence. Which results in such travesty’s of reporting as the recent What Women Want stories running in both the New York Times and Sunday Times newspapers. Blogged in a very ranty (and long) post here.

Problem #4 – Where journalists cover social/health/natural science research they often fail in even basic fact checking
To evidence this particular claim I referred to a recent piece in the New Scientist about the female orgasm and six things science has revealed about it. Which suggested the New Scientist don’t understand contemporary research on female sexuality, but are very happy to promote poor studies on sexual behaviour that often has links to drug companies or are just very badly designed. The post above has links to all the poor coverage from the New Scientist and critical appraisals of the studies reported so you can see for yourself just how shoddy this coverage has been. The result for the public, of course, is misleading information about sex, relationships and how our bodies work. Again, you’ll see from the links above this can cause distress and dissatisfaction.

During the talk I mentioned one piece of research that studied twins using a questionnaire and concluded orgasmic dysfunction was genetically inherited. I questioned how you could measure genetic inheritance from a survey and one audience member rightly reminded me that comparing answers from identical and non identical twins would enable you to see if there was heritability (gleaned from more similar answers from the identical twins). This is a fair point, although I think you’ll find from the report of the survey it did not really measure what it claimed to at all.

Problem #5 - On many issues, health/science coverage appears completely uncritical but when it’s important to have a balanced account, you can be sure coverage will be VERY judgemental
All the previous examples indicate that fake formula, PR surveys and shoddy research are unquestioningly accepted by the media on a regular basis. However, there are occasions when the media become very critical and the example I used here was around the response to changes to sex education on the UK school curriculum suggested last October.

Following a consultation on the issue, and (unusually for the current government) reviewing the evidence base, it was agreed sex education needs to change (see link above for documentation on how this might work). This was in response to our rising STIs, teenage pregnancy and wider problems around children growing in a highly sexualised culture.

Sadly the majority of media coverage ran with hysterical headlines and coverage implying CHILDREN AS YOUNG AS FIVE TO GET SEX EDUCATION! Of course the response from the public was negative, with fearful parents worried their little ones would be given condoms or corrupted with adult concepts of sexual behaviour. You can’t blame parents for being worried. But the truth is young children won’t be taught about condoms, they’ll be taught about relationships and confidence and emotional awareness. As they grow they’ll be prepared for puberty and learn about relationships, pleasure, sexuality, contraception and STIs. It was as though the media here deliberately wanted us to be afraid of something they knew full well wasn’t going to happen. When it comes to sex education and sexual health they do it all the time.

The only exceptions I’ve noted are The Mirror, who have attempted some balance on this issue and ITV’s This Morning whose agony aunt Denise Robertson has kept this issue on the agenda and reassured parents about the need for sex ed – and how to do it.

[After last night a few people asked me for further references about sex education and young people, so you may be interested in these blogs:
Fifteen tips for talking to your kids about sex

A critical take on delivering sex education (with a link on what to say to your kids at different ages)

Helping parents become sex education experts]

Problem #6 – The rise of the ‘fakexpert’ and the ‘sexpert’
The media currently relies heavily on ‘experts’ to stack up stories. Usually psychologists (or people pretending to be psychologists) folk are wheeled on to give quotes, analyse celebrities or make general judgements. In return for a plug for their book/website/product.

My colleague Gary Wood has written two excellent summaries about this practice in his blog Psycentral
Celebrity Body Language: fact or flim flam?
Gender, Cave People and an Apology for Psychology

I’ve also written a guide on how to identify an expert (and what they can/can’t do in the media) for PressWise

During my talk I complained about the problem of ‘sexperts’ in the media. Unqualified people who give sex advice. I wrote about this a few years ago in The Guardian – Beware the Sexperts, as well as some more in depth academic papers on this – both opinion pieces and ethnographic studies of my experience working as an agony aunt/sex columnist in the mainstream media. (If you can’t access the papers below and would like them please email me):
Advice for Sex Advisors: a guide for ‘agony aunts’, relationship therapists and sex educators who work with the media Sex Education (2007) Vol.7 (3) ps: 309-326

Understanding media coverage of sex: a practical discussion paper for sexologists and journalists Sexual and Relationship Therapy (2006) Vol.21 (3) ps:333-346.

Whatever happened to Cathy and Claire? Sex, advice and the role of the Agony Aunt (book chapter in Mainstreaming Sex edited by Feona Attwood).

Problem #7 - Media training for scientists and journalists is polarized and outdated
So we’re pitted against each other, have our training in isolation, and do not learn from each other. I’ve been trying to implement more training that brings journalists and academics together, along with archiving resources for journalists and training for journalists that lets them know what scientists’ working practice is really like. I have ongoing issues about media training for academics which seems to operate on training you to write press releases or present your research findings – when the modern media wants you to speak generally about your area of working practice in ways you’re rarely prepared for. I’d welcome further discussions on how we might improve this.

Problem #8 – We pitch our criticisms at the wrong level
There’s a tendency to pick on individual journalists or just blame ‘journalists’ generally for bad media coverage of science/health. The problem lies with editors, publishers and production companies. It is about economics and unspoken prejudices. We need to think of more effective ways to highlight poor practice but take the fight to those who hold the power. Most journalists are stuck having to deliver poor coverage because they’re told to do it and won’t be paid if they don’t. Bullying is rife in media (something journalists have in common with academics, sadly). I’m not saying we shouldn’t criticise poor practice, but I do feel we need to find ways to work more collaboratively together.

Finally, it’s very easy to highlight bad practice, but we don’t do half as well as praising good media coverage of science/health. So I’d like to see more efforts made to do this.

And not to forget the small things. The problems I’ve highlighted are often those related to surveys, ‘soft science’ and sex – things we treat as a joke or ignore. We can’t afford to let this sloppy coverage go as it’s all part and parcel of generally low standards. And whether it’s a cancer scare story, misreporting MMR, or a sexpert telling you something incorrect about your sexual health – it’s all harming the public.

If you’ve any questions about the event last night please let me know. I’ll add further links/updates as other bloggers report on this.

And thanks to those who turned up. I thought it was a great night with amazing talks from Ben and Vaughan. Shame hardly anyone from the WCSJ showed, but here’s hoping we can disseminate the audio from the event with journalists and encourage wider discussion.

Ever wondered what the age of consent for sex was in Honduras, Khazakhstan, Sri Lanka, or the Isle of Man?

Well, now you can find out thanks to HIV charity AVERT who have produced a table of global data on the age of consent for straight and gay sex. You can view it here.

It’s useful if you’re a young person and want to know when sex is legal within your country, what your country’s position is on homosexuality and teen marriage, or if you’re a parent or teacher and are worried about exploitation.

What’s worth noting is how some countries legally permit much younger ages of consent for girls than boys, some omit any information on homosexual sex (mostly for countries where this is illegal). Some countries make it clear that you can have sex at a young age (around 14) but only if you are married.

It’s a fascinating read and calls into question a lot of our assumptions about teenage marriage, pregnancy and sexual behaviour. Where one country may see sex as illegal until someone is 17, others permit sexual activity at 12. Perhaps unsurprisingly the countries where sex is only legal within marriage or where sex with 12 year old girls is permissible are most likely to legislate against homosexuality (particularly for men).

Along with the table of global age of consent data, AVERT also have a helpful report which asks ‘Am I ready for sex?’. Something you may want to consult if you’re thinking about having a sexual relationship, or if you’re a sex educator working with young people. I’d also recommend reading a similar checklist from the fantastic Scarleteen Ready or Not?

It’s worth noting that different countries have different approaches to how they manage underage sex. Some have legal limits but enforce them less strongly than other countries. Some may punish underage sex, extra/pre marital sex or homosexuality more stringently than others. So if you are unsure about your legal position you may want to think carefully about the ramifications of having sex. And if you are in a situation where you feel exploited or coerced, regardless of your age, you should seek support and help.

By now you’ve all heard the sad news about the death of Michael Jackson, and doubtless seen the increasingly ghoulish media coverage of the case.

I was contacted by journalists working on two separate stories about the case today.

The first wanted me to describe what ‘the impact of Jackson’s death’ would be on his children – and speculate on a ‘possible, forthcoming custody battle’ (and again describe what that might do to the children).

The second wanted an in-depth ‘profile’ of Jackson, with a specific request that I show a ‘link’ between Jackson’s childhood experiences and (alleged) subsequent ‘abuse of children’.

I refused to do either. Firstly, to discuss a celebrity and speculate on their lives is unethical and breaches my professional guidelines. (Let’s all repeat the very dull Dr Petra mantra ‘if I know the celeb personally I’m breaching confidentiality by discussing them, and if I don’t know them personally I’m just gossiping about them’).

Secondly neither of the questions I was asked to comment upon are within my area of expertise – so again it would be unethical for me to make any pronouncements about them.

Unfortunately this does not seem to have stopped a number of psychologists from the UK and US from commenting publicly on a variety of issues relating to Jackson and his family.

Here are a few of the worst ones I’ve spotted:
Michael Jackson’s children could be damaged for ever by custody battle, says top psychologist

Jackson: Inside His Mind

Was Michael Jackson’s Death Possibly Attributed to His Inability to Be His “True” Self?

What will happen to Michael Jackson’s kids?

It’s worth noting that not all the people giving ‘psychological insight’ are psychologists, and those who are qualified as psychologists may not belong to any official bodies that oversee professional standards. Sadly some are members of professional bodies but don’t seem to be following their rules.

The impression I got from the two journalists I spoke to was pretty clear. Any psychologist would do. They weren’t bothered about qualifications or expertise. They wanted a quick analysis – as one put it - an insight into ‘Michael Jackson’s Mind’.

Neither journalist accepted my statement that it was unethical for me to comment in the way they requested. Both told me ‘other famous psychologists’ had already ‘analysed Michael Jackson’.

Unfortunately this is a major problem with ethical practice. Because for every professional who is careful, there are many more who are happy to give the media what they want.

The result we’re seeing is the misuse of psychology within the media, the misrepresentation of psychologists, and in some cases the exploitation of a tragic story to promote someone’s products, services or name.

Ethical psychologists are those who don’t try and turn someone’s personal tragedy into their personal gain. They can take a view about whether it’s right for them to comment, and do so in ways that are appropriate. They are also able to decline an interview if they think there’s nothing more to be said, or if it’s better to remain silent.

I’m not saying reporting on the Jackson case is wrong. What is wrong is where psychologists are being invited to go way beyond their skills base and to speculate on someone – mainly based on what they’ve gleaned through the media.

We should always remember in cases like this that there are family members, friends and colleagues who can be deeply hurt by generalisations and speculations. And a wider public who may be misled into thinking what is shared is more than gossip and actually represents a psychological insight into a celebrity.

Penderel’s Oak Pub, Holborn, London
1st July 2009
8pm – Midnight

Next week the World Conference of Science Journalists will be coming to London. A few of us felt they were might not adequately address some of the key problems in their profession, which has deteriorated to the point where they present a serious danger to public health, fail to keep geeks well nourished, and actively undermine the publics’ understanding of what it means for there to be evidence for a claim.

More importantly we fancied some troublemaking and a night in the pub.

As a result, you have the opportunity to come and see three angry nerds explain how and why mainstream media’s science coverage is broken, misleading, dangerous, lazy, venal, and silly. Join our angry rabble, and tell the world of science journalists exactly what you think about their work.
All are welcome, admission is free. They may not come.

After the presentations (with powerpoint and everything, in a pub) we will attempt to collaboratively and drunkenly derive some best practise guidelines for health and science journalists, with your kind assistance.

Ben Goldacre has written the Guardian’s Bad Science column for 6 years, where he exposes misleading science journalism, health scare hoaxes, pill-pushing quacks and the crimes of the evil multinational pharmaceutical industry. He will talk about how the media promote the publics’ misunderstanding of evidence, focusing on health scares, journalists’ hoaxes, and their consequences, as well as cases where scientists have had their work misrepresented and failed to get satisfaction
from newspapers.

Vaughan Bell is a neuropsychology researcher and clinician in the NHS, where he deals with disorders of the mind and brain, and is a writer for MindHacks.com, where he deals with disorders of the media. His talk will be called “Don’t touch that dial! Technology scares and the media” and will discuss how the media loves to
tell us that new technology will give us brain damage and mental illness but is strangely adverse to discussing the research even when the science says there’s not a lot to be worried about.

Petra Boynton is a Social Psychologist and Lecturer in International Health Services Research. She specialises in researching sex and relationships health. For the past 7 years Petra has worked as as an Agony Aunt in print, online and broadcast media. She actively campaigns for free and accurate sexual health advice within the media both in the UK and Internationally. Petra will talk about the consequences of PR companies misusing surveys and formulas as a form of cheap advertising, the problem of unethical or untrained people posing as ‘media experts’, and what happens when journalists fail to fact check science and health stories.

Of note, attending the WCSJ will cost you £200 a day. You are welcome to come to our event entirely for free, beer/shrapnel in a bucket gratefully received. Journalists, corporate event organisers: welcome to the shits and giggles economy. Special thanks to Sid the Skeptic from Viz for booking the room at short notice.

What:
World Conference of Science Journalists 2009 – Troublemakers Fringe

Where:

Penderel’s Oak Pub, 286-288 High Holborn, London WC1V 7HJ (Holborn Tube).

When:
1st July 7pm for 8pm – Midnight

Please come along if you’re interested in how the media tackles science stories, if you’re a science writer/blogger/journalist, an academic who’s interested in working with the media, or just someone who fancies a pint and a lively discussion.

Hope to see you there! Please feel free to forward this message to anyone you think might like to attend.

You may remember my recent blog on the case of science journalist Simon Singh who was sued for libel by the British Chiropractic Association (BCA) following a piece Singh wrote about treatment of childhood ailments with chiropractic.

One of the criticisms those working in healthcare and science have made of the BCA hinges on their claim to base their practices on evidence without making it clear what this evidence is.

On Wednesday (17 June) the BCA circulated a third update on their case against Simon Singh which you can read here. Within this document the BCA included a list of 29 publications which they claim represent ‘good’ (but not overwhelmingly conclusive) evidence on the “effectiveness and safety of chiropractic treatment in children”.

So the BCA have now produced the evidence they feel enables them to make decisions about chiropractic treatment of infants. This allows the wider scientific community, healthcare professionals and anyone else with an interest in this topic to interrogate the studies listed.

Many bloggers have been quick off the mark and produced some excellent critiques on the evidence promoted by the BCA. Highlights include:
Jack of Kent’s blog BCA’s worst day (also contains links to some other fantastic blogs on this topic)

Holford Watch’s BCA demonstrates what Evidence Based Medicine isn’t

Evidence Matter’s British Chiropractic Association and the Plethora of Evidence for Paediatric Asthma

Gimpy’s discussion on the evidence around ear infections

David Colquhoun reviews the papers on colic presented by the BCA

Ministry of Truth’s analysis of the supplied evidence

along with a further excellent critique from God Knows What

also covered in depth by The Lay Scientist

Since the BCA don’t clearly explain their search strategy or appraisal of this evidence it is difficult to know why they selected these particular papers. It’s also unclear how said evidence is used. We don’t know if these papers are recommended reading for all chiropractors, for example. I have emailed the BCA to ask them for their search strategy, key terms used to identify papers, papers included/excluded in their search, how they appraised the papers finally selected, and how they use these papers to directly inform their treatment of infants.

What does seem immediately obvious (and is clearly outlined in the blogs linked above) is the evidence produced by the BCA does not seem to support their assertions for the success of chiropractic treatment on infants. Indeed, it would be worrying to think these papers have underpinned any practice. Also of concern is the BCA’s assertion they are using evidence based approaches and yet do not appear to demonstrate this fully in the papers they have cited. Not least because they have failed to contextualise how they searched for, selected, and interpreted these publications.

The blogs listed above have all done their bit to critique and summarise the evidence presented by the BCA. What the BCA makes of this remains to be seen. No doubt supporters of the BCA will argue those who’ve already blogged about the BCA’s evidence are biased and were deliberately negative in their reviews.

To counter this claim, as an exercise in good science practice, and a learning opportunity I would advocate that anyone who wants to evaluate the BCAs evidence should have a go themselves. Those of you teaching/researching within the health/social sciences may want to use this as a teaching activity.

I have archived all the papers cited by the BCA that are available electronically. If you’d like to see them, please email me.

Here are some additional resources to help you carry out a critical appraisal of said papers.

[I’ll be adding a downloadable table to help you appraise the papers shortly]

Trisha Greenhalgh’s excellent book ‘How to read a paper: the basics of evidence based medicine’ provides detailed instructions on how to critically appraise research using a variety of methods (RCTs, qualitative studies, questionnaires etc).

University of Sheffield’s School of Health and Related Research department have an easy to follow introduction to Critical Appraisal and Searching the Literature which guides you through the process of finding papers and making sense of them.

The Centre for Evidence Based Medicine has a slew of free tools to help you understand evidence and carry out research.

The National Collaborating Centre for Methods and Tools has an equally helpful stepwise guide to the process of collating and interpreting evidence that might improve/inform practice (includes some great links to tools to help with all areas of appraisal).

Surgical Tutor has a stepwise guide to appraising a paper (it focused more on quantitative research, but gives an idea on how to get started). This open access paper from Nature is similarly biased towards assessing quantitative studies, but it’s 10 steps for critical appraisal is still a helpful guide on finding your way around a paper.

Trent R&D have a fantastic resource pack with worked examples that takes you through how to search the literature, find papers, make sense of them, and apply them to your practice.

For a more reflective account on the whole process of understanding evidence, how we do it, and why it’s important I’d recommend reading ‘Just the Facts Ma’am’ courtesy of Canada’s National Coordinating Group on Health Care Reform and Women, which is written from a women’s health perspective but covers in lay terms the kind of questions you ought to be asking about research.

Learning to search the literature and appraise papers isn’t always easy, but is really just a matter of practice. The reason it’s important to use evidence to inform practice is to make sure whatever we recommend for patients/the public is safe and effective.

This is important whoever you are working with, but particularly important if your patients are children.

With that in mind I’d invite you to look again at the list of papers recommended by the BCA and see if you think what they’ve presented justifies chiropractic treatment on children.