Media must be more sceptical of self-serving surveys and wary about how they report them, writes NOEL WHELAN
THE SMALL Firms Association (SFA) chose the bank holiday weekend to publish details of what was grandly titled the Sixth Annual Business Crime Survey. In saying they published details of the survey, I am overstating the case. In fact they published a two-page press release outlining what they themselves concluded were the findings of their own survey.
The SFA was, however, well rewarded for its efforts. News desks, which presumably had only skeletal staff on duty last Sunday, lapped up the crime survey press release and used it to ground banner headlines suggesting that crime against businesses had risen significantly in the last 12 months.
The SFA analysis of its own crime survey was featured almost word for word as news in all national daily newspapers and in radio and television news programmes. Indeed, both of RTÉ's evening news bulletins last Monday used the survey to justify a headline claiming that more than half of Irish businesses were victims of crime in the last year.
What was striking about these media reports was that none gave details of the methodology used in the SFA survey. Anyone reading or listening to the coverage would have been forgiven for concluding that this survey established that thousands of SFA member firms had been victims of business crime. Such a conclusion fits comfortably into the "crime is out of control" narrative that many media seek to advance - but in fact the survey proved nothing of the sort.
This survey was not scientific and the methodology was rudimentary. The results of what was represented to be a sector-wide survey were based on a small, self-selecting sample. In a couple of phone calls this week, I was able to establish that this year's business crime survey involved staff of the SFA or some agency hired for the purpose e-mailing a link to an online questionnaire to 1,000 of the SFA's 8,000 member businesses.
Of those contacted by e-mail, only 189 businesses bothered to respond. This sample size is far too small, is not weighted and is undermined by the fact that those concerned about crime were more likely to complete the questionnaire.
Notwithstanding these obvious flaws, the 189 replies were relied upon by the SFA to make grand claims about rising levels of business crime. Its press release was copied by journalists into stories suggesting that business crime was on the increase without any journalistic fact-checking or analysis of the survey methods or conclusions.
The SFA did include in their press release that there were only 189 respondents, but almost all media reports failed to mention this salient fact. The SFA did not, however, state - nor it appears did journalists ask - how many businesses had been invited to participate in the survey, the sectors or locations represented by the 189 respondents, their business size or whether they were representative of the SFA's overall membership. Neither were we told what precise questions were asked.
This survey is only the latest in a long line of questionable surveys which have benefited from this type of unquestioning media coverage. Conducting a survey has now become the preferred method of attention seeking for organisations because it is a relatively cheap way of profile raising. Commercial concerns, financial institutions, representative organisations and even charities and NGOs are rushing to publish surveys in order to raise their profiles, and media are all too happy to oblige with coverage because it is also a relatively cheap and lazy way of generating so-called news.
This trend would be relatively harmless were it not that these organisations often go on to use these purportedly scientific surveys to shore up policy positions they want to advance.
While publishing the "results" of their crime survey, the SFA, for example, used its interpretation of them to support calls for a number of policy changes in how the Government and Garda deal with crime.
The absurdity of the headlines generated by the SFA survey is illustrated by the fact that in June media organisations carried the results of a similar crime survey published by the rival small business organisation Isme (the Irish Small and Medium Enterprises Association), which directly contradict those published this week by the SFA.
According to those media reports, the Isme survey with 1,500 respondents found that the number of businesses affected by crime in the last year had fallen dramatically rather than increased, as the SFA survey suggests. According to the Isme survey just one-third of businesses said they had been affected by crime in the previous 12 months, down from 45 per cent in its previous survey.
I am not suggesting that the Isme results should be preferred to the SFA results. I don't know enough about Isme methodology to say that. What I am suggesting is that neither survey tells us much about levels of business crime, neither is necessarily reliable and neither is actually news.
Media organisations need to be more sceptical of press releases bearing survey results and more wary about how they report their findings.