Repeat offenders

Tara Lyons, director of the Independent Fieldwork Company, wants to share her database with her competitors. It’s not a large database, just a list of 34 names and addresses; 34 men and women whom Lyons and her team have come to know rather well. Rather too well as it happens because these characters are research groupies, members of the public who lie, change their names and even their appearance to take part in focus groups.

It is an unlikely scam (but at 30 or 40 quid a night, a reasonably lucrative one) producing moments of pure farce – the chap whose cover had been blown, who reappeared using an alias and was minus a beard, the woman who unexpectedly recalled a long-lost twiníĶ..

But how prevalent is focus group fraud? Is it merely the odd rotten apple? Or, are the horror stories indicative of something more worrying? Is there a fundamental lack of quality control, a widespread collusion between respondents, recruiters and researchers, in which professional standards are routinely sacrificed in the interests of delivering the required number of “bums on seats”?

It is a murky picture, not least because the market research industry is divided over the scale and the causes of the problem. Occupying one camp are figures such as Discovery Research director Ken Parker and Criteria Fieldwork managing director Elaine Francis, who earlier this year raised the spectre of an industry in which “the quality of recruitment is on a downward spiral”.

On the other side of the fence are the industry spokespeople who accuse the would-be whistle-blowers of exaggeration and of failing to distinguish between fraudulent practices and the pragmatic adaptation of industry norms. So what are the facts?

The truth will out

First: dishonest respondents are a real and, possibly, growing problem. Yet what matters here is not the absolute number of those involved, but the damage being done to reputations. Janet Weitz, chairman and chief executive of FDS International, says: “Respondents are becoming more sophisticated, they know how to work the system and the price we pay is the loss of our respectability.”

Second, as market research briefs become more complex and time scales are minimised, the temptation to cut corners is growing. And compounding the former, market researchers themselves are often confused over the rights and wrongs of recruitment. So, for instance, while falsifying identities is condemned unanimously, there is less certainty over whether to exclude serial attendees who simply enjoy groups, are open and honest about this, and generally have something interesting to say. So what is the best way forward?

Most obviously, outright deception has to be made more trouble than it is worth. Nothing complicated is needed, just a few common-sense procedures, such as obliging respondents to supply proof of identity, creating a database of people who have lied, and following acts of deliberate deception with a strong letter of complaint.

Some agencies already do this, but there is a need for more consistency and a concerted effort by the industry bodies, the Market Research Society and the Association for Qualitative Research (AQR), to get everyone up to the same standard. One specific initiative that many practitioners hanker after is an industry-wide database holding details of fraudulent respondents.

Lyons says: “We keep a database of cheats, but there’s a limit to what we can achieve individually. It would be better if the details were held centrally and made available to other agencies. Data protection authorities would have to approve this, but it is an obvious way forward and something the trade bodies should lobby for.”

Checks and balances

Clamping down on flagrant abuses is only the beginning. As important as whether people are who they say they are, is whether they have been accurately recruited and can contribute to the group. James Parsons, director of research agency Flamingo International, says: “The most crucial issues are the ones that affect the outcome of the research. If people say they’re loyal to brand X when they’re not, or if they just don’t have the attributes that the client has stipulated, that’s a big problem.”

Ensuring that respondents genuinely reflect the specification is getting harder by the day. A key factor in this is the trend towards recruiting people for their skills or psychological make-up, for instance early adopters, opinion leaders or individuals who are unusually creative.

It’s no-go, Joe

Research International Qualitatif director Jane Gwilliam explains: “In the past, qualitative research was testing things out on Joe Public, now there’s more emphasis on working with consumers who can help to develop ideas and product concepts. Those people are much harder to find.” So how can the industry get on top of such challenges?

Developing better mechanisms for screening respondents is part of the solution. And some companies, particularly those competing at the premium end of the market, are already working hard in this area. One example is Link Consumer Strategies, which requires respondents to complete homework tasks as a condition of participating in certain projects.

Link managing director Louise Southcott explains: “If we wanted views on a brand, we might ask the respondents to prepare a collage board before hand. It’s a way of stimulating discussion and filtering out bogus users.”

Other effective approaches include combining groups with ethnographic research, recruiting and interviewing in situ, for instance in airport lounges and student bars, and conducting home visits to check that people genuinely use what they say they use. Lyon adds: “Checking a single household can cost £120, but it’s definitely worth doing in some cases.”

Spending more money on recruitment goes against the grain for many clients, who would like to regard accuracy as a given. But in reality more investment is precisely what is needed. One of the oddities of market research is that while research methodologies have advanced hugely in recent years, recruitment has been sidelined and allowed to tread water. The visible evidence of this is the frankly marginal status accorded to the overwhelmingly female recruiters who do the work, usually on a freelance basis. So what has to change?

To begin with, recruitment has to be recognised for what it is – not some peripheral activity, but an integral part of market research that can make or break the whole project. Lyons says: “It would be great to offer some extra incentives for freelance recruiters to join the AQR and to get actively involved in extending industry guidance on tricky issues, such as repeat attendance.”

Putting in some groundwork

Another action the trade body could usefully undertake, says Criteria’s Francis, would be to develop a modernised training and accreditation scheme focusing specifically on the professional development of recruiters employed in qualitative research. More important than industry initiatives, however, is action on the ground. And here major changes are required, beginning with a dispassionate appraisal of how the process is managed from start to finish.

Judith Wardle, AQR spokeswoman and partner at strategic research consultancy Wardle McLean, says: “Most recruiters do a fantastic job and take pride in their work. Where problems arise, it’s often because recruiters are not being supported by people higher up the chain.”

And she adds: “It’s tempting for a young researcher to say yes to the client, without questioning the brief. But if the specification is unrealistic, elements have to be left out or corners cut. Those compromises need to be agreed between the client and the researcher. It shouldn’t be down to the recruiter to save the day by pulling a rabbit out of the hat at the last minute.”

Wardle’s point that recruiters must be properly supported is well-made. The flip-side of this, however, is that the recruitment model needs to evolve and become more accountable and responsive to clients’ needs.

One option is to bring recruiting in house. For highly specialised projects this often makes sense. For instance, Research International recruits the majority of its research participants through freelancers, acting under contract to field agencies.

However, when the company is seeking “creative consumers” for innovation work the entire selection procedure is conducted internally. Gwilliam says: “It wouldn’t be fair to expect a recruiter to adjudicate on something as open to interpretation as creativity.”

Some field agencies are rethinking their approach too. For instance, Criteria has recently switched part of its recruitment activity from face-to-face sessions to call centres, staffed by agents working from databases of people who have been pre-screened by telephone.

Francis says: “The database helps us deliver samples at very short notice, without compromising quality. Call monitoring means that we ensure the screening questions are administered accurately and in full. We can log frequency of respondent attendance as well. It’s not perfect, because it doesn’t necessarily catch people who change their names, or identify people who are on other agencies’ lists – but it’s a big step forward.”

On the defensive

Market researchers get defensive when journalists ask about focus group fraud and often quote figures that supposedly set the issue in context. However, to bandy statistics is to miss the point spectacularly. Recruitment is a classic Cinderella occupation, undervalued and under-resourced, and is widely seen as such.

Not surprising, then, that qualitative recruitment has, as Weitz says, earned a reputation as “an easy one for the fiddle to flourish”. This perception has to addressed – and urgently – through industry initiatives, effective screening and, above all else, a greater willingness to invest in the recruitment profession.