Research gets wired
The Internet is rapidly emerging as the Wild West of the market research world. Few in the industry doubt its long-term potential to reduce the cost and improve the quality of market research. But, as all the potential benefits of conducting research through this frontier medium are being debated, many fear that standards are being compromised in the headlong rush to exploit the Internet.
John O’Brien, chairman of BMRB International says: “We should encourage researchers to experiment with it, but it’s better to be cautious and not claim that it can achieve too much.”
As a member of the professional standards committee of Esomar, the European-wide umbrella group for market research professionals, O’Brien has been involved in trying to inject some professional and
ethical rigour into how Internet-based research is conducted and
presented.
“Using the appropriate technology, conducting online market research can be quite straightforward,” he says. “But the problem with the Internet continues to be that it cannot deliver a representative sample of any population. And this problem of representation is fundamental to conducting mainstream consumer research on the Internet, in the same way as it was for the telephone 30 years ago.”
The problem is that it is becoming cheap and easy for anyone – including respectable market researchers – to generate large volumes of responses to surveys conducted on the Internet. Consequently, according to Peter Hutton, who heads MORI’s new media division, some clients may naively view the Internet as a route to conducting low-cost consumer research, because it can bypass the major expense of conducting fieldwork face-to-face or through properly random telephone-based interviewing.
“To compile lists of people with Internet addresses is very cheap. You can quickly push out an emailed questionnaire and generate a lot of numbers, even with very low response rates. However, what you end up with is rubbish,” he says.
“But there are those who will download email addresses from people in various news groups, lumping together those interested in garden gnomes with BMW owners, and then send out questionnaires. So you get a response from a large number, but a strange mix of email addresses. Then they collect the information and publish it. It’s easy to end up with a report that looks as though it’s gone out to large numbers of people – but it’s dangerous.”
Organisations like the Market Research Society and Esomar are trying to grapple with some of the methodological problems and professional issues involved in attempting to develop the market for Internet-based research. According to O’Brien, there is no fundamental difference in the Esomar code of practice as it applies to Internet research as compared to other
research techniques. The same care should be taken in sampling, selecting and interviewing people over the Internet as applies to target groups in other projects, says O’Brien.
Even greater care should be taken in the presentations and claims made for Internet-based projects by professional market researchers, because of the methodological weaknesses which exist in researching most segments of the population online.
“Internet-based research can provide a good first stab and a quick and cheap way to test things out among groups which are heavily online,” says O’Brien. “But if you are making major strategic investment, I wouldn’t rely on it.”
Professor Martin Collins specialises in market research and new technologies in marketing at The City University Business School. He has tracked the debate about online research through org.org, an interest group set up under the auspices of the Market Research Society and headed by Nicky Perrott of Reuters. Collins suggests that while the volume of domestic Internet connections in the UK remains as low as 2 million, its usefulness for general consumer research conducted through the Internet will remain very limited.
“There are plenty of people making a lot of sales pitches for it both here and in the US, but with domestic connection so low, it’s a fairly useless medium for general purpose surveys. And because of its demographic skew, it’s particularly unsuitable for researching the attitudes of female shoppers or housewives, on which much research is focused.”
The only solution to conduct perfectly representative online research may be to pay for the installation of PCs as well as online access for a typical cross-section of the UK public.
“But if we create panels of pre-recruited people and pay for them to be connected, it undermines much of the cost advantage over traditional research
techniques,” says Collins.
Panos Manolopoulos, research manager at NOP, believes the Internet already has significant potential for conducting market research among particular target groups which are well represented on the Web. But he agrees with Hutton, O’Brien and others that there are major barriers to be overcome in conducting broader consumer research.
“It’s suitable for testing reaction to a Website, or testing a Web concept among general users and it may be relevant to conducting things like staff surveys where the majority of the workforce has Internet access,” he says.
Professional groups such as doctors, who are often hard to reach by telephone-based field staff, may also be ripe for researching over the Internet, claims Manolopoulos. Beyond this, things start to get difficult.
But NOP, like other research companies in the UK and US, is also looking to compile lists of Internet users identified through traditional research techniques, who might be used as the backbone of a properly selected panel of online consumers in the future.
NOP is already conducting regular surveys aimed at establishing the pattern of Internet use in the UK. The by-product of this rolling research programme is that, among the thousand people identified as Net users and interviewed in detail every six months, up to three-quarters offer to be interviewed again.
“We have done about 16 online research projects, but all have been targeted using a database of Internet users,” says Manolopoulos. “We have not set this up to construct a panel, but it gives us the opportunity to generate a database of Internet users we can recruit.”
Currently, each Internet user survey by NOP involves screening 25,000 respondents before the figure is reduced to 1,000 regular Internet users. Three-quarters of these may agree to be re-contacted by telephone, or they may be asked to answer questions online at an NOP or client Website in the future.
“To get a proper panel, we have to go to some trouble,” says Manolopoulos. And, one presumes, some expense.
But David Walker, director of Research International, insists the Internet can now be used in all sorts of ways by market researchers, if the constraints surrounding it are recognised when it comes to drawing
conclusions.
“The Internet is a great way of identifying subject sampled or subject specific. For example, you can visit sites like MG sports car chat areas, where you get good and quite specific samples of product enthusiasts,” says Walker. “There are problems if you want to get a truly representative sample, but the Internet is a cheap way to do research, if you recognise that you are losing something in the area of representation.”
The fact that the Internet audience is highly skewed towards upmarket young males – an imbalance which makes it difficult to research many packaged goods online – may be an advantage for market researchers and clients attempting to investigate this target group, says Walker.
“If we get things right, there will be big cost-savings to be made,” he adds. “We will still have the same costs involved in setting up the design of the programme and the questionnaire, but the cost of field work can be significantly reduced – sometimes to virtually nothing. In essence the executive costs are the same, but the field work costs drop out when doing Internet research. And on a typical market research project costing, say, 8,000, field work costs can be a pretty hefty chunk of that total.”
Even though problems of conducting quantitative research on the Internet continue to hold back growth in the sector, Walker, Collins and others believe its potential to conduct “desk” and qualitative research should not be ignored.
According to Collins: “The important thing to remember about the Web is the amount of information just waiting to be picked up online – so if research is only focused on what clients and competitors are selling, profitability levels of companies and so on, it’s already an invaluable tool. And often, in the area of business-to-business research, it is this sort of information which is involved.”
Both Walker and Manolopoulos also mention the possibility of testing TV advertising and product design executions online, once graphical software and improved bandwidth allows for easier downloading of high resolution graphics. But if technical advances are occurring at lightning speed, the market research industry is reacting to the possibilities of opening up on the Web at a far slower pace, says Walker.
“There’s a lot of hype about the Internet from research companies, but there are not that many real projects going on,” he says. “The thing holding it back is that many clients are still unsure of it. A lot of market research buyers and old school agencies are wary.”
But a little caution is no bad thing, says Hutton, particularly when many IT-based firms are now offering to bypass established market research agencies, by cutting corners or ignoring the expensive recruitment techniques involved in conducting properly controlled market research.
“When you do market research, you simply have to control the sampling methodology. Otherwise, it becomes the Wild West. But some clients always want what’s cheap,” says Hutton.
“We should be making a stand. As research professionals, it’s no use saying ‘it’s cheap and it’s coming – so we might as well all join in and do things on the cheap for clients because otherwise we’ll be undercut’,” he claims.”Accepting that approach is a bit like selling missiles to Iraq on the basis that if we don’t, someone else will.”
