Working out the system

WORKING THE SYSTEM

A bunch of anoraks,” was how one speaker at a recent European seminar on the use of IT in market research described the audience, notably non-interactive when it came to any form of communication which was not electronic. And when considering the use of the Internet for research, most companies are painfully aware of just how limited the scope of any Internet-user sample is likely to be.

One research company director described a self-selected sample of Internet users as being “about as representative as a sample of people writing letters to the editor of The Times, with very strong opinions on a number of subjects. The silent majority would simply be pushed out”.

This is only one element – though perhaps the most important – in the distrust that research companies still feel towards the use of new technology as a research vehicle. But there are advantages, notably for employee research over Intranets, which make many clients keen to use the new media. And with an eye on the future, many researchers want to be sure they have the expertise and technology ready to respond to growing PC ownership and Internet use, as potential sample groups become more representative and research areas correspondingly wider.

“Even though the number of people using the Internet is relatively small, potentially you are talking about a huge multinational sample, easily and quickly accessible,” says Alex Authers, research director at strategic marketing consultancy New Solutions. “It would be highly applicable for, say, research into computer magazine use.”

Areas directly relevant to the Internet and PC-user, including hardware and software, are obviously legitimate research targets with this type of sample. NOP Research Group has introduced Web Check, a technique for checking the effectiveness of sites on the World Wide Web, whether operational or still undergoing design. A representative, specially recruited panel is used, with an online questionnaire completed and sent back to the NOP server. Yellow Pages is among the clients which have successfully tested the system.

NOP sees several benefits in using the Internet for qualitative research. Quite apart from the removal of geographical constraints, it allows access to elusive respondents such as senior business people, who can be retained as a panel without them ever having to leave their desks. It also means that sensitive topics can be explored, says NOP, and makes transcripts immediately available.

Voluntary questionnaires had been tried as an alternative to Web Check, says NOP, but response rates were exceptionally low. Even where response is higher, there is no way of checking its quality, says business development director at Marketing Direction Ricky Baxter. The good research practice of back-checking a percentage of respondents is difficult to envisage with the Internet, says Baxter, and yet many of the leading market research companies are using the Net as a medium. He and other concerned parties in the industry have approached Esomar, the European professional body, with a view to developing guidelines for its use in market research.

The danger, says Baxter, is that data is being gathered for reasons other than bona fide market research. “We’ve got to differentiate what we do from simple list building. Our business is an ethical one, but there is tremendous scope for misuse of the Internet in data collection by people purporting to be doing research,” he says. This can be seen as a logical extension of the way that selling under the guise of research – this time with the telephone as the medium – tarnished the image of market research a few years ago.

For qualitative research, there is no problem once you have established a reliable panel, as with the NOP Internet site product. “Once that happens, there are certain advantages over other techniques,” says Richard Jameson, director of international research at NOP Business. “You cut out the interviewer, so there are cost benefits. People are likely to respond to an e-mail, and if they don’t respond immediately, you know they’re not likely to respond at all.”

Effective research over interactive TV is another holy grail being actively pursued by many within the industry. NOP was one of the partners in the Cambridge Cable TV trials. This involved continuous monitoring of households’ use of the system, recording every button pressed, and converting this into statistics on viewing behaviour. Results showed, for example, that the total amount of TV use did not significantly increase with access to the system.

For many, the real interest in interactive TV will come when digital systems make the Internet available to households without the prior requirement – up to now a major limitation – of PC ownership.

Researchers have found that respondent interest is often reinforced by the novelty value of the new media, even if this effect cannot be expected to last. When BT carried out its Communication Audit Research for Employees (CARE) survey last year, via e-mail, feedback from participants was very positive. “Good idea to do it electronically. It’s clear and easy to use,” said one. “I am surprised by the quality and ease of use,” said another.

The electronic version of the questionnaire, created by interactive technology provider IML, aimed to build up a picture of what employees worldwide thought of BT as an employer. It was delivered on e-mail to 265 workers in mainland Europe and Australia. Crucially, the system guaranteed confidentiality, emptying the completed document – which included respondents’ own comments – encoding the data and automatically returning it to the mailbox, with no possibility of it having been read by anyone.

“The key element in our proposition,” says IML sales director Peter Knowles, “is that all the different levels of information collected within a large company, from staff questionnaires to board meetings, should be compatible, and can be related to other data such as customer survey work.”

Significantly, a key area of research for IML and many other technology providers, says Knowles, is neural processing, aiming to move beyond the data collection stage to analysis and pattern-identification, filtering and pushing information to the system user on an intelligent basis.

International staff questionnaires benefit from being able to offer simultaneously collected information, and technology can help consumer research in the same way. McDonald’s has carried out large focus group tests in different states in the US, with simultaneous viewing to pre-test TV advertising.

A handset with dial to register instant response allows real-time processing of data for an ever closer focus on sections of the video in a single session.

New technology also has a part to play in focus groups, even when they are held on a single site. Market research company Feedback Consumer Consultancy used IML’s group response system to gauge consumer feelings about three pack concepts for Sara Lee Household & Personal Care. At first, 15 respondents in each of two groups rated a design on a scale of one to seven, keying in the answer. Only after group discussion did the computer-collated results come up on screen, so providing a stimulus for further debate. This approach helps to even out the effect of having – as so often happens – two or three dominant individuals in any group, says IML.

“We found the system a very good tool to use,” says Sara Lee senior brand manager Dan Goldstone. “Especially to obtain instantaneous quantitative data, but with qualitative perspectives.”

BMRB International is pushing ahead with its Multimedia CAPI (Computer Aided Personal Interviewing) system for home interviews. Using a laptop computer, it bridges the gap between hall tests, where CAPI has been available before, and home testing, which had been restricted to telepictorials and story-boards. “The recall you get on advertisements with multimedia is likely to be more accurate – not necessarily greater,” says director Graham Wilkinson. With increased continuity between different advertisements in a campaign, respondents may be recognising one element such as a main character and not the advertisement itself.

“With hall tests, the quality of the sample was not terribly high. If you go into people’s homes you get a better quality sample, and you are seeing it in the right environment,” says Wilkinson. The system can isolate or exclude elements such as soundtrack or branding, says BMRB, and can also be used to record at the point of interview. This will give critical intonation patterns that a transcript would not record, says Wilkinson, and also allow easier client-access to responses on audio tapes.

But while the client is generally quick to see the benefits of new technology in collecting and collating data, this is not always true of the research team. “The fear has always been that these technologies are replacing some of the mystique and magic in market research,” says IML’s Knowles. “The truth is, they are simply tools for these companies to use.”

Some of the major advantages of new technology – the time and cost savings that it brings – are not always appreciated by the people who use it. As one researcher and user of electronic systems put it: “We used to bill the client for hours spent collating data, and now it’s all done in a millisecond.”

To a large extent, it appears, this technological revolution will have to be customer-driven, and it will be up to the market researcher to find value elsewhere.