OMG. Twitter is, like, so biased

Just how much of your brand reputation are you prepared to stake on readings you get from social networks like Twitter? Pressure to include social network analysis into marketing metrics and decision making is currently very strong. What other area of activity would dream of running a four-day conference in the current climate?

Yet almost all of the tools and techniques being deployed around SNA are in some way flawed. So any decisions about where to orient marketing, how to position the brand or even what products to introduce or de-list could be seriously misguided.

Let’s start with the Pareto Principle – it is at work harder on Twitter than on other social networks. A study by Harvard Business School of 300,000 random tweets (http://blogs.harvardbusiness.org/cs/2009/06/new_twitter_research_men_follo.html) found that 10 per cent of users are generating 90 per cent of content. On other networks, the ratio is more often 10:30.

It is not even that this top decile of Twitter users are particularly expert – on Wikipedia, 15 per cent of users generate 90 per cent of content, for example. So Twitter is attracting an extreme slice of highly active micro-bloggers.

If you have decided to use Twitter as a data source for brand reptuation monitoring, you risk being drawn into the slipstream of a highly self-selective group. And, of course, you have very little idea who these people are, either by standard demographics or attitude.

If looking at Twitter as a potential marketing tool, the HBS study did produce a surprise. On most social networks, men generally receive very little attention from other men or women. On Twitter, a male blogger is followed by another man in 58 per cent of cases and by a woman in 48 per cent of cases. Even more strikingly, men follow male bloggers in 66 per cent of cases and women follow men in 56 per cent.

That makes Twitter one of the few new media that reinforces the male voice – which may explain why so many marketing directors and business leaders are all over it. If your marketing plan had included tweets by a female director, then it might be worth thinking again.

Most social network analysis is being done by automated tools, however, with marketing teams getting reports on whether bloggers are positive, neutral or negative towards their company. Here there is even more to worry about, since studies into sentiment in blogs show that techniques are at a very early stage (even though it is often no worse than human performance on the same task).

An experiment by Gilad Mishne of the University of Amsterdam Informatics Institute into mood classification in blog posts found that grading of posts as positive or negative scored only 48.03 per cent accuracy on a small sample (800 texts), with active/passive classification performing at 50.51 per cent.

As the size of the sample rose, to 80,000 texts, so did the accuracy – but only to 59.67 per cent for positive/negative and 57.08 per cent for active/passive. Since the average size of a blog is only 200 words, that means most SNA tools will struggle to get enough data to work on, and even then they will be right only six out of ten times.

It is still early days for the incorporation of social networks into marketing, of course. But you might not want to shout too loudly in favour of them as a data source. Unless you are a man.