During our recent social media monitoring webinar, Matt Rhodes from Fresh Networks argued that in order to gain actionable insights from sentiment you need to look beyond the basic level. Citing the number of positive and negative mentions isn’t good enough. Instead, you should break the data down into topics and then keep on drilling.
For example, if you manage a restaurant chain you will want to know a range of different things such as; What are people saying about the menu? Did people enjoy their food? Was the service friendly? Were people served promptly?
Once you have drilled into this data, you can break it down further still. What do people think about the breakfast menu? What do people think about your spaghetti carbonara (or anything else you happen to serve)? What did people say about the service in one part of the country and how does this compare to others?
The problem is that the more you break down the data, the less likely it is that automated analysis will get it right. Automated sentiment works best with large amounts of data and can’t be relied upon for smaller samples. As Nathan Gilliatt pointed out, a lot of humans struggle with sarcasm and irony, so how can we expect computers to cope? This is particularly problematic when looking at Twitter.
There are also examples of posts which contain both positive and negative sentiment. For instance: “The food was fantastic but the service was terrible”. In this case a computer won’t know which way to turn.
“The plus side with automated analysis is that it is fast and close to real-time. It is very good at is spotting high level trends and extreme swings in sentiment such as swearing and complaints which makes it great for social customer services.”
Another benefit is that automation is less costly than having human analysts manually coding the sentiment, but the ideal solution is probably a mixture of both.
A decent monitoring tool will let you manually override sentiment. This means you can be instantly alerted to high-level trends and can quickly respond to particularly positive or negative mentions, but later human analysts can look over the details and check the accuracy when producing insights reports.
Undoubtedly automated sentiment analysis will continue to improve over time and Leon even suggested that soon we will be looking at intention analysis rather than sentiment. It will never be 100% accurate, but as long as you think carefully about how you use it you will see the benefits.
I’d be interested to hear other people’s views on this. Do you trust automated sentiment? Is it useful? And do you use human analysts to check it?