Is sentiment analysis useful?

Nearly all social media monitoring tools offer sentiment analysis, but is it actually useful? The short answer is yes, but only if you get smart with the data.

During our recent social media monitoring webinar, Matt Rhodes from Fresh Networks argued that in order to gain actionable insights from sentiment you need to look beyond the basic level. Citing the number of positive and negative mentions isn’t good enough. Instead, you should break the data down into topics and then keep on drilling.

For example, if you manage a restaurant chain you will want to know a range of different things such as; What are people saying about the menu? Did people enjoy their food? Was the service friendly? Were people served promptly?

Once you have drilled into this data, you can break it down further still. What do people think about the breakfast menu? What do people think about your spaghetti carbonara (or anything else you happen to serve)? What did people say about the service in one part of the country and how does this compare to others?

The problem is that the more you break down the data, the less likely it is that automated analysis will get it right. Automated sentiment works best with large amounts of data and can’t be relied upon for smaller samples. As Nathan Gilliatt pointed out, a lot of humans struggle with sarcasm and irony, so how can we expect computers to cope? This is particularly problematic when looking at Twitter.

There are also examples of posts which contain both positive and negative sentiment. For instance: “The food was fantastic but the service was terrible”. In this case a computer won’t know which way to turn.

Despite these limitations, Leon Chaddock from social media monitoring company Sentiment Metrics argued that automated sentiment can be a useful tool:

“The plus side with automated analysis is that it is fast and close to real-time. It is very good at is spotting high level trends and extreme swings in sentiment such as swearing and complaints which makes it great for social customer services.”

Another benefit is that automation is less costly than having human analysts manually coding the sentiment, but the ideal solution is probably a mixture of both.

A decent monitoring tool will let you manually override sentiment. This means you can be instantly alerted to high-level trends and can quickly respond to particularly positive or negative mentions, but later human analysts can look over the details and check the accuracy when producing insights reports.

Undoubtedly automated sentiment analysis will continue to improve over time and Leon even suggested that soon we will be looking at intention analysis rather than sentiment. It will never be 100% accurate, but as long as you think carefully about how you use it you will see the benefits.

I’d be interested to hear other people’s views on this. Do you trust automated sentiment? Is it useful? And do you use human analysts to check it?

You can read the full webinar write-up here and listen to the recording here.

Read Next

In this article

Join the Conversation

3 comments

  1. Is sentiment analysis useful? | Our Social Times | digitalnews2000 Reply

    […] on oursocialtimes.com Share this:TwitterFacebookLike this:LikeBe the first to like this. This entry was posted in […]

  2. Who won the #budget2013 Twitter battle? « Social Media Monitoring and measurement Reply

    […] course, this is looking at automated sentiment which is not able to account for irony or sarcasm. However, due to the large quantity of data […]

  3. Barry Schaeffer Reply

    I think we can gain valuable insight into how sentiment analysis is likely to work in today’s world, from, of all things, a personality test from the early 1940s.
    The Minnesota Multiphasic Personality Inventory (MMPI) was among the first pencil and paper attempts to determine the nature of an individual’s mental state
    It was designed originally to measure the mental health of people being treated or considered for therapy or incarceration, and has done a credible job in that endeavor. It is also used in an attempt to recognize personality traits and problems in otherwise normal individuals, but has not been as successful.
    The upshot of this comparison, I believe, is that like the MMPI, automated sentiment analysis will succeed with trends that vary significantly from the mean (neutral), but increasingly less so with trends closer to the neutral mean.
    This, I believe, underscores the need to use a mix of automated and human analysis for any trend that doesn’t stand out like a sore thumb.