Finally: Proof that you DON’T need to measure social media ROI

Finally, after thousands of wasted hours trying to track returns and develop complex models, we have conclusive proof that ROI isn’t the primary measure of success in social media. A SmartBrief report, that somehow slipped under our radar last year, included a slide (below) which shows clearly that only 35%, little more than a third, of businesses that have been using social media for more than 3 years (i.e. the purple columns) actually bother to measure ROI.

For any CEO or finance officer who remains convinced that ROI must be measured in all circumstances, may this serve notice on your misconception: Return on Engagement (ROE) is the primary measurement of success in social media.

The fact that 65% of businesses that have been using social media for 3+ years have created a detailed strategy indicates that, far from being an accidental omission, ROI is being pushed out of the equation by design. If ever proof was needed to make a seemingly self-evident point, this is it. If your boss suffers from ROI-Fixation Disorder, please make use of this table in their treatment. You might just save their business.

Social Media ROI

19 thoughts on “Finally: Proof that you DON’T need to measure social media ROI

  1. Philip Sheldrake

    Here are some quick examples of when the pursuits of ROI or decisions made in the name of ROI become farcical. I’m sure many readers will have come across such situations.

    When the boss rules – ‘Look, I’m told we’re investing in this. Now we just need to work up the numbers to get it through finance.’

    When efficiency rules – ‘This investment will speed the process up.’ ‘Er, but it’s not actually a bottleneck.’

    When the guru rules – ‘Well the book’s at number 1.’

    When last year rules – ‘Well we did it this way last year . . .’

    When the competition rules – ‘They’ve gone for it, so. . . .’

    When vanity rules – ‘We can afford it and it’ll be a testament to our time.’

    When experience rules – ‘Do you think the CMO’s background in advertising sways the budgeting process?’

    When rules rule – ‘Let’s treat it as three separate projects so each comes under the limit demanding cost justification.’

    When paralysis rules – ‘I just don’t know.’

    And when all else fails:

    When cost rules – ‘Just make a decision on a least-cost basis because this sort of thing never has a tangible ROI.’

    [From The Business of Influence, Wiley, 2011]

    I prefer it when strategy rules Luke. You can secure ROI with the strategic alignment and discipline of the Influence Scorecard – an augmentation to the Balanced Scorecard, the world’s dominant framework for business performance management.

    Reply
    1. Andrew Bruce Smith

      Experience and expertise are over-rated. See Tim Harford’s new book – Adapt – for Tetlov’s research into expertise (ie experts perform more poorly than they ought to in determining what will work in the future):

      And Doug Hubbard has a good line on experience:

      Experience is a nonrandom, nonscientific sample of events throughout our lifetime. Experience is memory-based, and we are very selective regarding what we choose to remember. No matter how much experience we accumulate, we seem to be very inconsistent in its application.

      Reply
    2. Andrew Bruce Smith

      On the subject of the balanced scorecard, I spent 14 years handling PR for the biggest provider of business performance management software (Hyperion, subsequently bought by Oracle). In that time, I interviewed senior finance and business execs from probably 40+ large enterprises on BPM, balanced scorecard, etc. A common theme was that the theory of the balance scorecard was great, but the implementation of it in most cases was difficult – for a whole host of reasons (though mainly cultural rather than technological). I guess my concern for the Influence Scorecard is that it faces the self same cultural barriers faced by the balanced scorecard – perhaps even harder given the slippery nature of reputation and influence.

      Reply
      1. Philip Sheldrake

        Andrew, [no sarcasm] if you’re saying business performance management is a tough discipline then I agree 100%, but it’s the kind of rigor that separates the great from the good.

        Business performance management is relentlessly hard work.
        Ditto influence performance management.

        Reply
        1. Andrew Bruce Smith

          I think we are in violent agreement 😉 

          I’m not saying the Influence Scorecard is wrong – quite the contrary – you’ve eloquently and exhaustively detailed how such an approach can be implemented – but as per my views on measurement, just saying that if people think there is no effort involved, they ought to look away now.

          Reply
  2. Andrew Bruce Smith

    What exactly is return on engagement? For that matter, how are you defining engagement? And are there ways to determine the causal impact of engagement on business performance? And if so, can you share?

    Reply
    1. Luke Brynley-Jones

      Hi Andrew. There are different ways of doing this – but I measure ROE by working up a scale, specific to an organisation, for different forms of engagement. For example, a Like on facebook might indicate a less engaged person than a comment, a recommendation, or a referral (see the link for ROE above for an example of such a scale). You can ascribe arbitrary values to each form of engagement (1-5 in the example) or simply aim to move individuals from left to right, from less to more engaged. Once a person reaches a certainly level of engagement, they may become a customer and ROI comes back into play. Does that explain it?

      Reply
      1. Andrew Bruce Smith

        Hi Luke. Thanks for that. 

        A few observations.

        1. The use of the word “arbitrary” is a bit worrying. In other words, I can make up anything I like?

        2. Why a 1 to 5 scale? Why not 1 to 100? Or any other scale? Won’t this introduce scale bias? And don’t scales in any case produce unintended mathematical consequences? Isn’t it the case that only a minority of a scale is used for the majority of the ratings given? If opinion moves a little, then it can mean a big change in the standing of the alternatives? And the presumption of regular intervals. Scores implicitly assume that the regular intervals of the 1-2-3-4-5 scale approximate the relative magnitudes being assessed.

        3. Moving the individual from “left to right” – what scale is being used here? Is your “engagement score” simply the addition of a bunch of abitrary numbers? How do you determine the causal relationship between “engagement” and business performance?

        Reply
        1. Luke Brynley-Jones

          Yes – it’s an arbitrary scale to you and I, mainly because it’s specific to each organisation. We could use number (1-5, 10-10,000), piglets (ugly to cute) or shoe sizes as the scale. The client decides what matters to them most (again, not scientific), creates the scale and measures against it. They can set targets and peg these to their goals. I’m sure there are more rigorous methods – as Philip cites below – and I would urge those with an urge for rigour to read up on them.

          Reply
          1. Andrew Bruce Smith

            Isn’t it also therefore arbitrary to the client? And in which case, shouldn’t we be helping clients to become more rigorous and scientific in how we approach this whole issue? 

            If there is no observable connection between the measurement approach taken (whether ROI, ROE or whatever) and impact on business performance, isn’t this one of the reasons why senior execs don’t take social media (or PR and marketing for that matter) more seriously? 

          2. Luke Brynley-Jones

            Actually, connecting this up, so that engagement targets contribute to higher goals is easy when you have a scale to work with. But there are merits in measuring engagement independently too. An example, a client of mine is using engagement metrics to indicate when their “community” is ready (i.e. sufficiently engaged) to make a social media marketing campaign – which requires user engagement to succeed – worth running. Their Senior Execs fully understand the logic and value of using engagement metrics in campaign planning.

          3. Andrew Bruce Smith

            Sorry to keep harping on about this, but again, where is the connection between engagement and business performance?

            A client arbitrarily decides on an engagement scale or metric.

            As a result of this arbitrary engagement metric, client decides community is ready (which begs question how do you know how much engagement means “ready” – if the metric is arbitrary, then the success trigger is arbitary, yes?

            “Readiness” is the trigger for running a social media campaign (again – what is connection between readiness and decision to run a campaign? How do you determine that user “engagement” is required to succeed – because engagement is an arbitary metric?

            Forgive me if I’m being thick.

          4. Luke Brynley-Jones

            Let’s say your focus is Facebook. You’re getting 100 Likes a week (engagement level 1 on your scale) and 10 comments (level 3). You know that your planned campaign requires a degree of engagement that is much more than writing a comment – e.g. uploading a video. Let’s call this level 5. You also know that, to succeed, you need min. 100 people to get involved in the campaign. So you work on increasing engagement at each different level until you’re confident that you have a community that is engaged enough to participate in a more demanding activity. It’s not scientific, but it works. 

            [Let’s call it a day after your next comment 😉 I have baying staff and clients]

          5. Philip Sheldrake

            Luke, I don’t think you’re answering Andrew’s question. Katie Delahaye Paine has the same concerns as Andrew, and here’s how she puts it… Ask “So what?” three times.

            On what basis might you decide 100 per your comment above. Why not 50, or 922? Such metrics are close to the marketing and PR activity rather than close to the business. The chairman’s contribution to the company’s annual report never includes statements like: “And it was essential for us this year to get 100 people involved in our social media campaign” or “We invested significant resource this year in doubling our friends and tripling our ‘likes'”.

          6. Andrew Bruce Smith

            Yes. The annual report might say: “by investing £1m in generating 1m Facebook Likes, we have seen sales of product category X increase to £10m from £5m, with a commensurate increase in profit margins from 15pc to 20pc, etc.”

            Will be interesting to see how much space Pepsi’s Refresh Project gets in the 2011 annual report ie we spent $25m+ on social media and what did we get for it? (The 2010 report was titled: Performance With Purpose).

            http://adcontrarian.blogspot.com/2011/03/social-medias-massive-failure.html

          7. Andrew Bruce Smith

            “It’s not scientific, but it works”

            And I guess that’s my point. Presumably “works” in this context means “so long as the client is happy, who cares?”

            To quote Phil from his book: “The marketing and PR professions remain relatively unscientific” – that to me is part of the reason why PR/comms/marketing isn’t taken as seriously by business as it should be. Or put another way, if we don’t display more scientific rigor then we probably don’t deserve to be taken seriously.

          8. Luke Brynley-Jones

            I use engagement metrics, according to the fine teachings of Ms Paine, to measure campaigns that contribute to business goals. It’s as scientific as it needs to be to get successful results (e.g. reliable enough information to make the correct business decision). Anything beyond that is wasted energy. Maybe you can create a better scientific model – but if we’ve already got one that works well, “So what?”

  3. Pingback: Finally: Proof that you don't need to measure social media ROI | Social Media Monitoring Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

New posts by email

Connect with us

Subscribe to our blog

Subscribe via RSS  readers

Our services

Our Social Times is a social media agency, providing a range of high-value social media marketing, monitoring and training services.

Consultancy

Events

Training

Recent posts