May 2, 2017
We kicked off a client project today at CRI and spent a fair amount of time deciding on our coding plan. This is the approach we’ll use when reviewing the conversations found using social listening platforms to code the conversations.
What do you visualize when you hear that we “code the conversations?” For some, it means reviewing each individual post and slapping a topic on it. For others, it might mean double-checking data like sentiment or gender. For us, however, coding a conversation requires an understanding of the client’s product, services, competitors, marketplace and more; preliminarily reviewing a sample of conversations to understand what types of posts we’re going to be reviewing; anticipating what people might say about the brand or topic at hand and more.
Our Coding Plan gives us the list of unanswered questions about the data. Social listening platforms can find things like sentiment, gender, location and even parse out a topic or theme, but not to a level of certainty we are comfortable with, nor at a level we would recommend you make marketing decisions upon. So yes, we review the data collected for accuracy.
But our Coding Plan also determines what else we want to know about the data the social listening platforms do not provide. Take a look at this snippet from a spreadsheet we recently coded:
You see entries for Sentiment, Account Type and Gender, all of which can be automatically detected. Our first step is to verify the automatic detection worked. Frankly, sentiment accuracy in social listening tools is incredibly disappointing. Our estimation is that only about 10-20% of posts are even scored at all. About 30-50% of them could be. Gender is only detected about 20-30% of the time, but can be determined manually for about 60-80% of the posts, so there’s some heavy lifting to be done.
The rest of the scoring columns you see are categories of information we determined would be insight-fertile categories for the client in question. We read each post to understand the context of the mention. Was it a promotion from a re-seller or a recommendation from a customer? Did they use any emotions in their description of the product or service (not did a word that describes an emotion appear in the text, which is what social listening platforms present)? What feature of the product did they mention specifically? Was there a specific issue or topic about that feature that stood out? What was the use case of the product or service (in this case, what type of location)? And were there other use cases that emerged than what the product or service was primarily sold for?
Planning your coding means anticipating where you’ll find the most useful answers in your research. It’s the social media analysis equivalent of crafting the right questions in traditional market research. We like to think we’re pretty good at that part. Hopefully, this helps you get good at it, too.
If you’d like to see what conversation research can reveal about your business, customers, competitors and marketplace, drop us a line. We’d be happy to discuss it with you.