March 23, 2017
On Wednesday, I presented a talk at IBM Amplify in which I explained the need for human analysis in social listening which produces what I call conversation research. I’ll admit the opportunity was intimidating. My task was essentially to look IBM executives, developers and users in the eye and say the social listening tool fueled by the famous Watson cognitive learning software was not very good.
But it’s not just Watson’s attempt at social listening that has issues. It’s all of them. Three of our most recent projects at the Conversation Research Institute tell a disappointing story. When we program these listening platforms to go find relevant conversations, we should see a respectable amount of just that in return. We don’t.
For our Dirt Devil project, we only scored 8.9% of the total posts our social listening software returned as relevant. It was worse for our industry report on the senior living space with only six percent of the results being the voice of the consumer. A brand study we did for a major healthcare company returned just 7.2% relevant results.
And we’re talking about three different social listening platforms. No one we’ve tested scores any better than these numbers.
What this means is without human analysis, scoring and curating of your social listening data — which is time and resource intensive — you’re paying for a lot of crap. And the technology is only getting incrementally better. It’s not growing by leaps and bounds the way the sales people tell you. Even Watson and IBM’s powerful engines have trouble weeding through and deciphering unstructured data like social conversations.
The truth is that when the data is unstructured, inconsistent and unpredictable, cognitive technology can only do so much. At least so far.
In the hopefully not-too-distant future we’ll be able to say, “Watson, find relevant consumer conversations about Dirt Devil vacuums and tell me the themes that surface around product problems,” then see meaningful results in seconds. But that day is farther off than you think.
In the meantime, CRI can help. Let us know how we might help you separate the signal from the noise and deliver consumer insights that help drive smart marketing decisions for your brand.
NOTE: Photo by Tim Moran, one of my fellow IBM Futurists.