By Graham Mackintosh
Social sentiment analysis is the new darling in the world of digital marketing and Big Data analytics. But while making sense of opinions posted publicly on Twitter sounds easy, it’s not. It’s a lot more complex if you’re looking at the meaning and tone of natural language conveyed by Twitter’s fire hose of 200 million active users.
Telemetry is the science of measuring data at a distance over communications networks. When put in this context, social media can be thought of as “human telemetry” – a virtual town square where we can understand our social interactions and preferences by analyzing everything that is said and shared via blogs and tweets.
Human telemetry applies to more than trending topics on Twitter. Consider healthcare. Within an urban center, regional hospitals already exchange real-time information – such as admission rates and bed-space data — to help with ambulance routing and finding the right type of doctor.
Including “human telemetry” data completes the picture. Making sense of the text in postings requires advances in text analytics and novel techniques to improve accuracy, even when people enter comments with slang, spelling mistakes, ambiguities and sarcasm.
Perhaps dozens of worried parents begin posting concerns via social media about their children’s severe flu symptoms, all within the same metropolitan area. This could be an important leading indicator for hospital capacity planners.
Moreover, if this pattern of social postings expands in volume and geographic scope, analysis will transcend any given neighborhood or city. It could provide deeper insight for pandemic detection for the World Health Organization or the Center for Disease Control, helping to identify and manage healthcare problems at and earlier stage, before they spread.
The very nature of social media means that the lines between systems and human telemetry are increasingly blurred. For example, many of us follow Twitter users who are not actual people. They are computer systems posting Tweets on useful information, from regional traffic updates to flight information.
Conversely, personal devices, such as smart phones, can publish the owner’s geographic location to other people in their social network. The phone owner thinks of that device’s telemetry as a part of his or her social media presence. Think of it as human telemetry in the palm of your hand.
This convergence and blurring of human and system data opens the way for new analytic applications. We will see apps that reduce congestion on roads by analyzing data from traffic sensors and insights gleaned from commuter blogs. Or apps that create more personalized retail offers that combine sales data with chatter from Twitter, blogs and comments.
This combined perspective of natural language and hard data will lead to deeper analysis that not only reveals trends and hotspots, but also uncovers the human element of why we do what we do.
We did this recently with the IBM Social Sentiment Index, which looked at traffic in a handful of cities in France, Netherlands, Spain and Germany. We found that in the city of Lyon, France, sentiment around rush hour is positive (40%) with essentially no negative sentiment. The city has embarked upon an ambitious project to build a more sustainable transportation network, which suggests that citizens are recognizing the efforts of the city of Lyon to improve systems there.
This shows how human telemetry can benefit society. It hinges on using big data together with advances in analytics and natural language processing. And to prove successful these applications must have reliable and useful information, such as sentiment, that can be extracted from social media content.
The innovation in text analytics will continue. We’ll see advances such as guided machine learning technologies for sentiment analysis that can be trained and tuned with examples to continually improve its accuracy. But more importantly, these advances can be used to make the planet smarter, where instrumented systems can gain insights into data that is truly interconnected – including “human telemetry” from social media.