Sunday, April 27, 2008

Mining Twitter for the Enterprise: Survey your Customers' Candid Thoughts using a Social Web 2.0 Service

One BEA customer recently posted on Twitter: "hating weblogic". Another said: "F@#% you weblogic for not handling HTTP 500 errors properly."

Whether you personally participate in one of the numerous internet social platforms or not, some of your customers are out there. WebLogic isn't the only brand being discussed - name the brand and there are customers voicing their opinions on these platforms.

This blog entry focuses on Twitter, and shows how you can mine Twitter for feedback on your brand using a tool called TweetScan. Because of the very informal nature of Twitter, and it's low barrier to participate, it is a good channel of candid opinions on your brand. Companies like Comcast have already used TweetScan to identify and remedy customer relationship problems in a highly proactive manner. How is your brand perceived on Twitter?

 NOTE: this blog entry was originally posted April 27th, 2008 on my previous blogging system (

What is Twitter?

Twitter is a service that allows you to publicly post messages in small doses - 140 characters or less. The basic idea is to answer the question What are you doing? but you can use those 140 characters to say whatever you want.

While blogs are a great source of public opinion, they tend to be more filtered because they require more effort to produce. Twitter lowers the barrier to post dramatically, and therefore is a better medium for capturing those minor annoyances and random thoughts we all have throughout our day. There is also less of a imperative to post something interesting - your readers only need to invest the time to read 140 characters and not paragraphs of a blog entry.


Me and Twitter

I am a reluctant poster to Twitter. Do people really care that I just ate dinner or that I am about to watch a movie? Not really, apparently, as I don't have too many followers. Granted, I haven't been at it that long, but is this time well spent?

Is participating in Twitter worth your time?

I really don't know, and certainly am not an expert. But I suspect it depends on how you view Twitter specifically, or the other social platforms in general. Are you using it to get something done, improve your social life, for pure entertainment, or what.

Sarah Perez  talks through the issue in her blog Real People Don't Have Time for Social Media, and an e-consultancy article "Social media's inconvenient truth" expands on it.

   "But there's an inconvenient truth that proponents of these sites tend to ignore - the average person just doesn't have the ability to participate fully in social media."


TweetScan - Improving the Signal to Noise Ratio for Observers

The silver lining is you don't have to participate to acquire great value from these social computing tools. Lots of people are using Twitter as a medium for publishing their unfiltered thoughts. As simply an observer, you can use these platforms to great advantage. But how?

A problem that cannot be ignored is the amount of noise on these platforms. While Twitter is a wealth of information on customers' opinions and comments, much of this is chaff - "sitting on toilet listening to lily Allen". But buried within the mountain of irrelevance are some nuggets of useful information - "Angry at BEA. Why doesn't WebLogic play nice in OSGi?".

With just plain old Twitter, its impossible to separate the crud from the gold. TweetScan is the solution - it is a search engine built on top of Twitter to help you find the diamonds in the rough, especially when you are focused on a searchable keyword like your product brand.



Monitoring your Customer's View of your Brand

Savvy marketing and customer relationship people have started to take advantage of TweetScan to help manage their PR and to capture raw customer data. Michael Arrington's adventure with Comcast is the most famous.

Michael Arrington was having trouble with his Comcast internet account. Calls to their service center didn't provide a relief, and so he fired off some Twitter posts in anger. Being who he is, others picked up on this and triggered a fire storm of net activity. Then something wonderful happened:

"And this brings me to the point of this post. Within 20 minutes of my first Twitter message I got a call from a Comcast executive in Philadelphia who wanted to know how he could help. He said he monitors Twitter and blogs to get an understanding of what people are saying about Comcast, and so he saw the discussion break out around my messages."

As far as PR and customer relationships go, this is a fantastic show of force by Comcast. But not everyone who posts to Twitter is a famous net personality like Michael. It is fair to assume that Comcast cannot afford to provide the same level of response to the average joes out there. At least one on one. Instead, Comcast can monitor the Twitter channel and detect general patterns in service problems. Michael sums up how this can benefit Comcast (and its customers):

"Well before most people they have identified blogs, and particularly Twitter, as an excellent early warning system to flag possible brand implosions."

A few more references to using Twitter like this:

  • Blip.TV - Blip.TV used TweetScan to eavesdrop on its customers and detected and resolved a service problem using this approach.
  • Using Twitter as a Research Tool - Krishna De talks about using Twitter derived information for research.


WebLogic on TweetScan

The brand I care about is WebLogic. As an architect on WebLogic Portal, I am heavily invested in this brand. Up until now, the primary way we received feedback on the product have been these channels:

  • Monitoring the WebLogic newsgroups
  • Customer support cases
  • Direct interactions with customers

All of these have been effective, but it is interesting to see how TweetScan can provide a new channel for feedback. Here are some selected entries in the current search for "weblogic" on TweetScan, with a focus on the negative posts:

We see negative comments on the newsgroups. But these Twitter posts are far more unfiltered and more of the flavor "how do you feel about weblogic", as opposed to the newsgroups/support cases which are "what is not working for you with weblogic". Now, most Twitter posts by definition don't have nearly enough information to really understand the underlying issue. But you get a flavor of what your customers are feeling, and you can always follow up for clarification by asking the user to clarify offline.

And to be clear, this list is fairly typical of ANY brand you search on. Pick a brand, and do the search, and you find a litany of complaints and negative comments. This is normal. Check your pride at the door before searching on your brand. Twitter posts on brands usually err on the side of being flames, not praises.


Comments from original blog:
  • Jott - looks useful. Notice they have a "custom service" capability where you can give it a server URL to HTTP POST the messages to, instead of predefined endpoint like Twitter. I love open systems.

    Posted by: plaird on April 30, 2008 at 10:39 AM

  • Excellent post, Peter, on an important topic.

    As people become increasingly frustrated with voice mail systems and email auto-responders when trying to report problems to companies, tools like blogs and Twitter will become the means by which honked-off customers express their frustrations. That trend will grow, and with it the need for companies to get proactive, pay attention, and track these sites and services in order to manage customer satisfaction and public relations.

    BTW: In conjunction with Twitter, I also use Jott, a free service that transcribes voice messages to text, and pushes those messages out to my Twitter account, or to one of my blogs, or directly to an email to a contact -- all based on how I configure the service. Cool stuff: I just dial the Jott number, use voice commands to select the message destination, then recite my message. Unless there's a lot of background noise, the transcription is pretty good, and it's a lot easier and faster (for me, anyway) than trying to type a Twitter post on a tiny cellphone keypad.

    Posted by: brhubart on April 29, 2008 at 1:38 PM

  • Someone asked offline what my twitter handle is so they can follow me:

    Follow: plaird

    Posted by: plaird on April 28, 2008 at 8:41 AM

  • I am no Michael Arington - by any stretch of the imagination. I'm just a guy. Last week I had a similar experience with twitter based support from SlideShare. The best thing is I sent out just 1 twitter message in frustration and within 10 minutes, 2 employees of SlideShare contacted me via twitter to give me the low down on the problem I was having. I blogged the story under How Twitter-Enabled Customer Support Saved the Day. I was completely blown away and felt much more special than my individual status really warranted. Extremly powerful use of technology. Dennis Stevenson

    Posted by: denthewise on April 27, 2008 at 8:52 PM