Social media analytics and measurement has rapidly emerged as a market over the last few years, and there are a lot of companies trying to carve out a niche (including my former employer, Scout Labs). It’s a huge problem space – a composite of many problem spaces really – but they all begin with the need to find relevant conversations (blog posts, consumer reviews, tweets, videos…) happening out on the web.
Before you can do the sexy bits – salience and sentiment analysis, influence tracking, visualizations, etc. – you need to find the stuff.
Very broadly, the challenge of finding stuff breaks across two kinds of topics:
First, there are very specific topics having to do with, say, a particular company or product or person. You’re concerned with every mention and how they add up. As a brand manager, you want to see what people are saying about you and your competitors, to understand whether they are net positive or negative and why. As a product manager, you want to understand what people like or don’t like about the stuff you (and your competitors) are selling. As a PR manager you want to gather and organize mentions. As a customer service exec, you want to nip problems in the bud.
The second type of topic is much more general. You want to follow everything March Madness, not just the news and scores for your alma mater and their arch rival. As a marketing manager or product manager, you want to understand where the market is going.
Structurally, these aren’t necessarily very different. Part and parcel to following a market is following the specific people and companies who make up that market.
Other issues with finding conversations are more tactical. There is the question of tools, and the question of where to look.
You can simply search. Google is getting better and better, returning results from every kind of source. There’s Google Blog Search too (and Technorati) of course, plus Google Video Search, etc. And there are the specialized search engines, like Twitter Search (formerly Summize) and sites like Blinx (videos), Swotti (consumer reviews), Boardreader (forums), etc. There are so many places people are talking now, and they just keep multiplying.
One problem with search, though, is that it’s a pull medium. You have to actively do it – although tools like Google Alerts will push the results to you; Google Blog Search and Technorati also let you subscribe to your searches via RSS. Still basically pull though. Another problem with search is that the ranking systems don’t care about what’s new. They are mostly concerned with relevance.
Another option is to curate your own list of sources. You can subscribe to the relevant companies’ own blogs, and those of the relevant pundits, etc. Then you can easily aggregate all your searches using Yahoo Pipes and even do some simple processing (like de-duping) of the results. A number of sites have emerged to gather and curate the good stuff for you. popurls republishes content from a number of popular blogs and websites. Original Signal does the same and organizes it across a few broad categories like Technology, Business and Entertainment.
One problem with the curating approach is that you get a lot of noise. The pundits talk a lot but rarely mention the specific companies or products you’re interested in following. This is fine for people interested in general topics, but not as good for brand managers and PR people who are interested only in actual mentions.
Finally, some sites combine search and curating into pretty powerful solutions. LOUD3R’s search is very good. You can take this strategy further on your own to create a pretty mean buzz-watching tool by using these as a starting point. Here’s how:
Subscribe to any number searches you perform on Google Blog Search, Technorati, Twitter Search, popurls, polymeme, Digg, del.icio.us and any other RSS feeds relevant to your topic. Aggregate the feeds using Yahoo Pipes, de-dupe them and sort them in descending chronological order.
Maybe I’ll do a more detailed post on that last bit another time.