Witness comments on Mashable's 5 Key Benefits of Monitoring ("I don't think that there are any monitoring tools better than Google Alerts and Tweetdeck.."), or from a UK agency ("Few would question the dominance of Google in search engine terms").
In our own work we have had clients tell us repeatedly that have reached a satisfactory level of monitoring via Google Alerts. The perception is Google Alerts is a good compromise between power, price and simplicity (see this, this and this).
Our position has been that Google Alerts is largely ineffective as a brand monitoring tool, no matter how free, simple or Google it is. Since no position should go unwarranted, we compared Google Alerts against our preferred brand monitoring tool: Brandwatch.
We setup a Brandwatch query and a Google Alert to both monitor the simple search "gatorade." From October 4th through the 19th we collected data, and after some time to work through our analysis we present our results below. Section 1 is on query writing, section 2 discusses user dashboard features and data accessibility, section 3 reviews our analysis of coverage, speed test and mention quality and our conclusions are in section 4.
1. QUERY WRITING
Query writing is an easy win for Brandwatch and a loss for Google Alerts. Obviously one can write as simple a query as they desire in both systems. For the data part of our comparison we used a one-word search query. Where Brandwatch shines is its ability to move beyond simple Boolean queries ("AND", "OR" etc) to more complicated proximity searches. For example, "gatorade NEAR/5 sports" finds all instances of "gatorade" within 5 words of "sports". Brandwatch queries can also be written to search for particular domain names and page titles, among others (here is an excellent blog post on query writing and Brandwatch’s capabilities).
As an example, lets consider the new "G" branding of Gatorade. Searching for "G" (or better yet "g") presents obvious difficulties in that people’s usage of "G" the letter will very rarely have anything to do with a statement about "G" the brand. Brandwatch allows us to build a query that puts "G" in context, reducing the otherwise huge number of irrelevant mentions (such as you would get through Alerts): "(g OR g2 OR gatorade) NEAR/4 (sport OR sports OR athlete OR drink OR drinks OR …)".
2. DASHBOARDS AND DATA HANDLING
Brandwatch has an accessible browser-based dashboard with a variety of built in features which allow you to slice the data as word clouds, lists of domains, site type, mozRank and query and custom topics volume over time. Here is an introduction to Brandwatch’s interface.
One of the largest issues with Google Alerts is the inability to manipulate data that has already come in, leaving Google Alerts a poor brand monitoring, research and response tool for anything that hasn’t just happened.
One of the best aspects of Brandwatch, the ability to simply download all the mentions of your brand into one large excel file, is also conspicuously absent in Google Alerts. To obtain the results below we had to use a heavily modified Greasemonkey script (original script). All we were able to extract after several hours of tweaking was page title, date tracked and URL. Even with the script, the process was by no means automated: the extracted data had to be copied out of a web page, pasted into a .txt file and have various "Find and Replace"s run on it (one delimiter we ended up being forced to use: |@|#|$|) before it was fit for Excel. Again, we could not over-emphasize the "just works" aspect of downloading data from Brandwatch.
3. COVERAGE, SPEED TEST RESULTS AND MENTION QUALITY
Our results cover three topics: breadth of coverage, timeliness of tracking and mention quality.
|Breadth of coverage|
|Total domains tracked||3965||684|
|Avg. mentions per domain||3.1||1.3|
|Avg. mentions per hour (over 16 days)||32||2|
|Domains tracked by both Google and Brandwatch||347||347|
|Brandwatch domains tracked by Google||9%|
|Google domains tracked by Brandwatch||51%|
|Top Brandwatch Domains||Mentions|
|Top Google Domains|
From the above chart (Figure 1) you can see that Brandwatch was tracking nearly 14 times as many individual mentions across nearly 6 times as many domains. This staggeringly larger amount of data demonstrates the limitations of the tracking aspect of Google Alert’s proposition as a brand management solution.
Google’s Twitter coverage was surprisingly lacking, with Brandwatch registering 5390 mentions and Google Alerts registering 7.
Both tracking systems exposed holes in each others’ coverage. Brandwatch registered mentions from 51% of the same domains as Google, while Google only registered mentions from 9% of the domains Brandwatch covered.
The chart below (Figure 2) highlights the top domains (by volume of mentions) that only one tracking system managed to cover:
|Domains not tracked by each site|
|Top Brandwatch domains NOT tracked by Google||Mentions|
|Top Google domains NOT tracked by Brandwatch|
Timeliness of Tracking
From a sample of 280 mentions, we were able to establish how fast Brandwatch and Google Alerts tracked the same mention. We had to discard many mentions to maintain a solid data set – either URL or page title didn’t exactly match.
|Mentions by timeliness||%|
|Brandwatch earlier than Google||11|
|Brandwatch within 24 hours of Google||76|
|Brandwatch within 48 hours of Google||99|
|Brandwatch within 5 days of Google||100|
|Average lag for Brandwatch||16 hours|
The above chart (Figure 3) shows that Brandwatch was able to record and process 11% of the 280 matching mentions before Google was. Brandwatch was able to capture 76% of mentions within 24 hours after Google had tracked and processed them, and only 1 mention took longer than 48 hours after Google for Brandwatch to process.
On average, Brandwatch was about 16 hours behind Google. While Brandwatch was slower, remember it was also tracking and processing 16 times more mentions per hour than Google Alerts (Figure 1).
Finally, the chart below (Figure 4) highlights the list of domains across which each tracking system established its competitive advantage in terms of speed:
|No. of days each site was ahead|
|Domains Brandwatch beat Google on||No. of days ahead|
|Domains Google beat Brandwatch on|
Interestingly, Figure 4 shows that Brandwatch’s overwhelming competitive advantage was across the blogspot.com domain (a Google property).
To compare quality we reviewed a sample of mentions from Google Alerts and Brandwatch. We randomly selected 200 mentions from each tool and categorized them as either spam or not-spam.
Out of 200, Google Alerts brought in 30 spam mentions (15% spam); from Brandwatch, only 18 (9%) were spam.
While this was an impressive difference, we want to clarify the extent to which we can judge based on these results. First, these are samples and as such there is always a probability that they do not accurately reflect the actual percentage of spam captured by each tracking tool. Second, from brand to brand the percentage of spam each tool will inadvertently collect probably varies a fair amount.
The conservative bottom line is: Brandwatch’s built-in spam filter was at least as good as Google Alert’s filtering. Brandwatch closes the deal on spam filtering by allowing one to do additional filtering via its more advanced query language.
From a personal-use perspective, Google Alerts allows us to keep a casual eye on many topics simultaneously due to its ability to support an unlimited number of free queries.
While Google demonstrates the power of its crawlers in terms of raw speed, the inevitable take away is that Google can’t possibly be considered an effective means of truly discovering what is being said online about a brand. With varying completeness of coverage across domains, Google Alerts reveals its inability to deliver even a representative sample of the conversation.
Of course one argument is Google Alerts is free while Brandwatch is not. Given that Google Alerts delivers a small unrepresentative fraction of the total volume of conversation around a brand, the question of whether to use Alerts on the basis of price is resolved by the simple statement: there is no such thing as a free lunch.
Again, Google Alerts is good for only the most casual of analysis – monitoring your own name or your favorite sports team. Building branding tactics and strategies off of Google Alerts research would be risky at best, dangerous at worst.