It was Mark Twain who said, "Get your facts first, and then you can
distort them as much as you please."
These days web sites will go to unseen lengths to get ads. So data about
who a site reaches, how often it reaches them and what happens when
users go there is being dressed in more ways than a supermodel at a
For example, late last year Britannica.com counted four million unique
users by its log files. Nielsen//Net ratings gave it 2.9 million. Media
Metrix gave it 1.5 million.
Now the internet, which set new standards for advertising
accountability, is again facing its problem of site traffic
accountability. It has lacked a third-party auditing force, but that's
about to change.
The Internet Advertising Bureau (IAB) has retained
PricewaterhouseCoopers to complete a "process audit" of how impressions
and audience are delivered among its members. And firms who have made
their reputations auditing offline media are suddenly finding new
business on the internet. The reason is pure and simple: offline brands
want online audits.
"I think a lot of people became distracted from this issue by the
economic factors that have hit the business," said Dick Bennett, VP of
Audit Services at ABCi. "Advertisers want a complete evaluation of their
ad campaigns and that comes from verifying site traffic. When offline
brands invest in an ad campaign, they want sophisticated, third-party
post-campaign analytics. We think we're well-positioned to deliver
ABCi's parent company is the Audit Bureau of Circulation. It is the
accepted source of print media audience measurement and has trusted
relationships with offline publishers and offline brands. Bennett says
he has seen an appreciable uptick in business from both these groups in
the past three months. Several big brands have hired ABCi to execute
online ad campaign audits.
ABCi will expand its efforts to measure site traffic on a
campaign-by-campaign basis, as well as going after contracts with
individual site publishers. Email audits are another area that Bennett
says needs more verifiable third-party results.
These developments put an old issue back on the radar. Namely, the
discrepancy between the audience figures calculated by panel-based
measurement firms, and data from the sites themselves.
In the fall of 1999 an IAB panel, commissioned to study the issue,
claimed the difference between panel-based and log-file figures ranged
from 10 percent to 315 percent. The problems stemmed from what media
owners said were statistically inadequate samples and unscientific
Log files measured and collected from the media owners' servers
contained too much impure data, such as international traffic and
automated tracking bots, to be a gold standard. And sites serving niche
audiences, such as Bolt.com, claimed they were being underrepresented
even more severely because panel-based methodology was not allowing
young internet users to be counted.
Back in late 1999, when the NASDAQ was over 4,000, traffic measurement
was a nightmare anyone would have taken over the current market
The initiatives undertaken to rectify the measurement problems were
jolted when the NASDAQ crashed in April 2000, then shelved as many sites
became more concerned with survival.
But for those still in business, the issue is important once again. The
reason is still the fact that traffic data can win over advertisers. If
you don't have a growing audience, you won't attract the big offline
brands currently driving online ad revenue. And if you feel your site's
audience is being underreported by Media Metrix or Neilsen//NetRatings,
that doesn't help your case.
"It's not a good thing," says Bolt CEO Dan Pelson. "I know Media Metrix
still under-reports our traffic, for the reasons they underreported it
in the past. The logical way to solve this issue is to involve a
While Pelson says the discrepancies still remain, he and others say the
reliance on panel-based measurements has changed. A Media Metrix rating
used to be critical to Wall Street analysts. Now they're more concerned
with monetization of content and customer-relationship tactics. And
advertisers have learned that log files are an acceptable form of
measuring an audience, without being a perfect form. They're more
interested in individual demographics and how a site's audience can be
Still, panel-based proponents say their methods have improved, and make
a strong case. For example, Nielsen//NetRatings' principal analyst Tim
Kelly says the company has increased its sample size from 7,500 to
70,000 active internet users over the past two years. The participants
are chosen via random digital-dialling and have Nielsen tracking
software installed on their PCs.
"It's the best way to measure the user perspective," Kelly says. "It's
the best way to measure demographic information, because log files can't
give you that. And it's the best way to measure the competitive
situation. Yahoo! can measure its own log files, but the only way it can
see how it measures up to MSN and AOL is by checking a panel-based
Actually the debate here is not about using one method over another.
Each proponent of panel measurement can now see the value of log
And Bennett at ABCi will need both in order to be an effective
Perhaps some ad dollars are in the balance.
"This is a very important issue," says L90 CEO John Bohan. "We need a
reliable source of auditing. We take a lot of data from log files, and
there are issues with that, unless a third party looks at it."