Close-Up: Live Issue - Is direct mail testing worth the effort?

DM testing has returned in a revamped, sophisticated guise. But is it practical, Claire Billings asks.

No matter how you say it, there are probably few phrases less likely to get agency pulses racing than direct mail testing.

Or so you'd think. In fact, testing is once again becoming a hot topic in direct marketing circles. The process has moved on from the old days of endlessly tweaking small details, such as whether a brown or a white envelope is most likely to induce a recipient to open it, or whether or not to put copy on the outside . This time, it's much more sophisticated.

Steve Aldridge, the creative partner at Partners Andrews Aldridge, explains: "More savvy consumers mean you need to recognise them through different creative approaches. Now there's a more sophisticated kind of testing, which allows you to ring-fence consumers by their attitude. You can talk to them about certain products and services in one way and the benefits in another."

Rapier tested its recent Telewest campaign to convince its client to use different formats to talk to different audiences. To sell the cable company's broadband service, the agency developed two mailings. To existing customers, it sent a standard letter, designed to inform them in detail about why they should take the service. But to prospects it sent a more sales-led, loud and colourful leaflet, based on the proposition that installing broadband was easy: all they needed to do was put their feet up and have a cup of tea.

Partners Andrews Aldridge is not alone. Claydon Heeley Jones Mason, which last week won a pitch to handle QVC's customer relationship marketing, is using a testing programme in an attempt to introduce a more sophisticated way of using of its database.

Integration is also playing a part in its comeback. Clients are getting used to all media consistently being tracked and measured, while digital offers real-time testing. Simon Hall, the managing partner at Hall Moore CHI, says: "Testing died a death when everything went brand response. But in our integrated world, all the work has got to work and return on investment is back to being king."

But not everyone is convinced, as testing can be an expensive and cumbersome exercise. Matt Button, the customer relationship marketing and database marketing manager for Lexus, says: "We have used the insights from our segmentation model to develop different creative approaches for a campaign and the results validated our thinking. Ultimately, it comes down to cost, because it would mean significantly increasing our creative and production costs for the sake of our campaign."

Steve Harrison, the chairman and creative director at Harrison Troughton Wunderman, feels testing is no longer feasible. "The advent of more targeted mailings to much smaller segments means it is simply not practical," he says.

"If your total mailing volume is just 50,000, you could do a couple of preliminary test cells of 5,000 each. But such small numbers would not give you a robust enough percentage differential to allow you to make a valid judgment on what will pull best."

But whether or not all clients and agencies buy into it, testing is clearly back on the agenda, offering clients more choice and, hopefully, better-targeted and more creative campaigns.

- Got a view? E-mail us at

INDUSTRY BODY - David Payne, consultant head of direct marketing, IPA

"People are having to work a lot harder to get response levels up, which means they've got to test.

"Financial services companies got out of the habit of testing. Print and data costs meant that it was more economical to turn the volume up, rather than fiddling around with testing. As a result, they didn't work as well.

"However, its return is partly being driven by the results we're getting from digital. Good rich-content e-mail marketing gets higher response rates. To some extent, mail has to work harder to justify its place in the schedule. And that means testing to get the results up faster."

CLIENT - Matt Button, CRM and database marketing manager, Lexus

"The speed at which we need to produce campaigns and react to the market means it's a dimension we simply can't afford.

"The results we generate have convinced me that we're justified in the route that we have chosen.

"We're moving toward real-time management of campaigns as they are unfolding. We're running acquisition campaigns online, where you can see where your impressions are being delivered and who's clicking through.

"There's a lot more flexibility because you know you're not incurring big print costs. Those results can then be used in our offline relationship marketing strategy, which is predominantly direct mail."

PLANNER - Alex Naylor, communications planning director, Rapier

"Creatives say an over-reliance on testing can result in people becoming rigid. Part of communications and creativity is about trying different things, because if people get too set in their ways, it can be inhibitive.

"However, communication is subjective, so however much planning you do, you're always going to find out more from what happens in reality than what you can work out from planning things effectively in advance.

"Instead of looking at micro changes, such as what colour envelope, you can use it to test radically different approaches. But you can't test too many things at the same time, or you don't know what they're reacting to."

CREATIVE - Warren Moore, creative partner, Hall Moore CHI

"In a fragmented media market, clients want to find out pound for pound what's going to work the hardest in each channel: what creative, what message, what proposition. If you're a client, you're so accountable for every last penny, you've got to make damn sure that every piece of the creative and the proposition is working as well as it can. To do that, you've got to test.

"The other big factor is online. The great thing about online is that you can get the work up, test it and see if it's working within hours. The atmosphere is one of media-driven offline and online testing, which is having a knock-on effect on classic direct mail."