The Royal Statistical Society has got involved with the issue after its attention was drawn to a number of ads by the Chartered Institute of Public Relations last year.
The ASA has also been pressed by the co-chair of the All-Party Parliamentary Group on Statistics, Lord Lipsey, who has made several complaints about ads including one that resulted in a ban for online dating service eHarmony.
Among the specific brands that the Royal Statistical Society has criticised for featuring claims based on inadequate sample sizes are Revlon-owned Elizabeth Arden (with a sample size of 25), L’Oréal Men Expert (54) and Johnson & Johnson-owned Aveeno (57).
"Major advertisers currently appear able to run, with impunity, campaigns that are based on comparatively miniscule and statistically inadequate sample sizes," the society commented in a submission it has made to the inquiry into the UK advertising industry currently being conducted by the House of Lords Communications Committee.
The submission highlights other bugbears: sample suitability, stretched evidence and substantiation. On the last issue the society states it is "disconcerted by the brevity and, statistically, the blatant inadequacy of the Code of Broadcast Advertising", which it says fails to explain what the ASA should regard as "adequate substantiation".
But despite its critical language, the society is keen to meet with the ASA, its director of policy and public affairs Iain Wilton told Campaign. He added that the ASA would be invited to an event about statistics in advertising being planned by the All-Party Parliamentary Group on Statistics (for which the society provides the secretariat) later this year.
Although Wilton said the society has not gone to the extent of submitting formal complaints to the ASA about ads it disagrees with, advertisers can nonetheless expect to be challenged by Lord Lipsey, who actually served on the ASA ad complaints committee from 1999 to 2005.
Lipsey told Campaign: "I have a long record of complaining against statistically bogus ads. Last time I enquired I was amongst the ASA’s top ten complainers and for good reason: the ASA system though I think it is a good one, is complaints-driven and if people like me don’t look out for abusive ads it won’t work."
Last week the ASA rebuffed his complaint about a Bulmers press ad, which claimed "2/3 of drinkers prefer the taste of Bulmers Original to Magners Original" with small text underneath which stated "Source: Cardinal. 65.8% in a head to head blind taste test, surveyed in Nottingham and London June 2017. Excludes those who expressed no preference. Sample size: 146 regular apple cider drinkers".
The ASA said it took expert advice on the methodology that the third-party researcher used for the taste test cited in the ad and gave it the green light.
In response, Lipsey said he regarded this as "a wrong-headed judgement contrary to the basic principles of using stats right". The Royal Statistical Society was more measured, with its vice-president for external affairs Dr Jen Rogers commenting that "in this particular case, we’re encouraged by the conscientious investigation conducted by the ASA … however, we believe Bulmers’ claim was somewhat overstated".
A spokesman for the ASA, commenting on the society’s general concerns, said: "The ASA already has high standards in place to hold advertisers to account when quoting statistics in advertising. However, we remain open-minded and have written to the RSS, whom we hope to meet soon to hear their views."