The Broadcasting Policy Group, chaired by the former Channel 5 chief executive David Elstein, concluded that partly or wholly funding the BBC through advertising would drive the corporation downmarket.
Its report said: "If the licence fee were fixed, but the BBC was free to pursue as much advertising revenue as it could raise, the pressure to popularise its output would strengthen.
"Conversely, if the licence fee were not fixed, the pressure, from those who pay it, on the BBC to increase its advertising revenue so as to reduce the licence fee would have the same effect."
However, the group said advertising could "perhaps" play a role in funding the BBC "at some point in the future". It called for the licence fee to be phased out and replaced by subscriptions, saying that different considerations would then apply.
"There is no reason why the BBC's wholly owned digital services, once they are fully commercial, should not be entitled to augment their subscription revenues with advertising, as do their competitors."
The Elstein report said public opinion polls showed support for an advertising-funded BBC but became more ambiguous when the possible consequences were spelled out. It said that while the corporation was publicly funded, it should not take ads because it would still enjoy a benefit largely denied to its competitors.
The group warned that allowing the BBC access to advertising would risk a legal challenge under European Commission rules if it retained substantial amounts of public funding.