A further investigation by The Times has found that the brands are unwittingly funding extremist videos. As a result the brands have pulled millions of pounds worth of advertising from the video platform.
The paper reported that the videos included "rape apologists, anti-Semites and banned hate preachers".
A government spokesman said that Google has been "summoned for discussions at the Cabinet Office" and that it has temporarily pulled its ads from YouTube "pending reassurances from Google that government messages can be delivered in a safe and appropriate way".
He added: "Digital advertising is a cost-effective way for the government to engage millions of people in vital campaigns such as military recruitment and blood donation.
"Google is responsible for ensuring the high standards applied to government advertising are adhered to and that adverts do not appear alongside inappropriate content."
Jaguar Land Rover was among brands that pulled its ads last month after a similar investigation by The Times.
A Google spokesman said it has "strict guidelines" and "in the vast majority of cases" its policies work.
He added: "We accept that we don’t always get it right, and that sometimes, ads appear where they should not. We’re committed to doing better, and will make changes to our policies and brand controls for advertisers."
Dan Brooke, Channel 4’s chief marketing and communications officer, said the channel is "extremely concerned" about its advertising being placed alongside offensive material on YouTube.
"It is a direct contravention of assurances our media buying agency had received on our behalf from YouTube," Brooke said. "As we are not satisfied that YouTube is currently a safe environment we have removed all Channel 4 advertising from the platform with immediate effect."
A spokeswoman for L'Oréal said the company was "unaware" and "horrified" that its ads were being placed on such YouTube channels.
She added: "We strive to ensure that our brands do not appear alongside this kind of content online and we manage our media buying very tightly, but unfortunately, no system is 100% failsafe.
"In this case, it seems some of the YouTube inventory sold to us by Google was incorrectly categorised by them and as a result our campaign was featured on these channels. We are taking immediate action to remedy this issue and will be working yet more closely with Google to prevent this from happening in the future."
A spokesman for ISBA urged Google to "immediately review its policies and controls on the placement of advertising and to raise the bar to eliminate the risk of brands being damaged by inappropriate context."
The industry body enocuraged Google to withdraw from sales any ad inventory "which it cannot guarantee as a safe environment for advertising." This meant reviewing the practice of placing ads against newly-uploaded YouTube content, before it has been classified, ISBA said.