Walsh said the expected 8% hike in Air Passenger Duty (APD) in April will deter visitors to the UK.
British Airways has paid £40m in sponsorship money to be the official airline partner of the London 2012 Olympic Games.
Walsh told the Guardian: "We're spending billions on the Olympics. It's going to create a great showcase – and then we undermine it by raising APD and keeping people away."
APD is set to rise by 8% in April this year, which will be announced in Chancellor George Osbourne’s budget. Current rates of APD are £12 per passenger for a short-haul flight and between £60 and £85 per person for a long-haul flight.
The tax increases announced in November last year led to an unlikely partnership between BA, Virgin Atlantic, easyJet and Ryanair, who united to lobby the Government together and urge it to scrap APD due to its "negative" impact on the economy.
Yesterday, at the announcement of IAG's financial results, Walsh said BA had paid "almost £500m in APD" in 2011, and that as a result of the latest increase, the airline reduce the number of new jobs it creates in 2012 "by around half".
IAG reported a leap in pre-tax profits yesterday, from €84m (£70m) in 2010 to €503m (£421m) in 2011.
However, IAG warned the Olympic Games could impact BA's traffic this summer.
A spokesperson for the company said: "While the Olympics will be positive for the long term position of London as a global destination, past experience in other host cities suggests that demand could be dampened during the games."
Follow Loulla-Mae Eleftheriou-Smith on Twitter @LoullaMae_ES