The agency pitched for the business against an initial longlist that included Wieden & Kennedy, Abbott Mead Vickers BBDO, BBH, Rapley Smith & Jones and the incumbent, Saatchi & Saatchi.
W&K and AMV were eliminated last month after a first round of pitching.
Fallon's London office will have lead status on the business, co-ordinating the creative activity across Europe. Its first work is due to appear in April next year.
The agency will produce conceptual planning and creative work to run in a variety of media, including television, online, radio, newspapers, magazines and outdoor in about 20 European countries.
Fallon will produce work for Sony's consumer audio, video and IT products in Europe.
Saatchis picked up the Sony business in 1999 from BMP DDB. During its three-year tenure, it introduced the strapline "Go create".
The appointment follows Sony Europe's decision to consolidate its £200 million pan-European media planning and buying into OMD last month, after a final pitch between Zenith Media and MindShare.
The Fallon win does not include advertising for the Sony PlayStation games console, which continues to be handled globally by TBWA Worldwide.
Also unaffected is advertising for the joint venture mobile handset, Sony Ericsson, handled by BBH.
Michael Wall, a partner at Fallon, said: "We are delighted and flattered to be asked to complement the team that will be writing the next chapter of the development and continued success of the Sony brand."