Charlton will lead the agency alongside the founder and executive creative director, Andy Fowler; the strategy director, Phil Teer; and the digital director, Kevin Brown. He will have an equity stake in the business.
Brothers and Sisters' clients include Sky Sports, Capcom, Skycig, Vue and Center Parcs. Nielsen billings in 2013 were £40 million.
When he starts at the end of April, Charlton will effectively replace Juliet Hay-garth, who stepped down as the managing director of Brothers and Sisters in March to join BMB as the chief executive.
Charlton was not required to go on gardening leave as part of his financial settlement with BETC London.
Since his sudden departure in January, Charlton has been in talks with "quite a few" agencies but said he believed Brothers and Sisters was the best fit.
Charlton said: "I wanted to work at an agency that was in charge of its own destiny. Where the people in the room are the owners and make the decisions."
He also praised the agency's creative capability, particularly in mobile, digital and content: "It is incredibly innovative, restless and interested in new opportunities."
Fowler said the discussions with Charlton had been ongoing for the past three weeks. He said: "We wanted to find someone whose views were as progressive as ours and we got on incredibly well."
One of Charlton's priorities will be establishing how a small agency can compete in an increasingly integrated marketplace, particularly in light of Bartle Bogle Hegarty's recent British Airways win.
He said: "Most clients will now expect agencies to do everything for them. Looking for opportunities for joint ventures will be high on the agenda."
Charlton left BETC London alongside the co-founder and executive creative director, Neil Dawson.