Goldstone is being replaced by Iwan Williams, managing director of Premier Food's bakery and grocery division.
Williams and Goldstone have been working closely developing Premier Food's Power Brands strategy, which involved the upping of its marketing spend to more than £50m in 2012.
It is thought that Williams will keep his managing director job title and will subsume Goldstone's marketing duties within his role.
During his tenure at Premier Foods, Goldstone was responsible for marketing campaigns including the Hovis "go on lad" campaign.
Goldstone will leave the company later this month. His exit comes amid a difficult time for Premier Foods, which is struggling to pay down a debt pile which stood at £1.27bn in November last year.
Goldstone's previous roles including marketing director at Hovis, between 2008 and 2011. Prior to that, he was marketing vice-president at PepsiCo-owned Walkers Snacks.
He has also worked as senior marketing manager at Campbell Soup company.
Follow John Reynolds on Twitter @johnreynolds10