The appointment marks Rose's highest-profile retail role since he stepped down as executive chairman of M&S in early 2011.
Rose will join the Ocado board on 11 March to ensure a smooth transition, as the online supermarket prepares for a key period of expansion.
Ocado is to open a second distribution centre, which will almost double its capacity, as rival supermarkets including Waitrose aggressively pursue their own online food offerings.
Last week, Ocado reported that sales for the six weeks to 6 January increased 14.2% to £91.6m year on year.
Waitrose is increasingly encroaching on Ocado's key business in the South-East and, over Christmas, its online orders rose 68%, helping the supermarket hit sales of £200m during the festive season.
Rose took a keen interest in Ocado during his time at M&S and reportedly led discussions with Ocado management about M&S acquiring the business, before the online supermarket floated on the stock market.
Speaking about his appointment, Rose said he had "been very impressed at the impact and progress Ocado has made to date".
He said: "As retail goes through a fundamental shift into the digital world, I believe Ocado's model and the high standards of customer service it provides will see it emerge as a powerful online player."
Grade described his six years at Ocado as "exhilarating and eventful" and said he believed Rose's "substantial retail experience and commercial acumen will add significant value".Follow @mattchapmanuk