King will leave the company at its AGM in July and Mike Coupe, currently group commercial director, will take over as the new chief executive.
After a decade at the helm of the supermarket, King said: "This was not an easy decision for me to make, and in truth it will never feel like the right time to leave a company like Sainsbury's.
"It has been a privilege to have led the Company for the past 10 years and I am incredibly proud of our achievements in that time.
"It is the 157,000 colleagues that make Sainsbury's so special and I would like to thank them for their amazing efforts over the past decade in making Sainsbury's great again.
"I am confident that under Mike's leadership the business will go from strength to strength."
Sainsbury's chairman said that Coupe was a unique talent. "No one knows Sainsbury's - or the industry - better than Mike. He has worked hand-in-hand with Justin over the past decade and has a proven track record of success making him the natural choice to take the company forward."
Coupe added: "It's an absolute honour to be appointed as the new CEO of Sainsbury's in this, the Company's 145th year, and at a time when thanks to Justin's leadership, we have been consistently outperforming the market.
"I very much look forward to building on that success for our customers, colleagues, suppliers and shareholders."