Walmart first entered the Japanese market in 2002 by buying a 6% stake in Seiyu, and gradually built up its stake before a full takeover in 2008.
But the company, like other foreign entrants lured by the high spending power of Japanese consumers but frustrated by cut-price competition, has struggled to make money in Japan.
KKR will buy 65% of Seiyu and Rakuten will acquire a 20% stake while Walmart will retain 15%, the companies said in a joint statement.
Japanese media reported two years ago that Walmart was seeking a buyer for Seiyu, and a sale could amount to around 300 billion to 500 billion yen.
Sources said at the time that it failed to find a buyer. Walmart said last year, amid speculation it was seeking to exit Japan, that it aimed to list Seiyu but retain a majority stake in the business.