The fashion retailer, that runs from inexpensive shorts to haute couture, said on Tuesday it would cut more jobs as well as close its Polo store on Manhattan’s Fifth Avenue as it looks to kickstart a tentative turnaround. The moves come on top of 50 store closures last year, a decision stemming from dwindling sales at its own stores as well as at the department stores like Macy’s (M) that carry its products. The need for further costs unnerved Wall Street and shares fell 2%.

And the news comes at a very bumpy juncture for Ralph Lauren: the iconic American fashion company’s CEO wunderkind Stefan Larsson, the man who had earlier turned H&M and Gap Inc’s (GPS) Old Navy into retail behemoths, is leaving the company in May after only 18 months in a previously announced departure. That announcement had caused Ralph Lauren’s stock to crater. What’s more, the once top-of-class retailer reported early this year that net revenue for the quarter ended Dec. 31, which included the holiday shopping season, fell 11.9% to $1.7 billion, while profit collapsed 37.4% to $82 million.

The latest moves will save Ralph Lauren about $140 million a year but the company will incur some $370 million in one-time restructuring charges. They are in addition to efforts last year to cut costs by $180 million to $220 million.

As for the Polo store, Ralph Lauren’s chief financial officer, Jane Nielsen said in a statement that the closing was aimed to free up the company to explore other retail concepts and hinted that its Ralph Coffee cafe as one idea that could be expanded.

“The decision will optimize our store portfolio in the New York area and allow us to focus on opportunities to pilot new and innovative customer experiences,” said Nielsen.

The latest efforts that are part of what the company called its “Way Forward Plan” will see RalphLauren carry out “a reduction in workforce” and eliminate some corporate operations.