White House Chief of Staff John Kelly has agreed to stay in his post through the end of Donald Trump’s term in 2020, two White House officials said, tamping down speculation that the president is planning another shake-up at the top of his team.

Kelly declined Trump’s initial request to stick with him through 2024 if he’s reelected, one of the officials said.

The commitment would provide stability for a White House staff that has seen high turnover, unless either Trump or Kelly changes his mind. Speculation that Kelly would leave has swirled in recent months as he has shown his exasperation and frustration with the job, said one former White House official.

Kelly came under fire earlier in the year for the White House’s handling of security clearances after domestic violence allegations surfaced against former Staff Secretary Rob Porter. Porter was fired, and amid the fallout, other aides said Kelly lost some of his clout with the president. But the stability he has brought to the White House after the persistent chaos and infighting that marred the first year of Trump’s presidency has been valued by White House staffers.

A retired four-star Marine Corps general who served as Trump’s first Homeland Security secretary, Kelly entered the White House last July with a level of authority aides say his predecessor Reince Priebus never possessed. He moved quickly to impose order — most notably by restricting access to the Oval Office for subordinates as well as the president’s many friends, unofficial advisers and confidants.

Kelly has called the chief of staff position the hardest job he’s ever had and bemoaned to White House aids how he had planned to be retired at this stage in his life, said the former White House official. He is often at the White House around 6 a.m. and leaves after 8 p.m. He has said he spends as much as eight hours a day with Trump.

The Wall Street Journal reported Kelly’s agreement to stay earlier.