Stock markets that have thus far remained relatively calm sold off Wednesday, as the picture of a tumultuous White House continued to build. Last week, President Donald Trump abruptly fired FBI Director James Comey, who at the time was leading an investigation into possible collusion between Trump associates and Russia during the 2016 election. But the controversy got even hotter on Tuesday when multiple news outlets, citing a memo written by Comey, reported that Trump had asked Comey to back off on the FBI’s investigation of former Trump National Security Adviser Michael Flynn. The White House has denied those reports.

By Wednesday’s close, the S&P 500 dipped 1.8%, the Dow Jones industrial average slipped 1.8%, and the Nasdaq fell 2.6%. The Russell 2,000 Index, which arguably would be the most heavily affected by Trump’s policies (as the companies are largely U.S.-based), fell 2.8%. Confidence in Trump’s ability to strengthen the U.S. economy also took a hit, with the U.S. dollar dropping to 0.89 of a euro, the level at which the currency was trading just before the November elections.

“Political risk has interrupted market optimism today following reports of a memo in which recently fired FBI Director Comey asserted that President Trump tried to kill the investigation into former White House National Security Adviser Flynn’s ties with Russia,” Terry Haines of Evercore ISI said in a Wednesday note. “The deepening seriousness of the controversy and the President’s self-inflicted wounds have dismayed his allies, and increase the risk that the political strain and distraction prevent progress on the Republican agenda.”

Investors instead turned to safe haven assets, with the price of spot gold rising nearly 2% to $1,261 per ounce. The fear index, which had stayed low in recent weeks, also has rose to 15.6 — the highest its been in roughly a month.

The stock market losses were led by the banks, which had originally been at the head of the Trump bump. The KBW Bank Index slide 4% by the market’s close Wednesday. Goldman Sachs shares fell 5.3%, Morgan Stanley shares dropped 5.6%, J.P. Morgan Chase shares sold off by about 3.8%, while Wells Fargo slid about 2%.

One potential reason for the sell off may be the due to the likelihood of deregulation falling, says Haines. That’s because Trump’s nominees to regulatory agencies are more likely now to get “held up in Senate—a further incremental negative for banks.”

Still, markets aren’t exactly panicking amid the news from the White House.

“Amid all the noise and distractions emanating from the Oval Office, no one on Capitol Hill is hitting the pause button on tax reform and we believe it is premature to argue a Constitutional crisis will kill tax reform until 2019,” Peter Cohn of Height Securities said in a Wednesday note.