FORTUNE — The Federal Reserve said a majority of the nation’s largest banks would be able to weather another deep recession. Four banks, though, have more work to do.

In the stress test results it released on Tuesday afternoon, the Fed said that 15 of the nation’s 19 largest financial firms have enough capital to cover losses under extreme circumstances: an unemployment rate of 13%, a 50% decline in stock prices and another 21% in housing prices. Three banks – Ally Financial, Citigroup (C) and SunTrust (STI) – would likely need new capital from either investors or the government in the Fed’s adverse economic scenario. A fourth financial firm, insurer Metlife (MET), would likely be in need of assistance as well. A senior official at the Fed said all four of the banks would need to submit plans that would detail how they would increase their capital to make them less vulnerable in a downturn.

Citigroup said it remains among the best capitalized large banks in the world. However, it said it would not be able to raise its dividend as it hoped, and would submit a revised capital plan to the Fed. Ally said it supported the idea of stress tests, but it disagreed with a number of the assumptions the Fed made, including overstating the bank’s potential mortgage losses. SunTrust could not be reached for comment. Metlife said it was unfair to apply the same tests to insurers as it did to banks.

All told, the Fed said that the nation’s 19 largest banks could lose as much as $534 billion. The majority of that would come from soured loans and trading losses from a “global financial market shock.” The Fed said that the majority of banks would be able to absorb the losses because they have increased their collective capital by more than $300 billion since the start of 2009. “The capital positions of our banks have improved substantially in the past three years,” says a senior Fed official.

The Tier 1 common equity ratio of the nation’s largest banks, a key measure of bank health, would fall to an average of 6.3%, from a recent 10.1%. The Fed generally considers anything above 5% to indicate a healthy bank. Ally came out the worst in the Fed’s stress test. The Fed said its capital ratio could fall to half of the required amount. Among the nation’s six largest banks, Wells Fargo (WFC) came out of the test with the highest ratio of 6%.

The Fed said the stress test was modeled on an extreme negative scenario and not its actual outlook for the economy. It was the first time the Fed had released a thorough test of the banks’ financial health since the early days of the financial crisis.