No one will ever call it his greatest trade ever. The worst? Maybe not.

John Paulson, the hedge fund manager who made billions betting against housing in 2007 and 2008, sold much of his stakes in the nation’s largest financial firms in late-2011, missing this year’s large rally in those stocks. Shares of Bank of America (BAC) and Citigroup (C) are up 45% and 24% in 2012. Paulson’s fund, though, no longer owns either.

He also significantly cut his stake in Wells Fargo (WFC), Capital One (COF) and SunTrust (STI). Paulson’s flagship fund, which is ironically called the Recovery Fund and hasn’t been able to profit from the economy’s recent turnaround, was down 50% in 2011, in part because bank stocks were poor performers. Paulson first bought into Bank of America, which had been his biggest bank bet, two and a half years ago in the second quarter of 2009, accumulating a $2.2 billion stake, and paying an average $13 per share. At first, it appeared like a good call. But by the time Paulson was selling his remaining $400 million stake, Bank of America had hit a low of just under $5 a share. Some had thought the capitulation on the bank trade would pay off for Paulson. Instead, the long shadow of Paulson’s bad 2011 continues to follow him.

But it remains to be seen just how bad the trade was. Clearly, Paulson’s timing was off. But it makes sense to be skeptical of the bank stock rally. On Wednesday, Bernstein Research analyst John McDonald cut his rating on Bank of America’s shares to a market perform. McDonald thinks things are improving at Bank of America, and at other bank stocks. But he believes the mix of low interest rates, their remaining bad loans yet to be charged off and legal fees from mortgage settlements are going to hold earnings down at Bank of America at least through 2012. Add in the bank’s stakes in European bonds, and you have an uncertain scenario at best.

Still, some other prominent investors are sticking with their bank bets. Mutual fund manager Bruce Berkowitz, who has bet big on Bank of America and been wrong until recently, still owns over 104 million shares of the company. And Paulson isn’t exiting the bank sector all together. At the same time Paulson was dumping Bank of America and Citi, Paulson added $157 million shares worth of JP Morgan (JPM). And it appears he will be sticking with Hartford Financial (HIG), where he is pushing for a break up of that insurance company.

Given that, it’s fair to wonder just how much Paulson has lost faith in his bets on Bank of America and Citi, or if he just felt forced to exit. Hedge funds and their powerful investors have become increasingly risk averse. And that seems to be hurting performance, not helping it. Hedge fund tracker Charles Gradante of Hennessee Capital told me many hedge funds these days automatically exit the market when it has been down four months in a row. That’s what happened last last summer and early fall. As a result, most hedge funds  missed last year’s late stock rally, and 60% ended up losing money in 2011, and underperforming the market, one of the worst tallies for hedge funds in recent history. Paulson’s recent failures could be another sign the age of big returns for hedge funds is coming to an end.