After adding thousands bankers in the past two years, financial firms again appear to be on the verge of cutting that many positions and then some. Consultants and Wall Street recruiters say banks could eliminate nearly 21,000 jobs from their securities divisions in New York alone. Worldwide cuts could be even larger. Recruiters say big banks are in the process of finalizing their downsizing plans, and that layoffs could start soon.

The latest round of job cuts could rival those that happened during the financial crisis. Back then, which was less than four years ago, Wall Street eliminated 28,000 positions. But that round of downsizing included the collapse of Bear Stearns and Lehman Brothers, and the biggest crisis in the financial markets since the Great Depression. By comparison, the stock market is up this year, and just last week banks reported better than expected earnings for the first quarter. What’s more, at the same time large firms are firing, many smaller investment banks have been staffing up. As a result, overall employment on Wall Street might not drop as much as it did after the financial crisis.

“Hiring is going on, it’s just not by the big banks,” says a top Wall Street recruiter Gary Goldstein, who runs Whitney Partners.

Nonetheless, consultants say the big Wall Street firms are coming to the conclusion that they have more workers than they need. Last week, The Boston Consulting Group released a report that predicted banks would eliminate 12% of their workforce in the “short-term.” Recruiters say those numbers sound similar to what they are hearing from the large firms.

“The estimate is possibly low,” says veteran financial industry recruiter Steve Potter at Odgers Berndtson. Potter says not only are the firms competing for few deals, but with their clients. More and more large firms are adding investment bankers to their staffs to save on Wall Street fees. “Large layoffs are a virtual certainty.”

Perhaps the biggest problem at the banks is that they didn’t cut enough jobs last time around. Mergers and acquisition activity also has not bounced back as expected, leaving a number of high paid bankers idle. What’s more, new regulations appear to already be significantly curtailing the banks’ trading operations. Also weighing on the banks is the fact that debt watchers Moody’s and Standard & Poors say they are likely to soon downgrade the bond ratings of the firms. The nation’s five largest banks have estimated that the downgrades could cost them $22 billion in additional costs or collateral requirements.

“There hasn’t been enough action on the cost front to keep up with the revenue short falls,” says Chandy Chandrashekhar, a partner at BCG who helped to produce the recent report. And unlike other rounds of layoffs, Chandrashekhar says many of the people who lose their jobs this time around could be senior bankers. For those that remain, compensation is likely to be down this year as well. In all, BCG expects Wall Street compensation expenditures to drop by as much as 30%. “Banks need to revisit whether they need all of their management layers.”

Recruiters say one of the firms likely to cut the most is Credit Suisse. The firm’s investment banking division has struggled recently. Last year, Credit Suisse said that it plans to eliminate 3,500 jobs, across the whole bank, not just its Wall Street business. About 2,000 of those job cuts have already been completed. Sources say a majority of the remaining cuts will come from the firm’s investment bank, and that the bank may end up cutting more workers than earlier announced. Credit Suisse declined to comment.

Other firms that sources say are likely to make deep job cuts in their investment banking divisions are Bank of America, which bought Merrill Lynch during the financial crisis, and Barclays, which acquired the U.S. investment banking division of Lehman Brothers out of bankruptcy. But recruiters say that all of the big banks, including Goldman Sachs, appear to be on the verge of making cutbacks.

“Banks haven’t come up with a model that makes up the profits they used to get from proprietary trading, CDOs and other structure deals they used to do,” says Goldstein. “I have heard about a lot of people who didn’t get the promotions they were expecting. That’s usually a sign that banks are getting ready to get rid of people.”