In the latest criticisms over new international banking rules, Federal Reserve Governor Elizabeth Duke recently suggested that regulators consider separate rules for the more than 7,000 community banks in theU.S. The Fed and other agencies are working to implement an international bank-capital agreement known as Basel III, which would impose restrictions on all banks.

Unlike big banks that profit from underwriting and trading, community banks simply make their money off of deposits and loans. And a large number of those loans are home mortgages, which the new regulations could make unprofitable for smaller banks to continue offering.

In a speech on Friday, Duke said as regulators craft rules to avoid the riskiest types of lending that led to the financial crisis, the problem is that some of the practices they’re targeting, such as higher interest rates and balloon payments, are key to the kind of lending that goes on at community banks. And evidence suggests that loans held by small banks didn’t cause problems during the housing bust.

MORE: Occupy Wall Street: The rebels found their cause

Duke suggests separate rules for community banks. Her argument has been made before by a growing number of bankers and regulators. What’s new, though, is Duke’s look at the role community banks play in the mortgage market. Smaller institutions “have been able to hold their own in the marketplace even as their numbers were diminished over the past few years by a number of bank closures and acquisitions by larger banking organizations,” Duke says.

The vast majority of Americans go to big banks like Wells Fargo (WFC) and Bank of America (BAC)for home loans. But taken together, community banks, defined as those with $10 billion or less in assets, accounted for 25% of new home loans in 2011. That’s up from nearly 16% in 2004 amid the height of the housing market bubble. By contrast, during the same period, mortgages issued at large banks held steady, accounting for about half of all new home loans.

Duke adds that community banks play an important role in the overall mortgage market, helping contribute to lowering mortgage rates and loosening lending standards at a time when it’s harder to get approved for a home loan. She added that Congress might also need to enact “broader exemptions” for community banks in the 2010 Dodd-Frank financial overhaul law.

That may be true, but the problems plaguing community banks are also beyond new regulations. Demand for business loans has remained especially weak at community banks amid a tepid economic recovery. Across small banks, business loans outstanding during the three months ending in June rose to $257.2 billion, a 6.2% rise from a year earlier, according to data from the Federal Deposit Insurance Corporation. By contrast, such loans at big banks rose  much higher to $1.2 trillion during the same period, a 17% increase. All this is made worse at a time of very low interest rates, which prevent banks from making more on the few loans they make.

MORE: Too small to survive?

As Fortune’s Stephen Gandel pointed out recently, analysts are predicting that thousands of community banks may disappear over the next several years. True, the FDIC has been pouring billions of dollars into saving mostly small banks, but community bankers, which Duke once was, make a strong case for why smaller institutions should get special attention.

“Mortgages are an important revenue source for community banks,” says Karen Thomas, senior executive vice president of government relations and public policy for Independent Community Bankers of America, a trade group for small U.S. banks. They serve a special slice of small-town America, taking on less conventional loans that aren’t necessarily saleable to mortgage giants Fannie Mae or Freddie Mac.