During a Congressional hearing of the House Financial Services Committee on the Federal Reserve’s oversight of banks, Sherman asked Janet Yellen, who was testifying, whether she had considered breaking up Wells Fargo in the wake of its recent scandal. Earlier this month, Wells Fargo was fined $185 million for opening up nearly 2 million accounts without getting permission from customers. Wells Fargo revealed it has fired 5,300 employees who were found to have defrauded customers, but that no senior executives had been fired.

On Tuesday, the bank announced that it was clawing back $41 million in compensation from its CEO John Stumpf, nearly a quarter of what he has been paid since becoming head of the bank, as well as $19 million from former banking unit head Carrie Tolstedt. It also said it was eliminating sales goals faster than expected.

At the Congressional hearing, Yellen refused to say whether the Fed thinks Wells Fargo should be broken up. “We hold the largest organizations to the highest standards,” Yellen said.

Sherman countered that it seems that few banks have been able to meet those high standards. Therefore, he argued, the best solution would be to break up the banks so they don’t have to meet as high a standard. “This seems to be a case where too big to fail means too big to manage,” said Sherman.

Yellen said that she didn’t think the Wells Fargo incident is a question of “prudential regulation,” meaning it’s not something that could cause the bank to fail or lead to major trauma for financial markets. But she did agree that there seems to be a repeat offender problem going on at the big banks. “There has been a very disturbing pattern of violations,” said Yellen. “We are taking a comprehensive look at the big banks.”