Back in 2010, a celebrity was viciously attacked by trolls after tweeting in favor of gun control. Costolo said he decided then that some people should not be allowed to use Twitter(TWTR), and told his team as much. But he “was talked out of that,” he said, because “there are all these edge cases and these are very complex problems.” So he let it go.

“I wish I could turn back the clock and go back to 2010 and say, ‘Look, we’re going to stop abuse on the platform by creating a very specific bar for how you need to engage in conversation,” he said. “I really feel like a lot of that is on me from six-seven years ago.” He noted that a leaked email showed that Twitter wasn’t doing enough and it was his fault, which he admits to. It’s a complex problem, he said, and there is more that can be done. Those things aren’t contradictory.

He later added, for emphasis, “I wish had done more about bullying … I mean it when I say I take responsibility for not grabbing the bull by the horns.”

Blocking specific words and phrases is a “ridiculous” solution, he says. Instead, Twitter should build products that force people out of their echo chambers. “I think there’s an opportunity to take these small clusters of people who are only talking to each other and listening to themselves in their echo chamber and find away to engage them in rational discourse without yelling at each other on these platforms,” he said.

That would likely involve manual curation, and finding a way to highlight authoritative voices, rather than what gets the most clicks and engagement. One of his biggest product initiatives, Moments, was meant to highlight those “high authority voices.” Moments launched in late 2015. Twitter recently changed its Moments tab to something called “Explore.”

“Of course people don’t have to go look at that, so you have think about … how do you get people to look at it,” Costolo acknowledged.

Beyond that, Costolo believes Twitter is working on a viable approach to the situation by treating trolls like spammers. In that sense, it’s an arms race. “You need to make it more economically expensive to be an abuser than to be abused,” he said. “For too long on Twitter, if you were abused, you had to fill out this long report … that’s a case where the economics are flipped in the wrong direction. I think [CEO] Jack [Dorsey] and the team are working on that.”

Costolo blamed his failure to fix the bullying problem, in part, on his status as the company’s CEO, but not its founder. “I always felt a little bit at Twitter like it wasn’t mine,” he said. “I wasn’t the founder of the company. Jack would be extremely gracious on stage about considering me a founder, but I personally never felt like it was my company. In some cases, like spam and abuse, I maybe didn’t act as forcefully or aggressively as if it was mine.”

He’s got the so-called “moral authority” of a founder now, having started Chorus, a fitness-focused startup that has a social media component. The company has raised $8 million in venture funding and plans to launch later this year. Costolo compared the level of security he had as Twitter’s CEO when ISIS threatened him to his life now as a scrappy startup founder where random people can walk into the office. “I told my VP of Engineering, ‘We’re out of coffee.’ He said, ‘Make some!’” Costolo, a former standup comedian, joked.

He’s also able to tweet more freely about his political views, including calling the son of Trump advisor Michael Flynn a “dumbass.”

“That’s probably something a head of company shouldn’t have done,” he said. “It’s easier and more freeing for me to be as engaged as I want to be.” He added: “I think CEOs can say more than they might be willing to say at first, and I think that they should.”