Dear J.J.: Interesting that you don’t mention having spoken directly to your toxic colleague. If you haven’t, the question is, why not? “The idea of talking to the person causing the problem usually scares the bejeebers out of people,” says Susan Scott, CEO of leadership development consultants Fierce Inc. and author of a book called Fierce Conversations. “So most people in this situation will come up with ‘good reasons’ not to—it would just make the person angry, for example.” The result is that “whole teams are held hostage by a single toxic employee whom everyone is afraid to confront.”

Alas, it’s not unusual. Fierce has found, in a new survey of 500 people across several industries, that four out of five employees either work now or have worked in the past with a colleague whose passive aggressiveness, negativity, habit of blaming others, or spreading malicious rumors has thrown the whole team’s morale—and productivity—under the bus.

Yet that same four-fifths majority also told Fierce’s researchers that management is “somewhat or extremely tolerant” of these troublesome types. The survey backs that up. Only 40% of bosses say they would get rid of a team member whose toxic behavior was damaging morale, versus 88% of employees who would.

Since your boss seems to belong to the tolerant camp, you’ll have to have a straightforward conversation with your difficult colleague if you really want things to change. “With toxic coworkers, most people withdraw and just try to avoid the person,” Scott says. “But a bad attitude won’t improve spontaneously.”

Start by sitting him down in private and saying, “I want to talk with you about…” (not “I need to talk to you about…” Scott advises, because “those words just make people defensive”).

Then give two or three recent examples of how his negativity or sarcasm has affected you. “It’s not about blaming or accusing,” says Scott. “Describe how you felt about these specific incidents. Then say something like, ‘If this continues, it’s going to be very hard for me to keep working with you.’”

The next step is crucial: Listen.

“There are a number of reasons employees can become toxic, and it’s important to get to the root cause of their negativity,” Scott says. For instance, feeling undervalued and unappreciated can make people bitter, “which can turn toxic quickly.”

The Fierce survey reports that more than half of employees argue with a coworker, or more than one, at least once a month. Scott points out that “not resolving these conflicts, which are inevitable in many ways, can leave people with feelings of futility” that emerge as a really bad attitude.

Something like that may be behind your colleague’s negativity, so encourage him to talk about it. “If you extend the invitation, you might be surprised at what you’ll hear about what’s bothering him,” Scott says. “It’s not about playing shrink and trying to resolve the other person’s issues but, just for a few minutes, try being an empathetic ear.” You may have to try more than once.

Suppose you do, and you get nowhere. Then it’s time to go back to your team leader, but this time, “explain what is at stake, namely the cohesiveness of the team, and everyone’s productivity,” says Scott. “Focus on how this affects the business. Just saying ‘I don’t get along with So-and-So’ is not going to lead to any action.”

What’s more, your boss “is going to ask, if she hasn’t already, ‘Have you tried talking with So-and-So about this?’” Scott says. “You want to be able to say yes, because saying no makes you look pretty bad.”

Approaching your negative colleague directly and, if necessary, bringing this up again with your boss won’t be easy. But look at it this way: Getting what you need from these tricky, important discussions is a skill worth developing, especially if you want to move up. Careers are built on relationships, and “the conversation is the relationship,” says Scott. “We succeed or fail one conversation at a time.”

Talkback: Have you ever worked with a toxic colleague? How did you handle it? Leave a comment below.