Mitt Romney’s remark that he “likes to fire people” will certainly end up being one of the best-remembered gaffes of a gotcha-laden primary season. Although the remark was taken out of context, it certainly reinforced the perception that CEOs are hard-nosed bastards more attuned to profits than to people.

I don’t want to speak for any other CEO, but I hate firing people. In fact, of the hundreds of responsibilities that fell to me in my six startups, letting someone go was always the most difficult.

Just to be clear, I’m not talking about firing someone for cause. It’s reasonably straightforward to fire someone for egregious misbehavior or obvious ineptitude, but if you’ve defined your position clearly, interviewed well and been reasonably diligent at checking references, a complete whiff should be the exception rather than the rule.

I’m also not talking about “reductions in force,” those bulk layoffs that larger companies use to shore up the numbers behind a bad quarter. Or more commonly, what they fall back on once they’ve lost their edge and are trying to check their long slow slide to irrelevance.

No, the hard part is when you have to fire decent people for reasons that frequently have nothing at all to do with them, except perhaps being in the wrong place at the wrong time.

There are a few common reasons to find yourself in these nasty situations:

• The Pivot – It’s unusual for young companies to get a hit with their very first swing, so unless each successive business model calls for the same expertise, an employee who was indispensible in December may be suddenly irrelevant in January. Just last week I spent hours on the phone with one CEO whose company was pivoting from consumer focus to a B2B approach; all of a sudden two of his key employees with domain expertise in consumer marketing and content were no longer as critical.

• The Upgrade – As a new company with a big idea but not much else, you hire the best you can. But things go well, you get some traction, and before you know it you’ve got a big chunk of Series B money in the bank and you’re ready for the big leagues. Even more exciting, you’re finding that all that TechCrunch buzz has given you the ability to attract the corresponding big league talent. And It suddenly becomes clear that the 20-something guy you persuaded to be your VP of marketing when you were 6 employees and working out of your spare bedroom, may not be the right guy to run marketing now that you are 100 employees and dealing with a real marketing budget.

• The Pruning – We all talk about how we only want to have A players, but most people don’t have the stones to do what it takes to actually make that happen. The hard truth is that having a work force that is materially above average, is less a matter of hiring well than it is a matter of being prepared to deal with your mistakes. If you’ve ended up with a C player, than the only way to address that problem is to move out the C so that you can take another pass a trying to land an A.

Firing someone in any of these circumstances is brutal regardless of your position, but for the start-up CEO it can be even more excruciating, because in many cases you are firing someone you spent months convincing to give it all up and set sail with you. Stand with you. Take a risk. And someone who worked tirelessly to help make your dream happen.

That’s when you know you’re really a CEO. When you have to take aside a friend and explain that their skills, however formidable, are just no longer up to the requirements of the position. Or that you’re bringing in someone from the outside to be their boss. Or that you simply can no longer afford to keep them around.

As decent human beings we always want to do what’s best for each and every one of our employees, and when someone’s not working out, it’s always tempting to “find a spot for them” so that we can keep everyone on board in perpetuity.

But we have a deeper responsibility, which is to consider what’s best for the many other stakeholders in this venture – many of whom took equivalent leaps of faith with their time, talent or money – and are expecting you to do everything in your power to bring them home safely, regardless of how unpleasant it may sometimes be.

It’s never easy to say goodbye to someone, but that doesn’t mean it’s not often the necessary and right thing to do. It’s at times like this that an honest appraisal of the situation and a generous severance will be the best thing for everyone involved. You included.

Marc Randolph (@mbrandolph) is a veteran Silicon Valley entrepreneur, high tech executive and startup consultant. Most recently Marc was co-founder of the online movie and television streaming service Netflix, serving as their first CEO. He blogs at www.marc randolph.com