Across the board, hiring managers routinely cite culture fit as the most important element in evaluating potential hires, ahead of relevant experience and education.

It makes sense. A company with a laid-back and cooperative corporate culture shouldn’t hire a hard driving, competitive executive who’s used to berating subordinates to get results. Nor should a firm that expects 24-7 availability and long workweeks bring on a professional who prefers to set clear boundaries between work and home.

But a focus on culture fit can easily create a shield for discrimination. When hiring managers and top executives feel that a potential hire “doesn’t fit in,” that may simply mean the individual in question is different from them in race, gender, class, sexual orientation, or education.

Not only is this unfair and potentially illegal, it costs these employers the productivity and innovation benefits that research has shown result from diverse work teams. While it may be comfortable to work with people like you, it’s bad for the bottom line.

In a study of top investment banks, management consultancies, and law firms, Professor Lauren Rivera at Northwestern University’s Kellogg School of Management found that job offers were strongly influenced by interviewers’ perception of fit, as they hired candidates they’d most enjoy having a beer with or being stuck with in an airport. By seeking “playmates,” they overlooked more skilled professionals with greater long-term potential at the company, Rivera concluded.

Our brains are hard-wired to separate friends from foes, to quickly identify members of our group versus outsiders. Unfortunately, these mental shortcuts can be a liability in the workplace. Need proof of this hidden bias? When systems prevent us from using factors like race or gender in a hiring or promotion decision, gender and racial disparities narrow.

What’s the solution? No need to throw culture fit out the window. Rather, employers should take a few steps to lessen the influence of personal fit—the similarities between an interviewer and the candidate—and to better articulate the company’s cultural values. An individual’s perception of fit may only indicate how well their personalities mesh or whether they both root for the Yankees, whereas a company’s values will be general enough to include a diverse range of employees.

Define the culture. Make sure to communicate to managers and potential employees alike the distinct elements of your corporate culture and how each is aligned with business goals. Then, use data and checklists to evaluate how well each candidate truly fits, rather than relying on subjective judgments of a hiring manager. Give candidates’ surveys and structured interviews to test traits and behaviors you have shown to correlate with better on-the-job success and retention.

Measure your results. If you’re aware of a potential discrimination problem, you can set up systems to interrupt it—what University of California Professor Joan C. Williams calls bias interrupters. Use internal research to identify areas of concern, design an intervention, measure the results, and then adjust your hiring process as needed until the bias is eliminated. This can be as simple as rewriting job ads to remove traditionally masculine words or a more involved change such as when Google redesigned its promotion process.

Challenge the ‘fit excuse.’ Above all, limit how much fit can sway hiring decisions. If a candidate is eliminated for fit reasons, hiring managers should be able to explain and quantify this decision. Stop accepting vague explanations of “a bad culture fit,” especially when they remove diverse candidates from consideration.

Decades of research have documented that when left to our own devices, we all gravitate toward individuals who share our race, gender, orientation, and backgrounds. By structuring the hiring process and giving interviewers tools to assess candidates objectively, employers can hire for culture without letting it become a free pass for discrimination.