Let’s start with materiality assessments’ ubiquity. A materiality assessment helps a company “understand the [ESG] issues that matter most to [its] internal and external stakeholders; […] and how they translate today and in the future into associated risks and opportunities for [the] company.” (Thank you, Novartis, by the way, for that clear and concise definition.)

Last year, KPMG reported more than three-quarters of Fortune 250 companies performed ESG materiality assessments. Any holdouts may soon be forced to come aboard: Over the next few months, regulators in Europe and the U.S. will be announcing which—not whether—“materiality” disclosures will become mandatory. By 2026 every company will be on the hook for some materiality reporting.

In essence, that should be good news for die-hard ESG-ers.

There is just one problem. A materiality assessment often doesn’t do what it’s designed to do.

In many cases, it doesn’t allow companies to understand which issues truly matter most to their stakeholders. And equally often, it doesn’t show what strategic opportunities this generates in the future. It was a partner at consulting firm Bain & Company who pointed this out to me recently.

(Full disclosure: Back in 2009-2011, my first job out of college was with Bain. Then I discovered my true calling was journalism.)

“ESG data fill an investor need,” Bain’s Zach First told me on Wednesday. “If you ask stakeholders to force-rank [ESG metrics], they will do so, but it doesn’t mean it matters most to them.”

It was late in the evening when we spoke, so I had to ask him again. Are you telling me that when employees say they care about, for example, their salary, in a materiality assessment, they don’t actually care about it? But even with that straightforward example, First didn’t relent.

Of course, employees care about their salary, he clarified. “But what matters is where the data is collected,” he said, “ESG data comes from the voice of the company or the government,” and is aimed at investors. Seen from the “voice of the employee,” salary would be net disposable income, not gross hourly wage. It’s a technicality, in this case, but in other metrics makes a world of a difference.

Take CO2 emissions. Often, customers will say they care about CO2 emissions, ranking it high in materiality. But what really matters is not how much they care about CO2 as compared to, say, water usage (another ESG metric), but how much they care about it compared to, say, the customer-friendliness of their sales agent or the overall reliability of their product (i.e. customer metrics).

There’s a second problem. Materiality assessments don’t easily allow management to weigh the needs of various stakeholders against each other, let alone those of segments of stakeholders. What strategic action should be taken, for example, if one group of customers cares about CO2 and the other not at all? And how would they even know of the conundrum, if an assessment simply aggregates the responses from all customers—or even all stakeholders?

The outcome—and I reckon many readers will recognize this—is that while companies overwhelmingly publish materiality assessments, they are often ineffective in strategy and transformation discussions. For management, First said, it means only two things can happen: “Either you decide intuitively which one or two ESG issues matter most. Or you throw your hands up in the air and say: ‘I don’t know.’”

I have seen this at work. And I’d argue it puts us on a slippery slope. On the one hand, materiality assessments theoretically are about engaging stakeholders. On the other hand, because of their faulty design, they effectively rely on the discretionary assessment of managers. If that’s the main impact of materiality assessments, is that really a good idea?

Various consulting firms I spoke to are working to remediate these shortcomings by setting up more targeted materiality assessments, allowing them to compare metrics and stakeholders and derive strategic insights. But what do you think? I’d love to hear your thoughts. We’ll continue discussing this topic—and its possible solutions, in the months to come.

Peter Vanham
Executive Editor, Fortune Impact and Connect
@petervanham
peter.vanham@fortune.com

Also on our radar:

On Valentine’s Day: ESG leaders, unite
On Feb. 14 at 12 p.m. ET, Fortune CEO Alan Murray and I will virtually welcome the members of the Impact Initiative for a discussion on how to navigate ESG reporting amid growing criticism—and growing regulatory oversight from the U.S. and Europe.

We’d love for you to join us. The Impact Initiative is where the themes addressed in The Impact Report come to life and where chief sustainability officers, chief impact officers, and other executives focused on putting people and plant at the heart of their company's business model come together. It’s where I get a lot of the big-picture thinking for my newsletter and have met quite a few of you.

Read more on the Impact Initiative here, and get in touch with my colleagues Elizabeth Tighe (elizabeth.tighe@fortune.com) and Sarah Wohrob (sarah.wohrob@fortune.com) to attend.

What a chief sustainability officer does: readers’ perspective
Last week, I wrote about the emerging “types” of CSOs, and that got a lot of you thinking. Here’s some feedback I received on the piece—and the idea that there’s four types of CSOs:

“I do not think it is positive that there is no consistent idea of what this role is, or what it is for, and everyone just feels free to define and structure it as they like. This is exactly why [...] the emphasis is on telling the story over getting stuff done.” —Alison Taylor, clinical professor, NYU Stern School of Business (U.S.)

“Ideally, there would be one category - the "Impact CSO". The one who has the ability and authority to bring about major changes in the way companies operate as opposed to repackaging or tweaking existing activities […] etc., all of which are secondary functions.” —Paul Griss, president, Boldon Group (Canada)