ESG rating disagreements: why is corporate virtue so subjective?

Rating agencies disagree substantially about how they assess individual firms, despite the rising use of environmental, social, and governance (ESG) ratings. This is highly problematic because without agreement on what constitutes good ESG performance, market participants can be misled by these ratings. We also have very little evidence on why data providers disagree so much – making it difficult to understand the potential consequences and remedies.

In our paper, we address this by focusing on the extent of a firm’s ESG disclosure, which we believe is likely to be significant in causing providers to disagree on ESG ratings. Contrary to evidence that disclosure reduces disagreement in other settings, we find the opposite is true when it comes to ESG issues, and that while ESG rating agencies have a role to play as information intermediaries, the lack of consensus on how to interpret the outcomes of good or bad ESG performance diminishes the corporate accountability function that ratings could perform.

Challenging the theory

Theorists argue that disagreement arises due to people having different information sets and/or different interpretations of information. Conventional wisdom and a plethora of evidence suggest that higher disclosure helps to lower disagreement by reducing differences in the information that people have. We argue that due to the subjective nature of ESG information, the opposite would be true, as disclosure expands opportunities for different interpretations of information.

For companies with higher levels of ESG disclosure, data providers need to make a judgment about whether the disclosure constitutes good or bad performance. For example, a company that discloses lost-time injury rates needs to be judged based on this disclosure. This gives rise to a level of subjectivity that leads to higher levels of disagreement.

We argue that this subjectivity increases as firms expand their disclosures. This prediction is consistent with the sociological argument that multiple evaluations could occur in newly emerging fields where analytical rules and norms are less developed. Higher disclosure also increases the likelihood that ESG rating agencies might use different metrics to evaluate a firm’s performance on the same issue, therefore leading to greater rating disagreement.

Moreover, as in financial markets, evaluators may disagree over which measures are more relevant to assessing ESG performance. Collectively, these arguments suggest that increased company disclosure will result in greater ESG rating disagreement.

Primary results

Using data from 2004 to 2016 on firms from 69 countries, we find strong support for our prediction. We analyse data on ESG disclosure scores from Bloomberg and ESG performance from three of the largest ratings providers: MSCI, Thomson Reuters, and Sustainalytics. To help corroborate our findings, we focus on the adoption of broad mandatory ESG disclosure requirements across countries. We find that after a country or stock exchange implements mandatory ESG disclosure requirements, the affected firms increase their ESG disclosures and are subject to greater ESG rating disagreement.

Input and outcome metrics

To understand what ESG ratings capture and how disclosure contributes to disagreement, we examine individual metrics that providers use to construct their overall ESG ratings. We categorise these into inputs and outcomes. Inputs refer to efforts that a company is making to achieve a desired outcome (e.g., the presence of a diversity policy) and outcomes refer to actual changes achieved (e.g., the percentage of women employed at the company).

We predict and find that providers tend to disagree less about inputs and more about outcomes. This is consistent with the notion that evaluating outcomes is more subjective and relies on having a shared understanding of what a good versus a bad outcome might be. We also find that greater ESG disclosure exacerbates these disagreements, especially in the case of outcomes, consistent with our argument that more pieces of information requiring subjective evaluation should lead to greater disagreement.

Financial consequences

We explore the market consequences of ESG rating disagreement. Assessing the short period when revised ESG ratings are released, we find greater disagreement is associated with higher stock return volatility and larger absolute price movements. This suggests that ESG disagreement is relevant to market participants and influences stock prices. We find some evidence that these results are becoming even stronger, which suggests that ESG disagreement is increasingly impacting markets.

We also explore the influence of ESG disagreement on firms’ financing choices. We find that firms subject to greater ESG disagreement are less likely to raise external financing and instead tend to rely more on internal financing. This is consistent with ESG disagreement creating market frictions by introducing uncertainty regarding a firm’s long-term sustainability.

Conclusions and implications

Overall, our results indicate that ESG disagreement is most pronounced for firms with high levels of ESG disclosure, contrary to the argument that disclosure reduces disagreement. This highlights the importance of developing a shared understanding of a) what constitutes good or bad ESG performance, and b) what metrics to use to capture ESG performance, as preconditions for ESG disclosure to decrease disagreement.

Our findings also show some of the challenges that ESG rating agencies face as information intermediaries. While thousands of companies now claim to integrate ESG issues in their business strategy and operations, it is not clear whether those claims are merely cheap talk.

In the context of incomplete information about a firms’ ESG performance, ratings could help investors and other stakeholders to choose companies that exhibit their preferred ESG outcomes. In particular, having rating agencies focus on ESG outcomes could motivate companies to show real outcomes in their disclosures (e.g., reductions in carbon emissions, improvements in lost-time injury rates) rather than highlighting the adoption of policies or initiatives that might not generate any real effects.

However, the lack of consensus about how to interpret outcomes – demonstrated by our results – might encourage providers to focus more on inputs, potentially damaging the corporate accountability function that ratings could perform. In this context, we believe disagreement inhibits the accountability process. Moreover, while ESG disclosure may have many positive effects, it needs to be placed in a framework that allows analysts to evaluate those outcomes with clear benchmarks.

Overall, given the concerns over ESG rating disagreement, a lot of work still needs to be done to develop rules and norms that determine what characterises good ESG performance.

 

 

 

This blog is written by academic guest contributors. Our goal is to contribute to the broader debate around topical issues and to help showcase research in support of our signatories and the wider community.

Please note that although you can expect to find some posts here that broadly accord with the PRI’s official views, the blog authors write in their individual capacity and there is no “house view”. Nor do the views and opinions expressed on this blog constitute financial or other professional advice.

If you have any questions, please contact us at [email protected]