On Thursday, the World Bank announced that it would discontinue its flagship “Doing Business” report. The sudden announcement that the World Bank would end its most famous report sent some shock waves through the business regulatory community.
International scorecards carry considerable power
The “Doing Business” report, first published in 2004, was the dominant international scorecard, providing an annual snapshot of which countries provided better or worse business environments — New Zealand and Singapore routinely topped the list. A separate unit within the World Bank carried out surveys and gathered data to construct 12 indicators, which the report aggregated to create a global ranking. This analysis paid particular attention to policies that made it easier to operate a business, including items such as getting electricity and the approval timeline for certain permits. The report came out every year, accompanied by a published methodology explaining how it had arrived at its results.
Scorecard publications like the “Doing Business” report are highly influential. In a recent study, my co-author, Beth Simmons, and I documented nearly 160 economic and social performance indicators on issues such as foreign aid and human trafficking. These global indicators can define norms as to what is appropriate for countries to do. And there are often many competing indicators, produced by organizations looking to influence countries on topics such as economic prosperity, regulation of business vs. labor rights, the environment, and sustainability.
We found that the “Doing Business” report was one of the most successful indicators. Countries see a high ranking in this report as a way to attract investment and grow the economy. Furthermore, the report’s rankings have additional clout because they can influence World Bank decisions about lending support and projects.
The result, as we document in our study, is that countries fall all over themselves to compete for the top spots. The World Bank also used countries’ willingness to compete to improve their reputation as a way to encourage governments to change their policies. The bank has documented more than 3,800 policy changes related to the Doing Business report. Even Russian President Vladimir Putin publicly decreed his intent to ensure Russia made it into the top 20 list. The report’s clout didn’t just affect domestic business environments, but arguably encouraged countries to deregulate their economies — some would say to ill effects — even though deregulation is not identified as a goal in the bank’s mission.
Countries criticized the bank when it changed its methodology, suggesting that governments care a lot about moving up in the rankings. Indeed, China and other countries at times tried to obstruct the “Doing Business” report and called for the rankings to be removed. This effort was nearly successful in 2013, but the report survived. All this evidence suggests that global ratings and rankings really do matter to countries.
Report authors face immense pressure
The challenge is that global ratings and rankings depend on their credibility, but the organization faces persistent pressure from powerful countries that want to influence the published outcome. This means that the creators of effective indicators will often be under pressure to change their assessments. Some ratings, like the U.S. Department of State’s U.S. “Trafficking in Persons Report,” are partially subjective, giving their creators a little more political flexibility. Human rights advocates have accused the State Department of caving to political pressure, but the department can defend itself more easily, because its ratings are based on fairly broad categories.
This balancing act is far trickier for indexes like the “Doing Business” report, which purport to rely on a sophisticated scientific methodology. If the authors hold the line, they retain credibility and influence — but they also face enormous pressure and criticism from countries who aren’t ranked as highly as they want.
A further catch is that scorecards are more likely to get attention when they are put together by organizations that have high standing. Although some less prominent organizations, like Publish What You Fund, have succeeded through diligence in holding major international aid agencies to account, most of the influential ratings and rankings come from high-reputation organizations like the World Bank or the Organization for Economic Cooperation and Development. But these organizations are more likely to have entangling political relationships that make it hard for them to be neutral.
Fixing these problems is hard
The bank’s announcement notes that it “remains firmly committed to advancing the role of the private sector in development and providing support to governments to design the regulatory environment that supports this. Going forward, we will be working on a new approach to assessing the business and investment climate.” However, the World Bank faces some difficult decisions.
The success of the “Doing Business” report led the bank to create other scorecards, such as the Index on Women, Business, and the Law and the Worldwide Governance Indicators. The bank may now have to figure out how to protect the reputation of these rankings. More generally, the World Bank’s experience shows how difficult it is to keep politics out of politically important rankings.
The World Bank attempted to build an organizational firewall between the units generating the data and the units interacting with countries, but the recent investigation suggests that this didn’t work. Some global indicators insulate themselves from politics by relying entirely on secondary data. The downside, according to my research, is that the less control an organization has over the data, the less inclined countries are to engage with the organization behind the rankings to figure out how to advance policies closer in line with global norms.
Judith Kelley is dean and Terry Sanford Professor of Public Policy and professor of political science at Duke’s Sanford School of Public Policy. She is the author of “Scorecard Diplomacy: Grading States to Influence Their Reputation and Behavior” (Cambridge University Press, 2017). [Disclosure: Kelley is a peer reviewer in an evaluation of the “Doing Business” report by the Independent Evaluation Group. This evaluation has not yet been shared with the World Bank Group Board nor is any commentary above based on it. The views expressed in this article are entirely her own.]
