‘Countability’ versus accountability: how process indicators may lead to false narratives of success

A question many organisations are asking in relation to the work they are undertaking to assess and address risks of modern slavery in their operations and supply chains is, “How will we know if what we’re doing is making a difference?”.

Mandatory criterion five of the Modern Slavery Act 2018 (Cth), requires reporting entities to “assess the effectiveness of actions being taken to assess and address modern slavery risks”. Interestingly, the Commonwealth Government’s Draft Guidance for Reporting Entities states that this criterion requires entities to explain how they assess effectiveness of actions, not whether their actions have been effective.

This highlights an important distinction between measuring process outcomes and impact outcomes and raises the question of whether transparency in reporting legislation of this nature can (or will) lead to measurable human rights improvements on the ground. It also raises the question of whether complex issues like human rights impacts can be evaluated using quantitative performance measures traditionally adopted by the corporate sector.

While the Australian Government’s list of process indicators, such as the number of modern slavery training and awareness programs delivered, the number of contracts that include modern slavery clauses, the number of actions taken to work with suppliers to improve their capacity and so on, may result in positive human rights impacts for workers in global supply chains, without accurate impact and outcomes assessments, the discussions will only ever be based on assumptions and perhaps wishful thinking.

This focus on the measurable aspects of business activities (the processes) will perhaps miss the more important, but less measurable ones (the outcomes), presenting a very real danger of confusing countability with accountability. 

While Australian reporting entities may proudly document so-called ‘effectiveness measures’ in their Modern Slavery Statements, for example, that 90% of their Tier 1 suppliers have signed their supplier code of conduct, this is, at best, a measure of ‘countability’.  Measurement of ‘accountability’ would involve an assessment of the number of suppliers who have changed their practices to reflect the requirements of the code of conduct resulting in better, fairer and safer conditions for their workers. Accountability measures would require detailed and repeated site assessments and importantly interviews with workers to ascertain whether real human rights outcomes have been realised on the ground.

There is no doubt that the ability to conclusively demonstrate this level of effectiveness and accountability across complex, global supply chains is difficult and time consuming. In some instances, it may even prove impossible. However, as was pointed out in a report prepared by the International Council on Human Rights Policy[1] (ICHRP), process indicators (like the one’s mentioned above) may in fact generate “micro-narratives of success” and a “reduced ability to engage honestly with failure.” The fear of loss of credibility may lead reporting entities towards results-based reporting where KPIs focus on the effectiveness of immediate (process) goals whilst ignoring the final important goal of measurable impact on people - improved working conditions, better access to justice and the elimination of slavery in all its forms.

According to the ICHRP, evaluation at its core “is about assessing how far expectations of one sort or another have been met.” In the context of Commonwealth modern slavery reporting requirements, the expectations placed on businesses are that they put processes in place to assess and address the risks of modern slavery in their operations and supply chains. The expectation (no matter how limiting) is that they assess the effectiveness of actions taken and not whether the actions are effective. This focus on ‘countability’, rather than ‘accountability’ will do little to protect the human rights of vulnerable workers along the supply chain.

Ultimately though, evaluation and effectiveness measures must be an ongoing process of reflection and improvement. As indicated by the ICHRP, “any worthwhile definition of accountability should explicitly embrace learning: an accountable organisation is one that learns and improves as a result.”

There is hope then, that ‘countability’ measures may, over time lead to more accountable organisations whose operating models embrace, promote and verify real human rights outcomes.

[1] International Council on Human Rights Policy, ‘No Perfect Measure: Rethinking Evaluation and Assessment of human Rights Work. Report of a Workshop’, January 2012