PRI Awards 2019 case study: Portfolio-wide Monitoring and Evaluation Platform

Project overview, objectives, and reasons for undertaking it

African Infrastructure Investment Managers (AIIM) is a wholly owned subsidiary of Old Mutual Alternative Investments. It has shareholdings in 26 renewables assets in South Africa under the Renewable Energy Independent Power Producer Procurement Programme (REIPPP). Each is obligated to spend a percentage of profit on socioeconomic development and enterprise development projects within a 50km radius.

AIIM’s investment philosophy is to provide superior returns to investors while making a tangible contribution to local communities. In line with this objective, in 2016 it commissioned the development of a Social Outcome Measurement Methodology to measure social outcomes arising from socioeconomic development, enterprise development and corporate social investment initiatives – all underpinned by the Theory of Change methodology.

The platform was borne out of the need to assess the degree to which assets are achieving their intended socioeconomic development and enterprise development goals, and to evaluate the positive impact of projects. Mainstream Renewable Power was a key collaborator in helping drive the launch of the platform, across their managed renewables plants. The objectives of the project were to:

• track performance of socioeconomic projects;
• track outcome indicators effectively;
• improve communication with stakeholders;
• align and improve reporting processes;
• and support investment decision-making.

Successful implementation of the project across the portfolio will enable AIIM to understand the scale of its impact, effectively report this to its stakeholders, magnify positive impacts through good decision-making, and transfer lessons learned to improve performance or inform course correction.

Financial scale of the project and impact

The pilot phase of the project involves a monitoring and evaluation initiative for eight renewable energy projects, namely: Cookhouse, Umoya, Khobab, Loeriesfontein2, Noupoort and Kangas Wind Farms and the REISA Solar Farm. The plan is to take on the remaining 26 assets and eventually scale this project to the wider REIPPP program via a secure cloud-based platform. The results of the monitoring and evaluation are mapped to the United Nations Sustainable Development Goals and South Africa specific goals. The scope of work included:

• a desktop review of existing development programmes;
• creation of social outcomes maps for the investments;
• identification of appropriate indicators;
• data collection;
• analysis and reporting.

AIIM hired a leading African sustainability consultancy, IBIS Consulting, to undertake the project. It is currently concluding the development of social outcomes maps and appropriate indicators. Data collection will take place in June 2019 with reporting in July or August 2019. A key indicator of success will be the representation of the data. AIIM envisages a concise impact measurement report per community investment project that focuses on numerical and graphical representations of the data and conclusions.

Project delivery challenges overcome

The project was piloted with the REISA Solar Farm in Q1 of 2017. Through 2017 and 2018 the social impact methodology was implemented in three assets: REISA, Cookhouse and Umoya. The flagship socioeconomic development and enterprise development projects were evaluated using the methodology, with reporting provided to the companies’ social and ethics committees.

Implementation for this early stage was through community operations teams, including community liaison officers, who undertook monitoring and evaluation.

Critical challenges included:

• It took time to understand the Theory of Change methodology, creating a time lag and incorrect indicator selection;
• The heavy administrative burden put on management teams;
• Difficulty in scaling, with assets spread across South Africa;
• Conflict of interest having teams assess and report on their own impacts;
• Reporting did not always meet the expectations of the regulatory authorities.

A new model was initiated in the latter half of 2018 to address these challenges and create an external platform of specialist advisers to undertake the scoping, data collection, analysis and reporting across a portfolio of renewables companies. In addition to this, a leading technical social impact and Theory of Change specialist was appointed to act on behalf of AIIM to continuously review the advisers’ approach and work.

Measuring success and lessons learned

AIIM’s goal is to make the platform available not only to all 26 of its own portfolio of renewables companies, but also to the whole REIPPP. In terms of broadly applied lessons learnt, the following are evident from this platform:

• A similar outsourced monitoring and evaluation platform could be applied across different geographies and sectors to achieve the same levels of quality and efficiency.
• There is a need to balance the qualitative versus quantitative data collected and used. In order to become more efficient and cost effective, experience shows that a greater use of quantitative data is required, while not losing the value of qualitative data in providing nuanced information about the actual change happening in the community.
• The cumulative spend of the assets in AIIM’s portfolio was in excess of ZAR350m (US$24m) at the end of 2018. Currently 95 renewable assets are part of REIPPP.