Enterprise Challenge Fund (ECF) is a pilot program for the Australian Government as a mechanism to harness the innovation of the private sector to create sustainable developmental impact in the region. Measurement and evaluation of impact is critical to learning from a pilot program.
ECF results measurement approach
The 2009 midterm review of the program flagged the need for a results measurement system which was more detailed, periodic, frequent, and one that allowed the program management team on a timely manner if a project was performing as expected.
In early 2011 ECF looked at the Donor Committee for Enterprise Development (DCED) Standard for Results Measurement in order to devise a framework for measuring impact.
This included results chains with plausible causal links from the grant to the development impact, strong measurement plans for each project, improvements to the system including bringing on measurement specialists and researchers.
The approach also required looking at how information can be better used to support and improve the development outcomes in partnership with the private sector rather than just collecting information for reporting.
In September 2011, ECF underwent the first DCED pre audit review finding that the system was partially compliant with the standard and the findings suggested improvement to that system for the final 18 months. A DCED full audit was conducted in 2013 and the report has been published on the DCED website.
Read more about how ECF has developed its results measurement system here.
The changes to the system have been critical to evaluation of the fund. In her 2013 report, Heinrich highlights the positive example in the measurement systems designed for AusAID’s Enterprise Challenge Fund, which only requests data from companies that have “direct commercial impacts” and regularly collect data.
Further information on measurement in challenge funds
Challenge funds have tended to have a light touch management approaches and limited evaluations with an assumption that grantees (i.e. businesses receiving grants) will know best how to manage the grant and the program management team only required to monitor and check if activities went as planned.
This has created an information gap and there is currently little evidence to show whether challenge funds work in the short and long term. A 2013 report by Melina Heinrich from the DCED titled Donor partnerships with business for private sector development: What can we learn from experience? highlights that despite the high expectations on development outcomes usually associated with such approaches, the evidence on their effectiveness remains elusive, and little has been documented on lessons learnt from experience so far.
David Elliott from The Springfield Centre also comments “Evaluation and evidence must focus on the areas of output and behavioural additionality, and findings must feedback into modified designs”.
- 2013 Partnerships: Donor Partnerships with Business for Private Sector Development. What can we learn from experience (DCED Working Paper) March 2013
- 2013 Exploding the Myth of Challenge Funds - a start at least
- 2009 ECF Mid Term Review
- DCED website