We then get an estimate of how the poverty status of both the intervention and comparison groups has changed since the baseline. Consider whether creating a quasi-experimental design could also be done at stages of the project other than the final stage. First, let me say that this is brilliant. But to really know what success entails, change how you perceive your place in this world and the life you live. However, we do try to get respondents to recall baseline data, particularly data we assume can be reliably recalled. You can unsubscribe at any time by clicking the link in the footer of our emails. I like the focus on reporting both effective and ineffective results. But you seem to have all of that on your radar. Be honest and define success … This would probably be the next best thing to having experimental data. Some goals are operational and unrelated to the nonprofit’s main purpose, but help it pursue its mission. I have posted a couple of interesting presentations on that issue on my blog: http://www.developblog.org/2012/10/evidence-of-what-worked-at-some-point.html and http://www.developblog.org/2012/05/participatory-statistics-and-more.html A more purposive approach would also enable you to limit RCTs to those kinds of projects where they make sense. The key to their success is capturing quality data on plausible comparison populations and key factors that influence programme participation, and this has worked out better in some cases than in others. We have now scaled down our ambitions to 30-ish, to both avoid overburdening the organisation and enable better quality control. For instance, if you wish to find out about the effectiveness of a specific theory of change in a specific sector, you could organise a set of case studies in a range of countries. Hopefully this will inspire others in the sector, not only to put evaluations out there more sytematically, but to make them more usable (are you listening e.g. Make sure your gift to Oxfam America goes directly to where it's most needed, without delay, with a secure online donation. Performance measures in the cyber-security field, being a newer discipline, does not have the same level of interest and KPIs in the way that other areas do. Measuring the success of your service. Wonk digression… the propensity score matching (somewhat) circumvents the need to spend money on expensive baseline surveys. I strongly support your experiment and transparency. While a candidate appears for the interview, the question that can put an impact on the interviewee is ‘How Do You Measure Success’ to understand how does the candidate measure or evaluate success. Simplicity is the ultimate sophistication. When our service desk does a simple count of the number of issues reported in a day, we have a measure. Oxfam’s research team is in the process of commissioning some qualitative research to drill down on this project to better understand what made it work. Required fields are marked *. Now all long-term programs, major campaigns, and key innovation projects have a rigorous monitoring, evaluation, and learning system. Sharing is perhaps the true measure of all success, be it a wealth of money, time, patience, knowledge, wisdom or good will. Here, the intervention group reported receiving, on average, about 48 hours of advanced warning of the devastating floods that hit Pakistan in the late summer of 2010, as compared with only 24 hours for the comparison group. Hats off Karl. Oxfam's first paid employee was Joe Mitty, who began working at the Oxfam shop on Broad Street, Oxford on 9 November 1949. The two cannot be separated. (Looking to make a monthly donation, fundraise for Oxfam, or give some other way? That is not just a matter of numbers – many development interventions are just too complex for RCTs to yield meaningful insights. The organization is run by … Oxfam America is rated 3 out of 4 stars by Charity Navigator. Just over a year ago now, I wrote a blog featured on FP2P – Can we demonstrate effectiveness without bankrupting our NGO and/or becoming a randomista? It concludes an investigation of significant scale and complexity. – stakeholder interest, I’ve just blogged about the reviews: http://ngoperformance.org/2012/10/17/hats-off-to-oxfam-but-are-they-asking-the-right-question/. We help people build better futures for themselves, hold the powerful accountable, and save lives in disasters. What we are doing is meant as a complement, not a replacement, for the more involving (and costly!) The key to their success is capturing quality data on plausible comparison populations and key factors that influence programme participation, and this has worked out better in some cases than in others. How can you say that 40 x £10,000 was good value for money without knowing what action the country teams actually took….that deifnes the value of the investment – ie, the objective of all this effort? THE POWER OF PEOPLE AGAINST POVERTY OXFAM STRATEGIC PLAN, 2013 – 2019 7 . As other NGOs, we’ve a lot to learn from your method and findings. Very interesting, thanks for sharing. women’s empowerment and ‘resilience’. Measuring impact: Case studies. Oxfam's 6 goals put local communities and the voices of poor people at the centre of change - our best hope for ending the injustice of poverty. Impressive. Oxfam is a global organization working to end the injustice of poverty. Some are also concerned about how much time and resources the effectiveness reviews are sucking up. This is a very encouraging sign from one of the world's largest non-governmental organisations. In late 2005, Oxfam established a department dedicated to learning, evaluation, and accountability. Oxfam America receives 82.27 out of 100 for their Charity Navigator rating. Help people claim their right to a better life However, we are struggling to ensure consistent application of this protocol.