Systematic Outcomes Analysis

A complete solution to outcomes, strategy, monitoring, evaluation and contracting

Case Studies


International Monetary Fund (IMF) evaluation

IMF evaluation strategy design

IMFreportthumbnail180px8k


The IMF Independent Evaluation Office were looking to develop an evaluation strategy for the International Monetary Fund's national and global economic survelliance work. Evaluating this activity presents the classic evaluation problem of attempting to evaluate whether an intervention is causing high-level outcomes to change. The difficulty in this case, as in many real-world situations, is that the high-level outcome (a well functioning economic system) is not controlled by the IMF - it is only one of the many factors which influence high-level outcomes. This IMF discussion paper uses Systematic Outcomes Analysis (under an earlier name - REMLogic) to work through this issue and provide an approach to monitoring and evaluating the IMF's work in this area. The paper discussing the evaluation design strategy is available here.

Academic research output assessment system evaluation strategy

Academic research output assessment system evaluation (PBRF)

TEC PFRF thumbnail 150px

This example details a monitoring and evaluation plan for a national system which was set up to assess academic research output (the Performance Based Research Fund - PBRF). Such systems present interesting challenges for monitoring and evaluation because they are monitoring and evaluation systems in their own right. Therefore the usual approach which is taken to evaluation - taking an aspect of a system and measuring it in more detail for the period of the evaluation - could end up just duplicating the system itself. This case study uses Systematic Outcomes Analysis under its earlier name of REM Logic, but it is exactly the same approach. The effort required to set out this monitoring and evaluation plan in narrative format and update the different components across the model (e.g. evaluation questions, indicators) as they were amended during discussions with stakeholders, led to the development of DoView outcomes software, which as used within the Easy Outcomes approach (a more user-friendly version of Systematic Outcomes Analysis) is a much more efficient way of setting out the content monitoring and evaluation plans.

New national building regulation system evaluation

National building regulation evaluation strategy

Newbuildingfrontpagethumb180px8kb


This example deals with the problem of how to monitor and evaluate a newly introduced national system of building regulation. The new system of building regulation was introduced to correct significant new building problems (weather tightness) which had occurred under the previous system. The new system is made up of a number of different interventions (professional registration, standards, new sign-off requirements) which it was hoped would impact in various ways to improve the quality of newly built buildings. A summary report and a full report on the evaluation strategy are available.

Copyright Paul Duignan 2005-2007 (updated March 2007)