[UNDER CONSTRUCTION] Systematic Outcomes Analysis is a comprehensive system and because of this, if you are new to it you may find quite a lot of detail in the system. This detail lets you do lots with the system once you start working with it, but it is helpful to start with a very simple overview of how it actually works. The system is made up of a set of building blocks which you can build one at a time. You don't need to build all of the building blocks to make use of the system. However, sometimes if you want to do some things in later building blocks, you will have had to do things in earlier in other building blocks - these are called prerequisites. The system can be used for any type of project, program, strategy, policy, organization, sector and even at the level of national and international outcomes. The way you apply the system is as follows:
1. Draw an outcomes model of all of the outcomes you want to achieve. When you are doing this, don't worry whether you can measure the outcomes or whether you can prove that that you alone made them happen. There are some rules for drawing these outcomes models so that will be useful later in the system, they are set out here. Check whether others think your outcomes model is realistic and whether there is any research to support your model. This is the 1st building block and it is called Outcomes models.
2. Work out your strategy. Do this by looking at your outcomes model and trying to work out the best intervention to do to get to your high-level outcomes. Work out who is going to do what. Strategy should be regularly reviewed and if you have done some of the other building blocks in the time between reviews they will help improve your strategy. This is the 2nd building block and it is called Strategy.
3. Measure outcomes. These measurements are called indicators. You may not have an indicator for every outcome in your outcomes model. When thinking about these indicators, don't worry whether they are ones which you can prove that you alone made happen. Draw the indicators back onto your model so you can see which outcomes you do and which don't have indicators. This is the 3nd building block and it is called Indicators[not-necessarily attributable]. Attributable means that you can prove that you did something.
4. Think about which of the indicators you found can be attributed to you (you can prove that you alone made them happen). These are likely to be on the lower levels of your outcomes model. Look for more attributable indicators that you have not yet identified. You may or may not have to do this step early on in the process. This is the 4th building block and it is called Indicators[attributable].
5. Focus on whether you can prove that your project has caused high-level outcomes to improve. This uses one or more of 7 possible outcome evaluation designs identified in the system. One or more of these may or may not be appropriate, feasible and affordable for you to use. Once you have worked this out you can defend your decisions about what outcome evaluation you can do to yourself and to any stakeholders who ask. This is the 5th building block and it is called Evaluation[outcome].
6. Focus on what other types of evaluation you can do. The system identifies six different questions you can focus on at this stage. These include how the outcomes model is being applied in your particular project (process evaluation), how to improve it (formative evaluation), the way people understand what is happening and the context of you project. This is the 6th building block and it is called Evaluation[non-outcome].
7. Work out what economic evaluation you can do and what comparisons you can make about whether your project is as effective as other projects. The system identifies two types of comparison you can do and three types of economic analysis each with four analyses in them. This is the 7th building block and it is called Evaluation[economic & comparative].
8. Identify the overall monitoring and evaluation scheme you are going to use. The system identifies two schemes, one where you do outcome evaluation for the full roll-out of a project and the other where you just do it for a pilot phase and use other types of monitoring and evaluation for the full roll-out. This is the 8th building block and it is called Overall monitoring & evaluation.
9. Start thinking about what contracting arrangement you will have between Funders and Doers. The system identifies three types of contracting; contracting for outputs only; for outputs AND managing for outcomes; and contracting for not fully controllable outcomes. This is the 9th building block and it is called Contracting.
You will find more information on the models used in the system here and a detailed checklist you can work through if you want to build any of the building blocks here.