Systematic Outcomes Analysis

A complete solution to outcomes, strategy, monitoring, evaluation and contracting

Outcome model standards

A set of standards has been developed for drawing comprehensive and technically sound outcomes models which can be used as the basis of any Systematic Outcomes Analysis. These standards are set out in full here and summarized below:

1. Use outcomes not activities. You can change an activity (doing) into an outcome (done) by just changing the wording (for instance changing: Increasing citizen participation to Increased citizen participation).

2. Outcomes models can include any of the 'cascading set of causes in the real world'. The steps that are put into models do not have to be limited to measurable, attributable or accountable outcomes. Attributable outcomes are those for which changes can clearly be attributed to an individual player. For a brief discussion of the different features of outcomes see here. There is usually a lot of resistance to including in outcomes models non-measurable and non-attributable outcomes. This is because stakeholders are wanting to manage their risk around being held to account for achieving all of the outcomes they put into such models. This is a genuine risk but it is managed in Systematic Outcomes Analysis by measurement, attribution and accountability being dealt with in a separate stage after the building the outcomes model. An outcomes model that is limited to measurable, attributable or accountable outcomes is usually useless for strategic planning (it limits you to trying to do the measurable and/or attributable rather than the important). It is also of limited value for monitoring and evaluation planning as it only lets you visualize what you already know rather than what you do not yet know (which is usually what you are trying to explore in monitoring and evaluation planning).

3. Don't force your outcomes model into particular horizontal 'layers' within the model. Often outcomes models are divided up into a number of layers such as - inputs, outputs, intermediate outcomes and final outcomes.  However whether or not something is an output is simply a result of its measurability and attributability (see the section on the features of outcomes). Therefore, in some models outputs may reach further up one side of a model than another. Forcing artificial horizontal layers onto an outcomes model distorts it and makes it harder for stakeholders to ‘read’ the logical flow of causality in the model. The concept of outputs is useful for accountability purposes and they can be identified at whatever level of a model they are located at a later stage after the outcomes model has been draw without demanding horizontal banding of into outputs, intermediate outcomes etc.

4. Don't 'siloize'. Siloizing is when you draw an outcomes model in a way that artificially forces lower level outcomes to only contribute to separate high level outcomes. In the real world, good lower level outcomes can contribute to multiple high level outcomes. Any outcome can potentially contribute to any other outcome in a model, the way you draw the model should allow for this.

5. Use 'singular' not 'composite' outcomes. Composite outcomes contain both a cause and an effect (e.g. increase seat-belt use through tougher laws). Outcomes like this should be stated as two separate outcomes. The use of words like through, or by in an outcome show that you are looking at a composite, rather than a singular outcome. Composite outcomes permanently lock an outcome with a particular strategy (e.g. increased seatbelt use in this case with the strategy of tougher laws); dividing these into separate outcomes gives more analytical power to your outcomes model because it allows you to consider the possibility that other strategies could lead to the outcome which is being sought.

6. Keep outcomes short. Outcomes models with wordy outcomes are hard to read. Include separate descriptive notes with each of your outcomes if you need more detail on them.

7. Put outcomes into an hierarchical order. Use the simple rule that you can tell that outcome A is above outcome B in an instance where, if you could magically make A happen, you would not bother trying to make B happen.

8. Each level in an outcomes model should include all the relevant steps needed to achieve the outcome(s) above it.

9. Keep measurements/indicators separate from the outcomes they are attempting to measure. Measurement should not be allowed to dominate an outcomes model. Within Systematic Outcomes Analysis measurement is introduced at a later stage after the outcomes model has been built. In those relatively small number of cases where a measurement also acts as an intervention in its own right (e.g. some audit procedures), then it can be included as an outcome within a model.

10. Put a 'value' in front of your outcome (e.g. suitable, sufficient, adequate). You do not need to define this at the time you build your outcomes model. If it is not clear exactly what it amounts to, it can become the subject of an evaluation project at a later stage.

11. Develop as many outcome 'slices' as you need (but no more). In an outcomes model you are trying to communicate to yourselves and to other stakeholders the nature of the world in which you are trying to intervene. Slices can be seen as a series of cuts through the world of outcomes in your area of interest. For instance you might have slices at the national, locality, organizational and individual level. The trick is to get the smallest number of slices needed to effectively communicate the relevant outcomes in the model.

12. Do not assume that you need a single high-level outcome at the top of an organization's outcomes model. Outcomes models should be about the external world, not just about your organization. Often organizations are delegated to undertake interventions in a number of areas that are best modeled separately. This is a better approach than artificially trying to force outcomes relating to different areas under a single integrated high level outcome.

13. Include both current high-priority and lower priority outcomes. Your outcomes model should be as accurate a model as you can draw of the  ‘cascading set of causes in the real world’; therefore, it should not just be about the current priorities you can afford to work on if they are a sub-set of the wider outcomes picture. Within Systematic Outcomes Analysis you map a (typically more limited number of) priorities onto your more comprehensive outcomes model after you have built the model. This allows you in the future to think strategically about alternative options and to change your priorities. If your outcomes model only includes your current priorities it gives you no steer as to how your current priorities map onto the real world.  In a public sector context this also allows outcomes models to support public officials providing ‘free and frank advice’ about how the world is – i.e. the 'cascading set of causes in the real world'. It is then up to elected government officials to decide what their priorities will be and these can be mapped onto the underlying outcomes model. This approach means that outcomes models do not have to change every time there is a change in particular elected officials or the government as a whole. If elected officials' priorities change this is reflected by mapping their priorities onto the more comprehensive outcomes model and by the public officials the moving to carry out these priorities.

[V1.1]

[This summary is drawn from Duignan, Paul (2007). Visualising outcomes in social policy: constructing quality outcomes sets for maximising impact. Social Policy, Research and Evaluation Conference, Wellington New Zealand, 5 April 2007.]

Copyright Paul Duignan 2005-2007 (updated March 2007)