Intervention Logic:
How to build outcomes hierarchy diagrams
Using the OH Diagramming Approach
V1-1 29-8-04
Strategic Evaluation Working Paper
Duignan, P. (2004, August 29). Intervention logic: How to Build Outcomes Hierarchy Diagrams Using the OH Diagramming Approach [WWW document]. From The Strategic Evaluation Web Site. URL http://www.strategicevaluation.info/se/documents/124pdff.html
Dr Paul Duignan1
Parker Duignan Ltd
Comments on this paper gratefully received to: paul@parkerduignan.com
If using models from this paper in any context please source them to this document, the reference format is on the front cover.
Introduction
This paper is a technical working paper for those interested in developing intervention logics (programme logics) using the OH Diagramming Approach. Intervention logics set out the connections between the outcomes an individual, organisation or group of organisations are trying to achieve and the steps, stages or intermediate outcomes which are needed to achieve these. They also, in some instances, include elements other than intermediate outcomes e.g. activities, outputs. There are a range of ways of setting out intervention logics which include table and diagram formats. This paper focuses on developing one type of intervention logic – an outcomes hierarchy diagram using Dr Paul Duignan’s OH Diagramming Approach. The technical principles behind outcomes hierarchies are set out in another paper2.
The OH Diagramming Approach
The OH Diagramming Approach has the following features:
· Outcomes hierarchies are set out as diagrams
· Final outcomes are placed at the top of the diagram
· Elements are expressed as outcomes rather than processes (see below)
· Any number of links are allowed between outcomes within a diagram
· Where there are large numbers of links between elements in a diagram, for the sake of readability, the outcomes with many links are often drawn as larger rectangles which indicate that they relate to a number of other outcomes (see below)
· A line around the outside of the core of the diagram is used to indicate the outcomes which are the direct concern of the project, programme or organisation for which the OH diagram is being drawn. This allows additional external outcomes which link out from the project, programme or organisation’s logic to be shown on the same diagram.
How OH Diagramming Outcomes Hierarchies can be used
OH Diagramming outcomes hierarchies can be used for:
· Programme planning
· Communicating a programme’s approach to stakeholders
· Identifying the important areas in a programme which need to be reviewed from the previous literature and experience
· Developing indicators
· Designing evaluation questions
· Identifying research and evaluation priorities (see the REMLogic Outcomes Approach3).
Building an Outcomes Hierarchy Diagram
An outcomes hierarchy is best initially built by a group of people which includes people close to a programme plus a person who has some experience or skill at building outcomes hierarchies. It is usually best to keep the initial working group as small as possible, three or four is a good number for the initial work on an outcomes hierarchy. The draft hierarchy can then be taken to a wider group of stakeholders for discussion and amendment.
OH Diagramming outcomes hierarchy diagrams are built using the following steps:
Step 1: Work out what are the final outcomes that an organisation, programme, project or intervention, is trying to achieve. Put these at the top of the diagram.
Step 2: Write the elements in the diagram as outcomes not at processes or activities. A process or activity (e.g. increasing employment) can be changed into an outcome just by changing its wording (e.g. increased employment).
Step 3: Distinguish between a process diagram and an outcomes diagram. Process diagrams (useful for a different set of purposes than outcomes hierarchies) focus on the processes rather than the outcomes that processes are trying to achieve. The diagram below sets out a process diagram for passengers going through airport security.
In contrast to the process model above, the diagram below sets out an outcomes hierarchy again for passengers going through airport security.
The process model can be used for training and workflow analysis. The outcomes hierarchy can be used to identify alternative ways of achieving the outcomes and for any of the purposes of an outcomes hierarchy listed above.
Step 4: Work down from the highest level outcome(s) asking the question at each stage, what lower level outcome(s) need to be achieved in order to achieve this higher level outcome(s).
Step 5: For any higher level outcome, the outcomes hierarchy diagram should include below it all of the intermediate outcomes which are needed to achieve that higher level outcome. The diagram below shows an example in the left-hand side outcomes hierarchy where an essential outcome is missing (the third one). Therefore the left-hand hierarchy is a badly formed outcomes hierarchy.
A practical example of this would be if the higher level outcome was reduced drink driving and the lower level outcomes were first – adequate levels of enforcement, second – adequate penalties and third – adequate driver awareness of enforcement. The outcomes hierarchy on the left does not include the third necessary lower level outcome - adequate driver awareness of enforcement - and therefore the higher level outcome would be unlikely to be achieved if this outcomes hierarchy was used as the basis for developing a programme.
Continue with the process of identifying the outcomes that are needed all the way down the outcomes hierarchy.
Step 6: In the process of developing an intervention logic it is usual that a number of different suggested lower level outcomes will be proposed by the group working on the outcomes hierarchy. The way to decide the relative order of two lower level outcomes A and B is to ask the question – if we were automatically able to achieve outcome A, would we bother to do outcome B? If the answer is no, then we know that outcome B is below outcome A. For instance, if automatically drink driving suddenly was at a very low level (not because of enforcement), we would be unlikely to continue to spend a lot on drink driving enforcement as the purpose of this activity would have already been achieved.
Step 7: Once an outcomes hierarchy has been build, check its soundness by starting at the top high level outcome(s) and ask the question – do the outcomes at the next level down set out all of the lower level outcomes which are needed to achieve this higher level outcome? Work all the way down the outcomes hierarchy in this way.
Step 8: Once an initial draft of an outcomes hierarchy has been built by a small group using the OH Diagramming Approach, it can be used as the basis for discussion and amendment by wider groups of stakeholders. It is useful if such discussions involve people who worked on the original version of the outcomes hierarchy so that they can help explain what is included in the outcomes in the diagram and why decisions were made to put the outcomes in the particular way they have been put.
Outcomes Hierarchies are Practical Tools
Outcomes hierarchies should be practical tools; there are always a number of ways to design an outcomes hierarchy. When designing an outcomes hierarchy keep the following in mind:
· Keep diagrams as simple as possible
· Remember that they are communication tools
· Where possible, keep them in one page elements so that they are easy to distribute, print, read from screen and understand. For instance, have a one page high level logic followed by other single page logics drilling down further
· Outcomes hierarchies are a two dimensional representation of a complex multi-dimensional world; because of this there is usually going to be more than one way in which they could be drawn
· Look for one good way to draw the hierarchy rather than the single only way of drawing it
· Remember that outcomes hierarchies are likely to evolve over time as more knowledge accumulates about a project, programme or organisation.
· Conceptually there can be a number of different types of outcomes hierarchies for a project, programme or organisation – the planned outcomes hierarchy (or intervention logic); the theoretically and research justified (based on the available research evidence and expert opinion); and the as implemented outcomes hierarchy (based on what actually happened in practice).4
Examples of OH Diagramming Outcomes Hierarchies
The diagram below shows an example of an outcomes hierarchy built using the OH Diagramming Approach.
Points to note in this diagram are that outcomes can link across levels in the outcomes hierarchy. For instance, Outcome 4.3 is a lower level outcome for both Outcome 3.2 and Outcome 2.2. In addition, Outcome 5.1 is an outcome which is linked to all of the other outcomes (at levels 2 and 3) in the diagram. Technically there should be lines between Outcome 5.1 and the outcomes which it is linked to, but to draw them in would produce a diagram which is too complex for the normal reader.
The diagram below is another example of an OH Diagramming outcomes hierarchy – in this case in regard to an evaluation course. This outcomes hierarchy has been used to develop a set of indicators.
