[UNDER CONSTRUCTION] There has been ongoing debate over many years about the best paradigm to underpin discussion of organizational and sector strategy, monitoring, evaluation and related issues. This is sometimes put as the perceived conflict between contrasting quantitative and qualitative paradigms. Sometimes it is also seen as the difference between non-positivist and positivist paradigms, and at other times as the difference between subjective versus more objective paradigms.
The reality is that these contrasting paradigms help us to understand the world in different ways. The more quantitative (if we can call it that for this discussion) paradigm points to the extraordinary power of measurement and the experimental study of the world. The more qualitative paradigm points to the fact that not everything can be measured and that we don't want our strategic direction constrained to only that which we can easily measure just with quantitative methods. It also highlights the richness which comes from understanding the qualitative experience of others and of looking in detail at the complexity of the settings in which we work.
Systematic Outcomes Analysis is based on the belief that the points made in both of these paradigms are important. It is a response to the challenge of developing a common framework for dealing with outcomes, strategy, monitoring, evaluation, evidence-based practice and contracting, but one which allows both paradigms to make the claims which they wish to make, at the points where that is appropriate. In Systematic Outcomes Analysis no attempt is made to reconcile claims where they are in direct conflict - such attempts will not satisfy supporters of either paradigm. The approach which is taken is to allow claims to be made transparently so that stakeholders can then decide to either accept or reject them.
For instance, some of those who believe in the quantitative paradigm believe that only true experiments (and regression discontinuity designs) provide sufficiently robust outcome evaluation information about the effect of interventions and that other methods are usually a waste of time. Systematic Outcomes Analysis in the 5th building-block allows those who hold this view to specify the designs they believe are appropriate (out of the 7 possible outcome evaluation designs) and then to assess their feasibility and affordability. If they wish, they can determine that the other five designs in that building-block are not appropriate for answering the evaluation question they want answered. Their thinking is therefore made completely transparent and any stakeholder looking at their Systematic Outcomes Analysis first knows exactly the basis on which it has been developed and secondly, can decide to accept or reject their claims as they wish.
Equally, someone who comes from a strong qualitative paradigm may believe that experimental methods are inappropriate for studying outcomes for a particular type of intervention. On the basis of this they may chose to reject most of the 7 high-level outcome evaluation designs set out in the 5th building-block apart from the last (key informant judgement design). They would then simply indicate that the other six designs are not appropriate and not proceed to look at the feasibility or affordability of any of them apart from the last one (key informant judgement design). Again, using Systematic Outcomes Analysis this decision is made completely transparent and stakeholders can then decide to either accept or reject this claim.
This is in contrast to what currently happens in a number of evaluations, where decisions are not made transparent and those reading an evaluation report which is described as an 'outcome evaluation report' do not realize that the author taking a qualitative paradigmatic position has rejected a number of quantitative outcome evaluation designs just on the basis of their particular paradigm position. A similar situation applies in those cases when reading a quantitatively orientated evaluation report which is not transparent about the impact of the paradigmatic decisions which have been made. For instance, the author may not have even considered the possibility of undertaking qualitative studies which some stakeholders may believe would be a better option in an area where an experimental study is unlikely to yield interpretable results. The advantage of having done a Systematic Outcomes Analysis is that the claims which are being made can be very quickly identified by the reader.
The approach adopted in Systematic Outcomes Analysis of letting supporters of the contrasting paradigms make their claims in a transparent way opens up the possibility of getting rid of apparent conflicts between the paradigms where they should not exist. For instance, there is absolutely no justification for demanding that outcomes in an outcomes model be only quantitatively measurable outcomes which can be attributed to particular players. Outcomes models should set out what it is that you are trying to achieve regardless of the difficulty of measuring or attributing it to particular players. Those developing a Systematic Outcomes Analysis are expected to develop a comprehensive outcomes model, regardless of the paradigm they are coming from. Systematic Outcomes Analysis therefore identifies the places where the different paradigms can stake their different claims but at the same time insists on identifying sufficient common ground (e.g. comprehensive outcomes models do not have to be limited to just the measurable and attributable) so that a common framework can be used across the contrasting paradigms.
The advantage of a common framework is that it allows a common structure to setting out an outcomes system and if used across a number of different outcomes systems this helps those developing such frameworks. It also helps those who have to quickly assess how comprehensive an outcomes framework is. If it is has been developed using Systematic Outcomes Analysis, the reader or peer reviewer will be able to quickly locate the parts of the analysis they want to examine as they will be located under one Systematic Outcomes Analysis building blocks.