[Draft Entry Currently Being Revised). The seventh step in Systematic Outcomes Analysis is economic and comparative evaluation (this uses the Economic and Comparative Evaluation Building-Block). It focuses on comparing the effectiveness of different interventions and, in the case of economic evaluation, assigning positive and negative monetary value to aspects of interventions and outcomes.
Step 7.1 Comparing the effectiveness of different interventions on similar outcomes
7.1.1 The effectiveness of different interventions can be compared to find out which intervention is better at achieving similar outcomes. This activity, which is an important part of evidence-based practice, relies on there being a good measure of the effect size of the intervention. The effect size is the amount of change in an outcome which can be attributed to a particular intervention. For a quantitative effect size to be available for this type of analysis, one of four outcome evaluation designs (Designs 1-4) need to have been used to answer the high level outcome evaluation question (see Step 5: High Level Outcomes Evaluation. Stakeholders may have differing views as to which outcome evaluation designs they'll accept as actually providing sufficiently robust proof of an effect on high level outcomes.
If this type of effectiveness comparison is used as the major factor used in deciding between different interventions, it should only be used where the each of undertaking outcome evaluation designs 1-4 (see here for these designs) is similar for each of the interventions. Otherwise the decision maker can end up just doing the easily evaluable rather than the potentially effective. It also needs to be realized that even in those cases where one can fairly compare a number of interventions in terms of improving outcomes, the type of comparative analysis dealt with here only shows which one is the most effective but is silent on how much different interventions cost to implement. A very effective intervention that most of the population cannot afford, may be less desirable than a somewhat less effective intervention which, however, costs a lot less. The effect size is the amount of change in an outcome which can be attributed to a particular intervention. For a quantitative effect size to be available for this type of analysis, one of four outcome evaluation designs (Designs 1-4) need to have been used to answer the high level outcome evaluation question (see Step 5: High Level Outcomes Evaluation.
Comparison 1: Quantitative multi-intervention effect size meta-analysis
Comparison 2: Mixed qualitative/quantitative multi-intervention comparison analysis
Step 7.2 Identify what economic evaluation analyses are possible
7.2.1 Decide which of the ten economic evaluation analyses used in Systematic Outcomes Analysis can be done in the case of your program. Whether you'll be able to do one or more of these depends on what you've discovered is appropriate, feasible and affordable in the fourth step (High Level Outcome Evaluation). The economic analyses below are grouped into three sets - those you can do when you do not have actual effect-size estimates for attributable outcomes above the intervention; those you can do when you have estimates for mid-level outcomes and those you can do if you have estimates for high level attributable outcomes. In summary:
- you can only do the first group of analyses if you have estimated the cost of the intervention in the 6th Step (6.1.1.2 Focus 3: Describing an outcomes model that is actually being implemented in a specific instance);
- you can only do the second group of analyses if you have measured the mid-level outcome effect sizes in the 6th Step (6.1.1.1 Focus 2: Establishing whether a particular intervention has caused an improvement in mid-level outcomes);
- you can only do the third group of analyses if you have measured the high level outcome effect size in the 5th Step (by using one of Designs 1-4 in Step in Step 5.2.2).
More information on what these types of economic evaluation analyses consist of can be found in the model here. The ten possible types of economic evaluation analysis within their three groups are: