Systematic Outcomes Analysis uses an exhaustive set of ten possible types of economic analysis (grouped into three groups of analyses) in the Economic and Comparative Evaluation 7th building-block of the system). This list is used to establish exactly what economic evaluation is, and is not, possible for any intervention or set of interventions. Moving through the three overall groups of analyses, if a later analysis can be done, then by definition one of corresponding earlier analyses can also be done. So if you can do Analysis 3.2 you can also do 2.2, 2.1 and all of the Analyses 1.1-1.4. The analyses are grouped into three sets - those you can do when you do not have actual effect-size estimates for attributable outcomes above the intervention; those you can do when you have estimates for mid-level outcomes and those you can do if you have estimates for high-level attributable outcomes. In summary, you can only do the first grouping if you have estimated the cost of the intervention in the 5th building block (Checklist Step 6.1.1.2); for the second grouping you also need to have estimated mid-level outcome effect sizes in the 5th building block (by using one of the outcomes evaluation designs 1-4 in Checklist Step 6.1.1.1); for the third grouping you need to have estimated high-level outcome effect sizes in the 4th building block (by using one of the outcome evaluation designs 1-4 in Checklist Step 5.2.2).
In addition, another important prerequisite of any type of cost benefit analysis (Analyses 1.3,1.4,2.3,2.4,3.3,3.4 below) is that a comprehensive outcomes model has been developed. The robustness of a cost benefit analysis depends on it providing a comprehensive measurement of all of the relevant costs and benefits associated with an intervention. It is easy to distort the results of a cost benefit analysis in any direction you wish by simply leaving out either the costs or the benefits of one or more important outcomes. In Systematic Outcomes Analysis all cost benefit analyses should be mapped back onto an outcomes model. This lets the reader of such an analysis quickly overview what is, and what is not, included in the analysis and how this relates to the underlying outcomes model. It is not easy to assess the comprehensiveness of a cost benefit analysis without using this type of approach.
The set of prerequisites which exist between building blocks 5-8 are set out in the Prerequisites building blocks 5-8 diagram here.
The ten economic evaluation analyses grouped into three groups are:
1: No attributable outcomes above intervention
Analysis 1.1 Cost of intervention analysis, single intervention.
Cost of intervention analysis just looks at the cost of an intervention not its effectiveness (how much it costs to change an outcome by a certain amount) or the benefits (the result of subtracting the dollar cost of the program from the benefits of the program estimated in dollar terms). This analysis allows you to say what the estimated cost of the intervention is (e.g. $1,000 per participant).
Analysis 1.2 Cost of intervention analysis, multi-intervention comparison. Same as 1.1 but a multi-intervention comparison. This analysis allows you to compare the costs of different interventions (e.g. Program 1 - $1,000 per participant; Program 2 - $1,500 per participant or to put it in terms of Program 2 costing 1 1/2 times more than Program 1 per participant.
Analysis 1.3 Cost benefit analysis, set of arbitrary high level effect size estimates, single intervention. Even where you cannot establish any attributable outcomes above the intervention, but you do have an estimate of the cost of the intervention, you can just use some arbitrary (hypothetical) effect sizes. These can be used, if they can be estimated in dollar terms, to do a hypothetical cost benefit analysis (e.g. for a hypothetical effect size of 5%, 10% or 20%). It is essential that this type of hypothetical analysis is clearly distinguished from Analyses 3.3 which is based on estimates from actual measurement of effect sizes. This analysis allows you to estimate the overall benefit (or loss) of running the intervention if any of the hypothetical effect sizes were achieved (e.g. there would be a loss of $500 per participant for a 5% effect size, a gain of $100 for a 10% effect size and gain of $600 per participant for a 20% effect size).
Analysis 1.4 Cost benefit analysis, set of arbitrary high level effect size estimates, multi-intervention comparison. Same as 1.3 but a multi-intervention comparison. This analysis allows you to compare the overall loss or gain from more than one program for various hypothetical effect sizes (e.g. for a 5% effect size, Program 1 would have an estimated loss of $500 per participant whereas Program 2 would have a gain of $200 and so on, you could even theoretically vary the arbitrary effect sizes if there was some reason to believe that there would be differences, e.g. a general population program is likely to have a lower effect size than an intensive one to one program, but this may not say anything about the overall loss or gain when comparing two such programs). It is essential that this type of hypothetical analysis is clearly distinguished from Analyses 3.4 which is based on estimates from actual measurement of effect sizes.
2: Attributable mid-level outcomes
Analysis 2.1 Cost effectiveness analysis, attributable mid-level outcomes, single intervention. In this analysis, estimates are available of the attributable effect of the intervention on mid-level outcomes. When combined with the estimated cost of the intervention this allows you to work out the cost of achieving a certain level of effect on mid-level outcomes (e.g. a 6% increased in X cost approximately $1,000 per participant).
Analysis 2.2 Cost effectiveness analysis, attributable mid-level outcomes, multi-intervention comparison. Same as 2.1 but a multi-intervention comparison. This analysis lets you work out the cost of achieving a certain level of effect on mid-level outcomes for a number of interventions (e.g. a 6% increase in X cost approximately $1,000 per participant for Program 1 whereas it cost $1,500 for Program 2). It is likely that the measured effect sizes of different interventions will vary, therefore you may need to adjust estimates to a common base. This may or may not reflect what would happen in regard to the actual programs in reality.
3: Attributable high-level outcomes
Analysis 3.1 Cost effectiveness analysis, attributable high level outcomes, single intervention. Same as 2.1 except you can work out the cost of achieving a high level outcome effect size of a certain amount.
Analysis 3.2 Cost effectiveness analysis, attributable high level outcomes, multi-intervention
comparison. Same as 2.2 except you can work out the cost of achieving a high level outcome effect size of a certain amount and compare this across more than one intervention.
Analysis 3.3 Cost benefit analysis, attributable high level outcomes, single intervention. In this analysis, figures are available for the cost of the intervention, its attributable effect on high level outcomes, and the costs and benefits of all outcomes can be reasonably accurately determined in dollar terms. If this information is not available this type of analysis cannot be done. This analysis lets you compare the overall loss or gain from running the program (e.g. the program cost $1,000 per participant and other negative impacts of the program are estimated at $1,000 while the benefits of the program are estimated at $2,500 per participant. Therefore there is an overall benefit of the program of $500 per participant.)
Analysis 3.4 cost benefit analysis, attributable high level outcomes, multi-intervention comparison. Same as 3.3 but a multi-intervention comparison. This analysis lets you work out the overall cost or benefit for a number of programs compared (e.g. Progam 1 has an overall benefit of $500 whereas Program 2 has and overall benefit of only $200 per participant).
[Note: This list of designs is still provisional within Systematic Outcomes Analysis. For instance, there could theoretically be a 'cost benefit analysis, set of arbitrary mid-level effect size estimates, single intervention or multi-intervention comparison', however it is not clear why anyone would do this rather than 1.3 or 1.4 which sets arbitrary high level effect sizes. Comment on whether this is actually an exhaustive list of designs would be appreciated (send to paul (at) parkerduignan.com)].
[V1.1]