Impact Area Selection
At the start of an assessment, researchers must decide what (potential) impacts they want to analyse. This is the stage in the assessment with the greatest risk of introducing bias: If a relevant area of impacts is not included, the study will inevitably underestimate the positive or negative effects of investigated options. This may also disrupt a ranking of options, resulting in good options to appear worse than they really are (because their particular advantages are not seen) and bad options to appear better than they are (because their negative implications do not become evident). Which impact areas are likely to be relevant for a given assessment should be determined by a scoping study before the start of the actual assessment. Such scopings can be based on analysing scientific literature and policy documents, or on consulting with experts. Ideally, stakeholders should be involved in the process.
Assessment studies may not be able to address all of the impact areas that have been identified as relevant, due to limitations in time, funding, personnel or data availability. This mostly applies to studies based on modelling or measurements. While those studies are still highly valuable for generating knowledge for a sub-set of impact areas, they can not provide a full assessment or justify a ranking of options. For this, the results of those studies need to be complemented with data from other studies. A practical way to achieve a full assessment in spite of research limitations is to complement modelling or measurement approaches with expert assessments. In this case, the BonaRes Assessment Platform recommends to first determine which of the relevant impact areas can be addressed by modelling or measurement and to generate values based on expert assessment for the remaining ones.