Impact Area Selection
At the start of an assessment, researchers must decide what (potential) impacts they want to analyse. This is the stage in the assessment with the greatest risk of introducing bias: If a relevant area of impacts is not included, the study will inevitably underestimate the positive or negative effects of investigated options. This may also disrupt a ranking of options, resulting in good options to appear worse than they really are (because their particular advantages are not seen) and bad options to appear better than they are (because their negative implications do not become evident). Which impact areas are likely to be relevant for a given assessment should be determined by a scoping study before the start of the actual assessment. Such scopings can be based on analysing scientific literature and policy documents, or on consulting with experts. Ideally, stakeholders should be involved in the process.
Assessment studies may not be able to address all of the impact areas that have been identified as relevant, due to limitations in time, funding, personnel or data availability. This mostly applies to studies based on modelling or measurements. While those studies are still highly valuable for generating knowledge for a sub-set of impact areas, they can not provide a full assessment or justify a ranking of options. For this, the results of those studies need to be complemented with data from other studies. A practical way to achieve a full assessment in spite of research limitations is to complement modelling or measurement approaches with expert assessments. In this case, the BonaRes Assessment Platform recommends to first determine which of the relevant impact areas can be addressed by modelling or measurement and to generate values based on expert assessment for the remaining ones.
Impact area selection is the process of deciding which impacts are going to be investigated in a given assessment. Because this selection is highly relevant for the assessment results, the selection process should be systematic, transparent and well documented. The rationale on which the selection is based should be purely thematic, i.e. based on the question which impacts are likely to be affected.
Selecting impact areas based on data availability will not produce reliable assessment results that are suited to evaluate sustainability or compare options.
Investigating impacts is complicated and usually requires thorough planning, as well as one or more of the following: calibrating and running models, conducting measurements, setting up and performing experiments or preparing literature reviews, stakeholder consultations or expert interviews. For this reason, impacts that are not specifically investigated, will most likely not be detected.
Where a management or policy option investigated in an assessment has significant positive or negative impacts, not addressing them will inevitably result in a biased assessment result. As an example, please consider two hypothetical management options A and B, both with their respective advantages and disadvantages. Figure 5 shows their performance for 11 impact areas from the assessment perspective ecosystem services. In the figure, impacts are given as percentages relative to a business as usual option, with values higher than 100 indicating a higher provision of the respective ecosystem service and values lower than 100 indicating a lower provision of the services.
Figure 5: Two hypothetical management options (A and B) and their performance for 11 ecosystem services. Values represent percentages of service provision relative to business as usual.
In the example, both A and B have advantages and disadvantages, with neither being clearly better than the other. However, if only a sub-set of impact areas are addressed, this may change and depending on the selection, either option A (Figure 6) or option B (Figure 7) may appear to be more favourable.
Figure 6: Two hypothetical management options (A and B) and their performance for 6 out of the 11 ecosystem services presented in Figure 5. Values represent percentages of service provision relative to business as usual. Option A appears to be more favourable than option B.
Figure 7: Two hypothetical management options (A and B) and their performance for 6 out of the 11 ecosystem services presented in Figure 5. Values represent percentages of service provision relative to business as usual. Option B appears to be more favourable than option A.
Not addressing all relevant impact areas will introduce a bias that makes the assessment results unreliable, even though all data generated may be state of the art. In a worst-case scenario, this may result in providing incomplete information to decision makers and lead to sub-optimal management or policy choices which miss opportunities or have unforeseen, negative consequences.
Impact areas likely to be affected by an option may be identified through a (short) literature review, expert consultation, or a combination of both approaches. Ideally, stakeholders are also involved in the process to add their perspective as to what impacts should be considered.
For including stakeholders’ views, assessments may adapt an approach applied in sustainability reporting, which is called “materiality analysis” which is. It is used to identify topics for reporting that are material (i.e. relevant) and rank them according to their importance. Only the material topics are be reported and the ranking determines the amount of detail with which each topic is covered (GRI Standards, 2016). For a materiality analysis, the Global Reporting Initiative recommends to create a matrix, where one axis represents the expected seriousness of economic, ecological or social impacts and the other axis represents the importance of the respective impact area for stakeholders. Impact areas that score high on either of the axes are considered material (GRI Standards, 2016).
Figure 8: GRI materiality matrix (GRI Standards, 2016)
As part of the GRI standards, there is no fixed requirement on how the importance for stakeholders should be assessed. Consultations and workshops are possible, as are in-house assessments by the reporting organizations. In the context of impact assessments, this platform recommends direct interaction with a well documented and balanced selection of stakeholders. Where this is not possible, an analysis of policy strategy documents to identify relevant impact areas may be used as an alternative (Hermanns et al., 2017), though both approaches should ideally complement each other.
A materiality analysis always needs to be performed for a specific assessment, because the impact areas identified and the prioritization derived are based on the socio-economic and geophysical conditions at a certain point in time and space. Therefore, results of materiality analyses cannot be readily transferred to similar assessments at other locations. Even at the same location, the relevance of impact areas and their respective priorities may change over times due to the dynamics of norms and values, as well as due to advances in knowledge.
Global Reporting Initiative (GRI). 2018. GRI Standards. Online: https://bit.ly/2ePTyM3 (accessed 09 August 2019)
Hermanns T, Helming K, König H J, Schmidt K, Li Q, Faust H. 2017. Sustainability impact assessment of peatland-use scenarios: Confronting land use supply with demand. Ecosystem Services 26(B): 365-376. DOI:10.1016/j.ecoser.2017.02.002