Implementing Impact Assessment
Measurement, Modelling or Expert Assessment
Once the impact areas have been selected, researchers are faced with the challenge to generate values that represent the impacts within them. Typically, researchers generate these values from either measurement, modelling or expert assessment. The three approaches are outlined below with a short description of their respective strengths and weaknesses.
Experiments or empirical studies can be used to generate the required information, usually in the form of quantitative data. Researchers first need to decide which indicators to use for each impact area. After that, they may either conduct the measurements themselves or analyse results of published studies. This approach is supported by the BonaRes Assessment Platform through factsheets that list sample indicators for each impact area and reference examples of their application in current research.
In the context of impact assessments, core strengths of approaches based on measurements are the high level of detail that can be achieved, the low need for a-priori assumptions and the comparatively low risk of bias introduced by the researchers. Weaknesses are the high amount of time and effort required, which may limit the number of impact areas that can be covered, and the difficulty to measure effects of future scenarios that have no parallel in the present. Furthermore, the generalisation of measurement results is often challenging, which limits their transferability to other locations or situations.
Where a high degree of prior knowledge on causal relationships already exists, researchers may use models for assessing impacts. Usually, models are designed to produce quantitative data, although qualitative models are also used. As above, researchers first need to select indicators to represent the respective impact areas. The information provided by the factsheets on the BonaRes Assessment Platform can be used to select indicators that are part of the output of existing models or to conceptualize requirements for new models.
In the context of impact assessments, core strengths of modelling approaches are the reproducibility of results, opportunities to investigate in detail the link between effects and underlying causes, including analyses of sensitivity, and options to analyse future scenarios that have no parallel in the present. Weaknesses are the high requirement of time, effort and information for generating new models, which also limits the options of tailoring a model to the needs of a specific assessment, and the high number of assumptions that need to be made by researchers in order to represent complex socio-ecological systems in a (simplified) model. Nevertheless, substantial progress has been made in the last decade in integrated modelling and model coupling to address complex socio-ecological systems and assess land use impact scenarios (Helming et al., 2011; Ewert et al. 2015; Lotze-Campen et al., 2018).
Experts may be consulted to asses impacts, based on their expertise and experience with regard to the topic and the selected impact areas. Values for the impacts are usually provided as qualitative data, often expressed on an ordinal or categorical scale (e.g., a Likert-type scale with ratings ranging from “very negative” to “very positive”). In the context of expert assessments, the indicators provided by the factsheets on the BonaRes Assessment Platform can be used to specify what the expert assessment is based on and to facilitate communication between different experts. For example, if experts evaluate how the ecosystem service: provision of habitats would be affected by a specific agricultural management, they could draw on the indicators to explain whether their assessment is mainly based on the expected species diversity of farmland-birds, or on their abundance, or on the abundance of endangered plant species, or on a mix of criteria.
In the context of impact assessments, core strengths of expert based approaches are fourfold: the adaptability to the specific context of the impact assessment, the low amount of time, work and data required, the opportunity for stakeholder involvement and shared learning. In addition, biases of impact area selection due to data availability can be avoided. Weaknesses are the danger of introducing bias through the selection of the consulted experts, the risk of low reproducibility with other experts or with the same group of experts at another point in time, and the lack of transparency of how the experts arrive at their assessment result. Thorough and comprehensible protocols for expert selection should be used to minimize the risk of introducing bias.
More information on expert based assessments is also provided in the section titled: Framework of Participatory Impact Assessment (FoPIA).
All three approaches have their limitations and advantages. For very complex assessments, it is therefore advisable to combine one or more of them. For example, an expert based assessment could be done first as a scoping activity to address a high amount of impact areas and identify critical ones. These impact areas could then be investigated in more detail through measurements or modelling activities.
Ewert F, Rötter, R P, Bindi M, Webber H, Trnka M, Kersebaum K C, Oleseng J E, van Ittersum M K, Janssen S, Rivington M., Semenov M A, Wallach D, Porter J R, Stewart D, Verhagen J, Gaiser T, Palosuo T, Tao F, Nendel C, Roggero P P, Bartošová L, Asseng S. 2015. Crop modelling for integrated assessment of risk to food production from climate change. Environmental Modelling & Software, 72, 287–303. DOI: 10.1016/j.envsoft.2014.12.003
Helming, K., K. Diehl, T. Kuhlman, T. Jansson, P. H. Verburg, M. Bakker, M. Perez-Soba, L. Jones, P. Johannes Verkerk, P. Tabbush, J. Breton Morris, Z. Drillet, J. Farrington, P. LeMouël, P. Zagame, T. Stuczynski, G. Siebielec, S. Sieber and H. Wiggering. 2011. Ex Ante Impact Assessment of Policies Affecting Land Use, Part B: Application of the Analytical Framework. Ecology and Society 16 (1): 29. Online: https://bit.ly/2GQMWcs (accessed 09 August 2019)
Lotze-Campen, H., Verburg, P., Popp, A., Lindner, M.,Verkerk, P., Lindner, M., Moiseyev, A., Schrammeijer, E., Helming, J., Tabeu, A., Schulp, C., van der Zanden, E., Lavalle, C., Walz, A., Bodirsky, B. (2018). A cross-scale impact assessment of European nature protection policies under contrasting future socio-economic pathways. Regional Environmental Change, 18(3), 751–762, DOI:10.1007/s10113-017-1167-8
-
Measurement, Modelling or Expert Assessment
- Multi Criteria Analysis
- Framework of Participatory Impact Assessment
- Benchmarking