Benchmarking studies can be applied with different degrees of complexity. In the simplest form, only one criterion is used to evaluate performance and a best practice is identified which scores highest for this criterion. The best practice is then used to evaluate all other samples and to serve as an example. For instance, several wheat producing farms in a region could be compared in a benchmarking to identify the one that achieves highest yields. All other farms could then be evaluated relative to this and farmers could try to use the example to optimize their own management.
In a more complex form, several criteria are evaluated and the information of fall samples are mathematically combined to form a so called “efficiency frontier”. The most common parametric method for this is Stochastic Frontier Analysis (SFA), the most common non-parametric method is Data Envelopment Analysis (DEA). Both methods have been developed as economic tools to evaluate the performance of companies, enterprises or non-profit organizations, which are referred to as “decision-making units (DMU)” in the literature (Malana & Malano, 2006). The abstract term DMU is used to highlight that the methods are applicable for all types of organizations where management (decision making) occurs. In the context of soil related impact assessments, the DMUs are usually farms.
Both SFA and DEA address the question of how the highest amount of positive outputs can be achieved with the lowest amount of inputs and the lowest amount of negative outputs . For this, they calculate an efficiency frontier line, which represent the (hypothetical) optimal combination of inputs (Oh & Hildreth, 2016). The efficiency of each DMU is calculated as the distance to this efficiency frontier and often reported as a percentage (with DMU positioned on the frontier line having an efficiency of 100%). SFA and DEA can be used to estimate overall efficiency of farms or management processes and rank them based on their performance. Additionally, the methods can be used to identify areas of inefficiency in order to support improvement.
Stochastic Frontier Analysis (SFA)
Stochastic Frontier Analysis is a method developed from stochastic frontier production function models (Aigner et al., 1977). SFA is based on parametric models. Such models are defined a-priori except for a finite set of unknown parameters that need to be estimated from data. In SFA, these parameters often include effects of noise (random shocks unrelated to management, such as weather effects) and efficiencies (Bogetoft & Otto, 2010). An efficiency frontier is calculated based on actual input/output relationships, and the efficiencies of all DMU are measured relative to it. It is not necessary for any DMU in the sample to be positioned on the efficiency frontier, i.e. to achieve full efficiency.
With SFA approaches, hypotheses about production structure and degree of inefficiency can be assessed statistically. However, the method requires explicit parametric functions representing relationships between inputs and outputs, and distributional assumptions. For considering measurement errors and effects which cannot be controlled by the individual DMU, such as weather, an error term is incorporated in the calculations. The validity of SFA results depends on the accuracy of the assumptions made (Odeck, 2007).
Data Envelopment Analysis - DEA
DEA calculates the efficiency frontier line (highest positive outputs from lowest amount of inputs and negative outputs) from linear combinations of the samples. DMU that score highest in at least one criterion will always be positioned on the efficiency frontier. Two types of DEA models can be distinguished: The input-oriented model aims towards minimizing inputs while holding the amount of outputs constant. Output-oriented models on the other hand aim to increase outputs with the same amount of inputs (Malana & Malano, 2006).
As a difference to parametric models, non-parametric approaches such as DEA do not require any a-priori assumptions regarding the quality or distribution of the data, avoiding potential errors in this regard (Oh & Hildreth, 2016). However, this makes the methods sensitive to measurement errors or other noise affecting the dataset. In DEA, all parameters are calculated from the data in the sample. Environmental factors are not explicitly considered. It is therefore important to carefully select the sample to avoid comparing farms that work under different environmental conditions (Odeck, 2007).
Both SFA and DEA allow to evaluate performance with regard to multiple inputs and outputs and facilitate an integrated evaluation (Bogetoft & Otto, 2010). Both SFA and DEA also have specific advantages and disadvantages to be considered when choosing methods. Alternatively, the methods can be combined to overcome the disadvantages, and such combinations have been applied in several studies dealing with benchmarking processes (Oh & Hildreth, 2016).