a magazine on ecological agriculture
a one stop treasure of practical field experiences

Creating a Sustainable Food Future – Indicators of sustainable agriculture: a scoping analysis

Quantifiable indicators of the environmental sustainability of agriculture—by which we mean minimizing the environmental impacts of agriculture—are an important tool for helping move the world toward a sustainable food future. Indicators enable policymakers, farmers, businesses, and civil society to better understand current conditions, identify trends, set targets, monitor progress, and compare performance among regions and countries.

The World Resources Report’s Creating a Sustainable Food Future: Interim Findings describes how the world faces a great balancing act of three needs. It needs to close a 6,500 trillion kilocalorie per year gap between the food available in 2006 and that required in 2050―a 69 percent increase―to adequately feed the planet. It needs agriculture to contribute to economic and social development. And it needs agriculture to reduce its impact on climate, water, and ecosystems.

What indicators are most appropriate for tracking progress and motivating actors toward a sustainable food future? To address this question, the World Resources Institute (WRI) conducted a scoping exercise to identify a preliminary list of candidate indicators at the nexus of agriculture and environment. This working paper summarizes the results of WRI’s ATI (Agricultural Transformation Index) scoping exercise.

Indicators of the environmental sustainability of agriculture

Our first step in developing indicators of the environmental sustainability of agriculture was to review current indicators, indices, and datasets at the nexus of agriculture and the environment. What sources exist? What are their strengths and shortcomings? What can we learn from them?

To answer these questions, we identified, reviewed, and synthesized indicators, indices, and datasets related to the environmental sustainability of agriculture. Through discussions with experts at WRI and elsewhere, and an extensive literature review, we identified more than two dozen sources. We screened each for relevance and eliminated those deemed irrelevant. No indicator perfectly reflects reality; each has limitations.

Parameters for selecting candidate indicators

To identify candidate indicators for the environmental sustainability of agriculture, we pursued a three-step process. First, we identified the most relevant “thematic areas” for indicators. These are the topics at the intersection of the environment and agriculture that we consider most significant—that is, where agriculture is a leading cause of environmental damage. These areas are water, climate change, land conversion, soil health, and pollution.

Second, we identified the types of activity that indicators can seek to influence— what we call the “causal chain.” Third, we identified three generic stages of the “causal chain” of action that indicators can represent or seek to influence. These stages are public policy, farmer practice, and biophysical performance.

Stages in the Causal Chain

Indicators and indices seek to reflect and ultimately influence multiple types of behavior. For agriculture, they can reflect policies, practices, and performance—a sequence of behaviors and results or “causal chain.” More specifically, government policies can influence farmer practices, which in turn can determine on-the-ground biophysical performance or conditions. For example, a regulation (“policy”) that requires a farmer to measure the water she withdraws for crop irrigation can create an incentive for her to implement conservation irrigation techniques (“practice”) which, in turn, can improve water-use efficiency and produce greater crop yield per unit of water used—or “crop per drop” (“performance”).

Ideally, a portfolio of indicators on the environmental sustainability of agriculture should reflect all three parts of the causal chain. Policy indicators reflect the policies that could create the right enabling conditions or incentives for sustainable agriculture. Practice indicators reflect the on-farm practices that help realize sustainable agriculture. Performance indicators reflect the desired, on-the-ground, biophysical state associated with sustainable agriculture. Although performance indicators are the best reflection of what is happening on the ground because they measure biophysical conditions, they are the hardest to mandate and to monitor. Policy indicators, conversely, reflect the existence of policies, some of which may be ineffective or unenforced, but they are arguably easier to monitor than biophysical conditions.

Screening criteria

We selected a suite of screening criteria against which to assess candidate indicators. For each of these activities, we referred to existing indicators, indices, and datasets, as well as WRI expertise. We selected seven screening criteria against which to assess candidate indicators. These screening criteria are availability of data, accuracy of data, consistency in how data is gathered, frequency of data, data’s proximity to reality, relevancy of data, and ability for data to differentiate among countries.

Available: Are the data underlying the indicator currently available for most countries?

Accurate: Are the data underlying the indicator accurate, reliable, and representative of on-the ground conditions?

Consistent: Are the data collection methods consistent and the data comparable across all countries?

Frequent: Are the data regularly collected or updated such that they are relatively current?

Proximate: Is the indicator or its data indicative of the environmental sustainability of agriculture with respect to the theme being considered? In other words, is it a good “proxy” for reality?

Relevant: Is the indicator or its data highly pertinent to policy decisions involving environmental sustainability of agriculture?

Differentiating: Is the indicator or its data specific enough to show distinctions among countries?

To assess how well a candidate indicator meets one of these criteria, we developed a simple three-part scale of “high, medium, low” or “green, yellow, red,” respectively.

Shortlisting indicators

We identified a “long list” of candidate indicators of environmental sustainability in agriculture for each of the five thematic areas and for each of the three stages in the causal chain. Indicators came from our analysis of existing sources, as well as WRI expert input. We then evaluated each of these possible indicators against the seven screening criteria. Those that fared best became the “short list” of candidate indicators.

“Evaluation of Candidate Indicators of Environmental Sustainability of Agriculture” (an Excel workbook available at http://www.wri.org/resources/data-sets/foodindicators) presents the list of possible indicators. Each worksheet is dedicated to a thematic area (e.g., water, climate, soil health) and is organized by step in the causal chain (i.e., policy, practice, performance) on one dimension and by selection criteria (e.g., available, accurate, consistent) on the other. Each possible indicator is evaluated against these criteria, accompanied by comments for clarification.

Some caveats

A few caveats are important. First, given that this working paper summarizes a scoping exercise, the candidate list represents those indicators that we deem most suitable for further research and vetting―particularly with regard to data availability, accuracy, and frequency of collection. Second, we did not restrict selection of candidate indicators to those for which data are already available in all countries. Although some suggested candidate indicators may fare poorly on the data availability criterion, they would be accurate, proximate, relevant, and differentiating. We include them as a signal that the international community should consider generating and collecting data for these indicators. Third, in the Excel workbook, we offer ideas for how to collect missing data. Fourth, the candidate list does not include demand-side aspects such as measuring rates of post-harvest food loss and waste. This is outside the scope of the analysis.

Box 1: The World Resources Report: Creating a Sustainable Food
Future
How can the world adequately feed more than 9 billion people by 2050 in
a manner that advances economic development and reduces pressure on
the environment? This is one of the paramount questions the world faces
over the next four decades.
Answering it requires a “great balancing act” of three needs—each of
which must be simultaneously met. First, the world needs to close the gap
between the food available today and that needed by 2050. Second, the
world needs agriculture to contribute to inclusive economic and social
development. Third, the world needs to reduce agriculture’s impact on the
environment.
The forthcoming 2013–14 World Resources Report, Creating a Sustainable
Food Future, seeks to answer this question by proposing a menu of
solutions that can achieve the great balancing act. Some menu items
address the demand for food, such as reducing food loss and waste and
shifting diets. Other menu items address the supply of food, such as
boosting yields through crop breeding, improving land and water
management, and improving pasture productivity.
Since the 1980s, the World Resources Report has provided decision
makers from government, business, and civil society with analyses and
insights on major issues at the nexus of development and the environment.
For more information about the World Resources Report and to access
previous installments and editions, visit www.worldresourcesreport.org.

Integrating the indicators into an index

Integration involves weighting and aggregating the constituent indicators of the index. The constituents are assigned weights based on statistical criteria or expert judgment. Then they are aggregated in either a linear or nonlinear fashion.

When integrating indicators into an overall index, it is important to keep five points in mind. First, no single integration approach for designing an index is considered statistically or scientifically superior to another: all represent value judgments. Second, the approach selected depends largely on the index’s intended purpose.  Third, avoid using constituent indicators that overlap or cover the same issue; they will “double count” in the aggregate index. Fourth, avoid constituent indicators that are the opposite of each other; they will zero each other out in the aggregate index. Fifth, recognize that an aggregate index may be too broad for some audiences to derive a clear message regarding the meaning and implications of the index. Too much information may be integrated, making the result unclear or even misleading. Therefore, stakeholders considering combining indicators into one index should proceed with caution.

Proposed next steps

Designing indicators or an index for the environmental sustainability of agriculture will require new work. It is not possible to simply adopt or repackage existing material into a sufficiently robust index or set of indicators. Although data exist for some indicators, information gaps hinder designing a suite of indicators and an associated index that sufficiently covers the range of important thematic areas. Closing these gaps will require a collaboration of partners with a variety of expertise, ranging from data gatherers and statisticians to agriculture and sustainability experts.

Entities to engage in this process include those that could provide data for indicators, those that could track the indicators, and those whose actions might be influenced by the indicators. These entities include (but are not limited to) the FAO, the OECD, the CGIAR research centers, national agriculture ministries (for feedback on indicators and their application), national environment ministries, the World Bank, bilateral development agencies, and research organizations. One institution should become the “lead” for developing the indicators (and index).

Concluding thoughts

Quantifiable indicators of the environmental sustainability of agriculture will enable policymakers, farmers, businesses, and civil society to better understand current conditions, identify trends, set targets, monitor progress, and compare performance among regions and countries.

If appropriately designed, they can foster incentives for the sector or nations to improve performance. And they make managing the nexus between agriculture and the environment easier; it is hard to manage that which is not measured. For these reasons, indicators are an important ingredient in achieving a sustainable food future.

This is an abridged version of the original – Reytar, K. et al. 2014. “Indicators of Sustainable Agriculture: A Scoping Analysis.” Working Paper, Installment 6 of Creating a Sustainable Food Future. Washington, DC: World Resources Institute. Available online at http://www.worldresourcesreport.org.