Background Paper Prepared for the
Organisation of Economic Cooperation and Development (OECD)
Workshop on Pesticide Risk Indicators
21-23 April, 1997
Copenhagen, Denmark
(revised May 19, 1997)

Department of Fruit and Vegetable Science
Cornell University
Ithaca, New York, USA

E-mail: LCL3@cornell.edu

Support for this research has been provided by NRICGP/USDA Grant No. 95-37313-1940 and by the Department of Fruit and Vegetable Science, Cornell University, Ithaca, NY USA.

Many groups of individuals and types of institutions--including farmers and other land managers, consumers and consumer groups, food retailers and agribusinesses, regulatory agencies and regulatory "watchdogs"--have a stake in better understanding the non-target impacts of pesticides used in agriculture, landscaping, materials preservation, and elsewhere in modern society. In the past, much of the attention on pesticides focused narrowly on monitoring costs to producers and efficacy in controlling target pests. When non-target impacts were considered, the quantity of pesticides applied was generally used as the only indicator of risk. However, especially as new classes of more potent chemicals have been developed which require far lower dosages than older types of pesticides, it has become increasingly apparent that pesticide weight is not a sufficient proxy for risk. Thus a diverse research community is working to develop methods for more accurately estimating the impacts of pest control products and methods on one or more environmental indicators.

This paper is an overview of pesticide impact assessment systems which compare the characteristics and effects of different pest controls and generate an index or ranking of pest control options, or which compare pesticide risks over time or in different places. These type of assessment tools are sometimes called "pesticide risk indicators." The paper begins by defining and critically examining this EIA methodology. A typology is suggested for distinguishing among three categories of assessment systems: (1) decision aids for farmers/growers and other land managers, (2) research and policy tools for use by governments, industry or academia, and (3)"ecolabeling" systems designed to influence consumer opinion and market behavior. Systems are differentiated by objective, arena of activity, target audience, and by how an economic component figures into the assessment. Methods for constructing and calculating the rankings are described in the fourth section. The fifth section illustrates the procedures and specific uses for a number of farmer decision aids and policy tools. The paper concludes by looking to the future, at trends in the development and demand for pesticide ranking and indexing systems.

• simulation of environmental effects--either by computer modeling or physically in microcosms or mesocosms;
• sampling, monitoring and tracking changes in biophysical indicators (indicators such as species diversity, soil respiration rate, and chemical levels in the environment);
• surveys and qualitative research methods, which include observation, interviewing, and participatory action, with tabulated or descriptive presentation of results; and
• indexing or ranking the extent and severity of pesticide (both chemical and non-chemical pest controls) impacts on one or more environmental indicators.

The last of these approaches--indexing systems--draw on, synthesize and compare information collected by other EIA methodologies and thus become tools for decision-making and policy formation. Many types of environmental variables can be evaluated with the indexing methodology, not only those which can be sampled, monitored or mathematically modeled. Indexing systems can thus make the leap from assessing test endpoints to also assessing decision endpoints. To illustrate the difference using an example from pesticide impacts on honey bees: Measures of pesticide toxicity to bees can provide information about pesticide lethality (LD_{x HB}), or the effective dose at which certain behaviors (such as nectar-collecting activity) change (ED_{NECTAR-COLLECTING}). However, a beekeeper is less interested in these test endpoints than in knowing how a pesticide application will affect hive survival or, perhaps, crop pollination. Therefore, the decision points of interest to beekeepers are how the impact on honey bee colonies might be reduced by using a different pest control, a lower dosage or a different time of application.

Pesticide indexing and ranking systems can help answer such questions by comparing alternatives. These methods can also contribute to policy analysis by monitoring trends in pesticide risk and evaluating the success of risk reduction initiatives. Especially when there are significant economic, safety and environmental ramifications from an assessment, it is vitally important that it be based on meaningful environmental criteria and indicators. Natural questions to then ask are: Which environmental parameters should be included in an assessment as descriptors of the environment? What indicators should be used to register effects on the selected environmental variables? and How should these impacts be measured, scored and interpreted?

The images which come to mind upon hearing the term "environmental impact" are different for each of us, depending upon our vantage and how we value the components of the environment. In some contexts the environment is considered distinct from public health and other social impacts, but in this paper all non-target effects are included under the rubric of environmental impacts. A complete set of enviro-social parameters for assessing consequences of pest control activities includes:

• health of farmworkers, consumers and the general public--including impacts on particularly vulnerable sub-populations;
• lethal and sub-lethal impacts on other non-target biota;
• direct and indirect impacts on natural- and agroecosystems, including effects on habitat and food sources;
• consumption and degradation of natural resources;
• pollution of soil, air and water--which can have consequences for human and non-human biota; and
• costs to society for remediation of environmental problems.

The choice of assessment parameters from this vast array of potential indicators is not trivial, as illustrated by the radically different results from several pesticide ranking systems, shown in Figure 1. The rank order of pesticides depends in part upon the components of the analysis--the pesticides considered, the variables assessed, the choice of specific measurable endpoints as the indicators of impacts on these variables; the mathematical structure of the model, including relative weighting of variables and scoring of results; the method for filling data gaps; and whether usage data are factored into the equation (i.e., a ranking by hazard or by hazard potential/risk). A similar comparison by Pease et al. (1996) found only one insecticide--permethrin--among the ten most hazardous pesticides, as rated by both the Environmental Impact Quotient (EIQ) and by a US National Oceanic and Atmospheric Administration model (Kovach et al. 1992). Results differed because of the factors just listed. More specifically, the NOAA focus is on estuarine species while the EIQ has a terrestrial emphasis. Thus dimethoate and parathion get high hazard ratings from the EIQ because both are acutely toxic to birds, but a relatively low NOAA ranking because that model does not consider impacts to avian species. Also, a number of the fruit and vegetable pesticides surveyed by the EIQ were not ranked by NOAA because they are little used in California, where that model was applied. Conversely, the wood preservative pentachlorophenol and persistent insecticide toxaphene are not included in the EIQ dataset, but receive high hazard ratings from the NOAA model because of pentachlorophenol's high bioconcentration factor (an environmental parameter not included in the EIQ) and toxaphene's toxicity to fish and invertebrates.

Note: These rankings are not weighted by pesticide release, usage, environmental concentration, exposure or typical dosage and thus should not be interpreted as presenting the greatest danger or risk.

The question of whether and how to weight and integrate impacts on different organisms and environmental components is also not trivial. The answer can determine how policy-makers and the public come to view different products. The lists of "most dangerous" compounds--which are the succinct results of pesticide impact assessments--are remembered long after the basis for drawing the conclusions is forgotten. Despite these issues regarding the application and interpretation of ranking systems, many constituencies are demanding some type of handle for comparing the environmental impacts of different pest control products and methods. Thus researchers from a range of disciplines are attempting to construct meaningful and reliable evaluative tools.

In part the difference in terminology simply depends on what one is accustomed to using, but it may also reflect a difference in disciplinary orientation and methodological approach. The products generated may also be somewhat different, and put to different uses. Systems ecologists and other systems analysts think in terms of describing and quantifying energy and resource flows in complex systems, with a primary objective of identifying causes and effects of change in systems, and a penchant for understanding interactive effects. Pesticide impact assessments drawing from this disciplinary heritage--particularly those used as farmer decision tools--often retain a systems perspective and resultant complexity, as well as the related terminology. On the other hand, policy makers looking for a more terse analysis may have a predilection for "indicators" of trend highlights, which do not pretend or attempt to holistically model systems.

In this paper the term "indicator" generally refers to the component parts of assessment systems rather than to the totality of the assessment. These indicators are the measurable endpoints that provide information about an effect or impact on the environment. For example, the half life of a pesticide in the soil is an indicator of how long the pesticide persists in the environment. An indicator of pesticide risk to birds may be a function of the pesticide’s acute toxicity to birds, its foliar persistence and its impact on food sources for birds (e.g.: on insects, worms, seeds). I.e.: Risk Indicator_BIRDS = f(Acute Toxicity, Foliar Persistence, Food Source). Risk to birds may be calculated from the LD_{50 BIRDS}, Half Life _LEAVES and LD_{50 WORMS} which are, respectively, the measureable endpoints that indicate the impact on these toxicity and exposure variables. Yet another example of an indicator is the Groundwater Ubiquity Score, calculated as GUS = log 10(t1/2) x [4-log10 (K_OC)] (Gustafson 1989). This algebraic model is used by itself as an indicator of pesticide leaching potential and is also used in combination with other indicators of different environmental impacts to create multiattribute impact assessments. Composite or integrated assessment systems are generally composed of a set of such risk or impact indicators.

Another semantic distinction is that in this paper the term impacts is generally used, rather than either hazards or risk, to describe consequences of pest control activities. The rationale is that the word hazard is supposed to convey a type of harm, e.g.: "lethal hazard to amphibians," while risk is supposed to convey a probability of harm. By incorporating pesticide usage and exposure indicators, indexing systems typically convey more information than merely a listing of hazards, but they are generally constrained by their structure and available input data to be less than a probablistic assessment of risk. The use of the term impacts is intended to capture this approximation of "hazard potential" and risk characterization.

One of the primary objectives for developing methods for assessing environmental impacts of agriculture is to be able to respond to questions such as this from farmers and other land managers. Decision makers in the field need information and methods for choosing pest control practices which have the least negative impacts on the environment, and on human health and safety. Policy makers then need to make broad brush appraisals of the impacts of such choices.

More broadly, the impetus to develop and use pesticide impact ranking systems is to facilitate the shift to more benign pest control products and practices, and thus reduce pesticide load on the environment. It is an uphill battle: Total worldwide pesticide use is at a record high--4.7 billion pounds in 1995--and continues to grow. In the US, pesticide use reached an all time high of more than 1.2 billion pounds in 1995. Despite the fact that many newer pesticides are used at very low volume per unit area, this is more than double the amount applied 35 years ago when Rachel Carson wrote Silent Spring (1962). The biggest users remain in North America (29% of sales in 1996) and Western Europe (26% of sales), followed by all of Asia with 25% (PANUPS 1997). However, the largest increase in sales is in Latin America, accounting for one third of global sales growth and more than 10% of use. International cooperation in pesticide impacts assessment is therefore essential, because some of the most hazardous chemicals that have been banned from some countries continue to be used elsewhere, often in tropical climates where risks may be greater due to social as well as biophysical factors such as higher temperatures and greater skin hydration (PANUPS 1995).

Simply projecting risk from pesticide usage data is not sufficient, and is even less meaningful now that highly potent chemicals are used at very low volumes. Risk-weighted indexes of usage are a more powerful measure and avoid the shell game of shifting risk from less potent chemicals used at higher volumes to more potent chemicals used at lower volumes. However, the analytic task of pesticide impacts assessment has become more complex as the number of chemical classes of pesticides has quintupled from approximately 25 in the 1970s to about 130 today, and the modes of pesticide activity affecting the environment have also diversified (Wauchope et al. 1994).

About 1% of the 8 million chemicals listed in the Chemical Abstract Services Registry are commercially produced. Although fewer than 1% of these are used for pest control (Davis 1994), the compilation and interpretation of hazard data for 500-1000 pesticidal chemicals (the higher number reflects newly identified microbial and biochemical pesticides that have been added to the synthetic chemical arsenal of pesticides) is a daunting task. It is, however, a necessary prerequisite for development of reliable and meaningful comparative and summary assessments of pesticide impacts on the environment. Data limitations and complexities are just one of the challenges in developing pesticide environmental impact ranking systems. Other issues are the identification and integration of suitable environmental indicators; estimation of situation-specific variabilities; the lack of a common currency for comparing disparate types of impacts; and the bias against considering future impacts.

Choosing Environmental Indicators and Deciding How to Integrate Them

No one species or group of biota reacts most sensitively to all pesticides, and thus is useful as a surrogate for all others in toxicity testing. With other environmental perturbations as well, we cannot rely on a single indicator species or abiotic effect to tell all we need to know about impacts of management decisions. Thus there is a tension in the development of assessment systems between the advantages of ranking pesticides based upon impacts in one environmental dimension versus advantages of composite assessments of impacts in multiple environmental dimensions. The drawback to the former is that no single parameter can fully describe an environmental impact, and thus conclusions can be misleading unless system objectives and limitations are made explicit. On the other hand, the challenge of multi-parameter optimizations are in specifying test endpoints of significance and then integrating results--either into a composite picture of environmental impacts or by prioritizing the most critical impacts in a given situation. To grasp the conceptual challenge this poses, think about how you would weigh impacts on human beings in relation to impacts on other biota, especially if they were dissimilar in magnitude and type.

There is also no one set of social or environmental indicators that is most appropriate to use in assessing impacts under all sets of circumstances. Even if a system developer were to decide upon a protocol for weighting, valuing and integrating impacts in the context of one assessment scenario, these issues would re-emerge in a different configuration when the question again arises on a different scale, for a different target audience, or with different situation-specific conditions. To illustrate: the types of data required for a farmer to choose a least impact but efficacious pest control may not be the same as the data required for a national assessment of agriculture practices. IPM farmers want to avoid pesticides that harm parasites and predators of the pests in their fields, but these producers might be misled by a decision model based on national data for pesticide impacts on beneficial organisms. The only such data included in the US EPA Ecological Effects dataset, for example, are acute toxicity of pesticides to worker honey bees (Atkins et al.; US EPA 1996). Even were toxicity dose responses comparable for honey bees and other beneficials, the significance of effects on these groups of organisms is likely to be quite different. When honey bees are repelled from a field by pyrethroid pesticides, for example, they survive and move on to another nectar source. However, if beneficial parasites and predators are repelled from a field, they are then not available to work in that field as biological control agents. As shown by this example, the honey bee acute toxicity data may not be sufficent or appropriate to use in a micro scale assessment for advising farmers about pesticide effects in their agroecosystems, but may be valuable as a proxy for other ecological effects in a macro scale analysis of pesticide risk trends. The design of an assessment system must, therefore, be appropriate to the objectives of the audience served.

Some assessment systems include variables that do not really assess environmental impact. These include the variable "availability of alternatives" which could bias comparative assessments of pesticides toward dangerous pesticides with no conventionally-known alternatives. Production cost factors are also sometimes included in assessment equations alongside environmental factors.

Dealing with Situation-Specific Variability

Environmental impacts result from interactions among inherent pesticide properties and situation-specific factors, including particulars of the site, weather conditions, the diurnal and seasonal timing of application, and the application method (packaging, equipment, location, and degree of care). Interactive effects are highly variable, so that it is difficult to make reliable, application-specific predictions of ecological effects. The challenges of incorporating this variability have been addressed in various ways, depending in part on the decision-making context and designated user group. The complexity of indexing systems is generally a function of methods used to estimate exposure: the simplest systems generally consider only toxicity (and to one group of organisms) or use toxic release or production figures as a surrogate for exposure. More complex systems also evaluate one or more exposure pathways (Davis 1994).

Finding a Common Currency

Data Limitations

Data are required at all stages of environmental assessment of agriculture. Because many environmental impacts are generated on different temporal and spatial scales than they are experienced, the complete set of data for assessing these impacts cannot be collected on-farm. This important factor distinguishes environmental assessments from farm production cost assessments.

However, toxicological and ecological effects datasets of pesticides are incomplete. Of the 150 commonly used pesticides evaluated by Pease et al. 1996, data for invertebrates were available for 44 pesticides and persistence (field half-life), for 106 pesticides. In addition, some of the existing ecological effects data are inappropriate to use for assessing relative impacts because standardized testing protocols were not used and so the data are not comparable (Levitan et al. 1995). Moreover there are very limited data and no standardized datasets on impacts of new biopesticides, such as microbial and fungal pesticides. These gaps and inconsistencies in data have been less of an impediment for pesticide screening systems and case-by-case evaluations (as is done for pesticide registration), but complete and reliable datasets are critical for comparing relative impacts or creating a rank order of pesticides.

Most available data on pesticide environmental impacts result from single-species toxicity tests mandated for pesticide registration. Ecotoxicologists have questioned the predictive value of such tests, noting that interactive effects of pesticide inputs at the community and ecosystem levels can be different than inferred from single-species effects, and that the higher level impacts can be of greater long-term environmental significance (Cairns 1986, 1991; Karr 1992). Unfortunately, no standardized datasets exist for pesticide impacts at these higher levels of ecological organization---and data availability strongly influences which parameters and types of impacts can be included in a comparative assessment system.

In addition to limitations associated with testing single species of organisms, most ecotoxicological data are also limited because the test pesticides are generally applied in single doses of individual active ingredients, whereas biota in the environment are constantly exposed to chemical mixtures that change spatially and over time (Yang 1994). Acute impacts may result from exposure to pesticide mixtures, but long-term impacts may also result from mixtures of pesticides with other chemicals in the environment. Also datasets for impacts at standard pesticide dosages may not suffice for assessing risk from aberrantly high doses, such as those following a spill or toxic release, or at the low dosages that are only recently becoming sufficiently suspect to cause concern. Existing datasets also may not be sensitive to impacts on particularly susceptible populations--the unborn, the very young and rapidly growing, pregnant females, people on medications, and biota (human and other) whose immune systems are compromised. Evidence is accumulating which show that mixes of certain chemicals at individually-low doses can have impacts that may be magnitudes of order greater than simple additive individual low-dosage effects (McLachlan and Arnold 1996). Cumulative impacts from repeated or extended exposures can also be different than impacts from single, larger exposures. Bioconcentration of fat soluble compounds has long been known to be a cause of cumulative impacts. However, cumulative effects also result from weakened immune systems and from allergic and other auto-immune responses triggered by chemical sensitivity. These cumulative effects involve short-lived and water soluble compounds as well as persistent culprits. Little is known about such cumulative and interactive effects, especially at low or fluctuating levels and in mixtures, and particularly in terrestrial systems. Yang (1994) concludes that:

• the toxicology of long-term, low level exposures to chemical mixtures produces subtle effects, unlike acute toxic responses to higher doses;
• such toxic interactions are possible at environmentally-realistic levels;
• the toxic responses may be from unconventional endpoints that are not usually tested;
• there is a possibility that residual effects may become interactive with later exposures; and
• these exposures may pose a safety risk to the public.

While these comments are intended to apply to human subjects, these principles and concerns can probably be extrapolated to non-human biota, some populations of which may be more vulnerable to such risks because of limited mobility and physiological factors. We should have to assume (and indeed anecdotal evidence is being compiled which supports this point) that not only do we learn about possible impacts on human beings from mammalian studies, but also that the sublethal disorders noted in human populations may well have parallel effects on other non-target organisms.

Many ecologists have suggested that environmental assessments consider indicators of impacts on organisms in all trophic positions and for all key roles in basic ecological processes. This would include the role of invertebrates and microorganisms in decomposition and humus formation, and the role of herbivores in interactions with vegetation (De Snoo et al. 1994). The problem with this idealized vision is of course that there is no dataset covering this array of ecological indicators. Moreover, for many of these indicators there are no protocols for collecting data and/or no accepted correlations between the direction and magnitude of presumed cause and effect. This results in an interesting tension between the knowledge among biophysical scientists of what would ideally constitute an array of test endpoints for a holistic assessment of environmental impacts, and the paucity of data to make such evaluations possible.

Bias Against Future as Compared to Present Impacts

Long-term and cumulative impacts are more difficult to comprehend and quantify than short-term impacts. As a result, less data are available and less weight tends to be given to these impacts in environmental assessments. We also tend not to consider impacts associated with future events, such as the future leaking of improperly stored pesticides, changes in soil pH leading to release of previously bound chemicals, narrowing of the gene pool, or increased pest resistance to pesticides (Riha et al. 1997). In general, there is a bias against variables for which data are difficult to collect or for which results are uncertain.

The remainder of this paper takes a pragmatic look at how assessments of pesticide impact can be structured to meet specific objectives, within the limits of current knowledge and technical information.

• decision aids used by farmers, farm advisors (e.g. Extension Agents) and other land managers in choosing among pest control options and evaluating the impacts of their choices
• research and policy tools generated and used in government, industry and academia
• "ecolabeling" systems (also called "green labels") designed to influence consumer opinion and market behavior.

Not only are there multiple societal values involved in estimating potential hazards of pesticide use, but also methods, scale of analysis, and results differ depending upon target audiences and intended uses of the indicator. Some assessment tools wear more than one hat, but most are not equally suited for multiple roles. This section and the one which follows lay out some of the principal and idealized characteristics and distinctions among the types of assessments on the basis of the following factors:

• objectives
• decision maker
• arena of activity
• unit of analysis
• economic dimension
• factors considered
• format of results
• method of calculation

Another handle or typology for discriminating among assessment tools is by whether systems are based on impact or behavioral criteria and indicators. Indicators of impacts include many pesticide test endpoints, such as single species toxicity test results (e.g.: the LD₅₀), estimates of exposure, measures of residues on food and in the environment, sublethal effects (e.g.: impacts on behavior, reproduction, genetic diversity), secondary impacts on habitat and food sources, and impacts at higher levels of ecological organization (e.g.: impacts on species richness, biomass, ecosystem productivity). As were detailed earlier, assessments based upon impact criteria have been constrained by data limitations and inconsistencies.

In contrast to indicators of impacts, examples of behavioral indicators include: Was X crop produced by organic methods? Does this farm practice Integrated Pest Management (IPM)? Which/ How many/ What types of IPM techniques? Were biochemical or synthetic chemical pesticides used to control target pests? Assessments based on this type of indicator are not similarly constrained by gaps in the data because data points are observational. There are really two types of indicators of behavior: those based on current behaviors and those based on future or promised behaviors. Examples of the latter include the promise that fewer pesticides will be used by producers in coming years, or promises of better health safeguards for farmworkers. Assessments based on promised behaviors perform some very useful functions: They are used as ecolabeling criteria where the objective of the system is to use incentives to develop compacts between producers and accreditors in order to shift behaviors. They may also be useful in developing policy tools which project future risk trends.

Systems based on behavioral indicators are a creative means for circumventing some of the difficulties of multiattribute optimization, and the problem of gaps in impact data. They do so by assuming a positive relationship between certain sets of behaviors and impacts. More specifically, behavioral assessments are generally predicated on the assumption that IPM and organic methods are more benign in their environmental impacts than post World War II "conventional" chemical pest control strategies. This is why people "buy organic" and why the IPM research community is interested in developing IPM accreditation standards and IPM ecolabels. However, assessments based on impacts data are essential for undergirding, supporting and challenging these implicit assumptions about the impacts of certain behaviors.

In the same way that the term "environmental impacts" is used in this paper as a shorthand for the broad array of impacts that are sometimes separately listed as public health, ecological, economic or social, the phrase "farmer decision tools" is used here as a shorthand to refer to all site-based, field-scale decision aids intended to be used by property managers. This group does include farmers, but also includes homeowners and others (e.g.: lawn maintenance companies, golf course managers, foresters, etc.) who must deal with pests of buildings and grounds. In other words this typology and these assessment tools, are not limited to agricultural applications.

The objective of this type of assessment tool is to inform people who make pest management choices about potential environmental consequences of their decisions. Farmer decision models are generally structured as a comparison of pest control options, based upon some subset of key enviro-social indicators. As decision aids, the models are most reliable when tailored to meet situation-specific conditions for a given field or farm. However, the manipulation of situation-specific data (e.g.: soil type, local climate, application timing and equipment) can be cumbersome and overwhelming, so that succinct and user-friendly measurement tools may find more widespread use. Some of the seemingly more simple models make use of variables which impute situation-specific characteristics, rather than actually measuring and utilizing situation-specific data.

Because of the complexity of determining which pest control management practice is "best" under a particular set of situation-specific conditions, the development of reliable decision aids for farmers presents a formidable challenge. The results (recommendations) of most such assessments should be linked with the decision making process because the ranking of pest control options may differ depending upon the situation-specific conditions. Thus farm-scale decision tools may best be presented in a workbook format or as a computerized expert system. These formats, unlike standalone rankings of pesticides by risk, permit "if-then-else" routines that can be responsive to situation-specific variability.

The typical unit of analysis in producer decision tools is either a discrete pest control application or practice, or else the compendium of decisions made during one or more production seasons. Together these decisions constitute a pest control strategy. Results are often expressed as a listing of unitless impact points for an array of pest control products. Results may also be expressed as a similar impact index of pest controls, but specifically as these are used to combat a given target pest (see figures accompanying the PestDecide© farmer decision tool for an example). The reason for this level of specificity is that pesticides are applied at different rates and times, with different expected efficacy, to combat different target pests. Non-target impacts may differ accordingly.

Farmer decision tools typically identify pest controls by trade product names--rather than only by the common name of the pesticide active ingredient--because efficacy and non-target impacts also differ with formulation. Unfortunately, input data are rarely specific to formulated products and, therefore, the impacts of formulations are at best adjusted by the concentration of active ingredient and the type of product (e.g.: granular, liquid, vapor). However, impact data do not generally reflect the actual characteristics of the trade product, as affected by the mixture of adjuvants and other inert ingredients in the formulation. The full season time horizon that is considered by a number of farm-scale assessments provides a broader and more realistic picture of environmental impacts than a snapshot evaluation of a single pest control application. Some examples: The application of a lower impact (and thus seemingly preferable) pest control at one point in the season may necessitate additional subsequent applications of various pest controls later in the season, causing a larger cumulative impact. Repeated applications of certain lower impact pesticides may lead to more rapid build-up of pest resistance, and thus ultimately to a renewed reliance on more hazardous pesticides. Or the buildup of weed seeds following a period without herbicide use could necessitate more herbicide use later in the season or in subsequent seasons. These interactive and cumulative factors would be missed by an assessment limited to the impacts from a single pest control application. Some of the full season assessments are crafted to do double duty as ecolabels, with accreditation offered to the yields of production scenarios that meet threshold criteria (or which receive a certain number of impact points).

The value of assessment tools for farm managers depends on their ability to provide valid information pertinent to specific pest management decisions. The problematic of these decision aids is to integrate the many factors contributing to each management decision--a farmer’s personal concerns for applicator safety and the agroenvironment, altruistic or regulated interests in protecting human health and the broader environment, business interests in production costs and in securing a market niche. An assessment model would be hard pressed to deal with all of these factors, but it is not unusual for farmer decision tools to include production cost estimates for each pest management choice or scenario. In contrast, assessments designed as policy tools may instead weigh economic costs to society at large, whereas ecolabels tend to leave the economic assessment to the consumer (or intermediaries) in the marketplace.

In this typology a wide range of assessment systems used for many distinct purposes fall under the aegis of policy tools. The types and objectives of policy tools include:

• screening systems to enable regulatory agencies and industry to quickly assess preliminary data for danger signals about new or potential pest controls
• analyses by government agencies and environmental watchdog groups, either to monitor trends in pesticide use and risks over time, or to compare risks of using different pesticides and pest management regimes
• evaluations of potential risks from individual pest control products and practices, to use as basis for regulations, restrictions and warnings. (For examples, see the WHO and US EPA human health toxicity warning systems designed for pesticide labels.)
• criteria for programs to tax pesticide use
• evaluations of the success and/ or costs and benefits of programs and policies. Also, evaluations of which risk reduction policies and research trajectories have the greatest impact or provide greatest value for their cost. (e.g.: Is it more effective to tax pesticide use or to regulate spray buffers around fragile areas or to invest in better application technology?)
• assess adoption of IPM on farms and set IPM research and extension priorities. Evaluate the success of IPM in meeting the objectives of controlling pests in a cost effective and more environmentally benign manner. Determine if IPM makes a difference. To whom? At what cost?

The types of decision-makers targeted by policy tools are as different as this list of objectives. They include IPM research and extension teams and their funders working in academia, extension, and budget offices of government and foundations, as well as policy-makers at all levels of government and in the environmental research and activist communities. The arena of activity for policy tools could be said to be "at the roundtable," in contrast to the farmer’s field or the marketplace. The unit of analysis for assessing the success of IPM programs may be a single production unit (i.e.: a farm) or the set of farms in a region which produce similar commodities. For other policy tools, the unit of analysis may be the total quantities of a pest control product used in a state, a nation, or internationally. Thus the situation-specific variability and details of importance in making pest management decisions at the field scale often wash out at this larger scale. Because results of the analysis will not differ, for the most part, on the basis of situation-specific responses from producers, risk indexes used for policy purposes may be published separately from the decision-making process, as tables in a scientific paper or as a pesticide label.

Another distinction is that most national datasets lump all forms of a pesticide active ingredient and all the formulated products in which it is found, whereas the ideal farm-scale decision tool will be sensitive to the different toxicities of formulated products (due to adjuvants and different formulation types) and to factors which mitigate exposure to non-target organisms (including packaging and application equipment). A major difference between the objectives of farm-scale decision tools and most assessments for policy purposes is that the former are generally constructed as comparisons among available options for control of specific pests--since a choice must be made from among the finite group of alternatives. Risk assessments for policy purposes are rarely constrained in the same way. Thus in a ranked list of hazardous pesticides all of the options for a specific task may be among the most or least hazardous.

Although a number of assessment tools have been created for the policy and research arenas, systems which holistically measure and express global (or other large scale) pesticide risk and trends in risk reduction are still at basic stages of development. Procedurally, most such ranking systems weight toxicity factors by pesticide usage figures to create a set of most hazardous, highly used pesticides. For the most part, risk to just one or a very few enviro-social indicators is assessed (most usually these are acute and/or chronic lethality to human beings, and water pollution). Data which would enable a more comprehensive assessment of non-target impacts are uneven and unavailable for many other important test endpoints, and for many pest control chemicals and alternative products. Were data available, methods for integrating and weighting multiple criteria are still imperfectly developed for these purposes, as they are also for farm-scale decision aids. Beyond these challenges, additional questions remain regarding development of pesticide risk reduction yardsticks at a national or international scale:

• Can/should broad scale measurement tools be constructed as extrapolations from (in essence a summation of) farm-scale decision aids? Is it possible to sum from different situation-specific realities?
• How should geographically-different factors be accounted--including both regional climate and soil differences, and behavioral and institutional differences in such things as protections afforded pesticide applicators?
• How should pesticide risk policy tools deal with variability in non-target exposure? At what point in the assessment should exposure factors (including both fate and usage rate variables) be considered?
• Is it adequate or even preferable for policy purposes to develop pesticide risk measures based solely on (relatively) intrinsic physico-chemical and toxic properties?
• Is there a mandate to produce a composite hazard index (grouped by category or ranked by score) that can be used separately from the process of generating the results? Can such a product be a useful policy guide or is too much information lost and too many assumptions made for it to be reliable? Is a "checklist" or an algebraic equation the preferred approach for a risk yardstick of this type?
• Should comparisons of impact and risk consider only the set of direct impacts from pest controls (i.e. toxicity of pesticides) or should assessments also include a broader array of environmental consequences resulting from other methods of pest control (i.e. the consumption and degradation of fossil fuels, water and other resources)?
• How should an economic dimension be incorporated into pesticide policy tools?

To address this last point, there is an opportunity--if not an obligation--to consider economic cost in a broader social context in policy tools than they are typically considered in farm-scale decision tools. For the latter, system developers have an obligation of sorts to consider production costs borne by the farmer, as well as the environmental and public health effects on society. However, economic costs also manifest as medical expenses, diminution of the quality of life, remediation and restoration costs, etc. Thus even when these costs are not explicitly considered by an assessment model, analysts can tacitly recognize these multiple dimensions of economic cost by labeling "production costs" as such, rather than giving them the title "economic costs" which allows the narrower meaning to "own" the more encompassing term.

It is also important to consider how economic costs are incorporated into environmental impact assessments. Several of the most widely used farm decision tools include production costs among the variables determining the pesticide ranking. If the ranking is interpreted as an environmental assessment, however, then it incorrectly implies that the magnitude and severity of environmental impacts varies with the producer’s cost for pesticide use. Since this is not the case, it is important to assess such costs separately from environmental impacts. One such method is shown in Figure 2. When the economic costs of environmental protection are high (i.e., if farm costs for producing without pesticides are burdensome), there should be a policy debate about whether and how to shift that economic burden from the farmer (or the consumer) to the broader society. However, these costs should not be allowed to unduly influence the assessment of environmental impact and potential risk.

Reduced pesticide risk is only one of a number of environmentally-sensitive criteria reflected by ecolabels. The "organic" or "bio" ecolabel is probably the most common label based upon pest control criteria, but many other labels which reflect environmentally-sensitive criteria--such as recycling, local production, no rainforest destruction or cruelty to animals--have become familiar in the marketplace. Many of the emerging ecolabeling programs are built on incentives based on promises of behavioral change in the future: For some specified duration of time, products or companies may be granted a third party ecolabel on the basis of having agreed to change and improve on past environmental performance. Promises may include some percentage reduction in pesticide use, an effort to use less of potentially dangerous pesticides, or a willingness to better protect worker safety.

An idealized objective of a number of programs is that their ecolabels reflect a product "life cycle assessment" of all enviro-social impacts from "cradle to grave"--or from the extraction of basic inputs to the eventual disposal of waste goods. More realistically, however, most ecolabels reflect a subset of enviro-social criteria of importance to those affixing the label. For example, some of the criteria for ecolabels on manufactured goods are that recycled materials or energy-conserving processes are used in production; that the product is less caustic than alternatives; that inputs derived from animals are not used in production; that products are not tested on animals; or that old-growth forest and rainforest resources are not destroyed in the production process. Criteria underlying ecolabels on agricultural products may include:

• "organic" or "IPM" standards that exclude or limit use of hazardous pesticides, or which shift acceptable pest control towards preventative practices;
• soil management to reduce erosion and the runoff and leaching of agrochemicals;
• local origin of farm produce--either as a measure of support of local rural communities and open space, and/or as an indicator of less fossil fuel use in transport;
• guarantee of safe pesticide disposal methods or farmworker safety standards.

Some products carry the stamp or label of more than one certification program, each with criteria in a different enviro-social domain. Some ecolabels incorporate social criteria and human rights standards, such as fair trade practices and adequate recompense for labor. The emergence of ecolabeling as a factor in the market is causing concern in some circles about its potential as a barrier to free trade (Dawkins).

Common characteristics of most ecolabeling programs are that:

• the unit of analysis is the consumer product as found in the marketplace, rather than particular production inputs, or processes. In practice, however, entire production facilities (e.g.: farms), product lines (e.g.: name brands), or commodities are given the ecolabel accreditation.
• the arena of activity is the marketplace, rather than the farm field or the regulator’s roundtable.
• the targeted decision-maker is the consumer, rather than the farmer or regulator.

Although the consumer is the decision maker who decides whether to buy the labeled product, the defining characteristic of ecolabeling is that label standards are set before the product reaches the marketplace by an accrediting body that decides whether to confer the "green" designation. Consumers are able to weigh the information conveyed by the ecolabel logo as they also balance their need for the product against its cost. However, the consumer has neither the onus nor the opportunity to independently assess the factors going into the designation. They cannot modify the decision about whether or not a product is labeled. Consumer generally remain unaware of the decision criteria behind the visible logo. The results of the assessment are thus conveyed via a product logo or label that is always viewed separately from the assessment process.

The accrediting body may be a government or international agency; a third party non-governmental organization (NGO) or private firm; or a self-regulating group of producers and/or product distributors. Increasingly, coalitions of different interest groups are developing labeling criteria cooperatively. This broader base of support imbues projects with internal "checks and balances" and thus increases their exposure and legitimacy in the public eye. Some labels rest on the transformation of government regulations (or international treaty regulations) into a marketing tool. The US "organic" designation will eventually be backed by USDA National Organic Standards, for example; and the "dolphin-safe" label on tuna fish reflects the standards of the Inter-American Tropical Tuna Commission (Lefferts and Heinicke 1996). Government agencies and NGOs support ecolabeling because it is seen as voluntary, rather than regulatory, and thus a less heavy-handed mechanism for inducing a shift in behavior.

A mix of personal benefits and altruism motivate consumers to buy labeled products. While some ecolabels are similar to health and safety labels in flagging products that may cause personal harm or directly improve consumer well-being, most ecolabels only promise benefits for the environment and the greater good. For example, consumers do not benefit directly by using recycled paper products in the same way that they believe they derive health benefits by heeding warnings from the Surgeon General or by eating food without pesticide residues. In fact, retail- and industry-sponsored food-labeling programs are often adamant in claiming that advantages of the ecolabel accrue only to the environment, and that ecolabeled food is not safer for consumers or superior to the other food products they sell.

Ancillary to the issue of price premiums is the question of who should pay for them. While it may be fair for consumers to pay a premium if their product choice is motivated by the chance of gaining an extra margin of personal safety, is it also fair for environmentally-conscious and conscientious consumers (and risk-taking altruistic producers) to shoulder the additional cost when their buying habits are motivated by broader enviro-social concerns? Conversely, is it equitable for the global society to be burdened by the costs of environmental remediation necessitated by more hazardous production practices? Ecolabeling focuses attention on the potential of market mechanisms to shift these costs to appropriately targeted producers and consumers. It also calls attention to the interface of market mechanisms with policy initiatives--such as the application of surcharges for purchasing environmentally dangerous or degrading products--and raises questions about which approach is more efficient, effective and tenable. Also at issue with ecolabel programs (albeit perhaps more of a concern with manufactured goods than most farm products) is that there may be significant costs of contracting with the accrediting group. If these costs are borne by producers, then the benefits of the program may be biased towards larger firms which can afford the accreditation procedure. Government and not-for-profit labeling schemes can perhaps circumvent this bias by waiving costs for small-scale producers.

• a criteria "checklist"
• an algebraic equation

Other measurement systems have potential application in policy as well as ecolabeling arenas. Methods for assessing adoption of IPM, for example, can be adapted to the farm scale for accrediting IPM producers or to the national scale for estimating trends in IPM adoption (Vandeman et al. 1994; Benbrook et al. 1996). Likewise the Environmental Working Group’s study of fruits and vegetables most likely to have pesticide residues has been published in the popular media as a type of "commodity ecolabel," but is also having reverberations in regulatory and policy-setting arenas (Wiles et al. 1995).

This is the "how to" section. Methods are described for calculating results for the various types of pesticide impact assessments discussed in the previous section. "Results" may be in the format of continuous, numerical scores or may be categorical groupings (such as high, moderate, low or no impact) which describe the extent of impact, hazard or risk. In some assessments, the categories are translated into the colors at a stop light: "red" indicating a high impact or risk; "yellow," a moderate impact and the need for caution; and "green" for "go ahead"--indicating there is little or no impact from the practice. Some systems assign scores to these categories, and the scores serve as the "common currency" that is weighted and summed to create a composite assessment rating for a pest control practice. Numerical scores can thus be derived from categories of impact, may be derived directly from toxicity tests (such as an LD₅₀ value), or may be a ratio of environmental concentration to an effective concentration that causes a measurable impact. Whatever the format of the results--whether numerical or categorical--they are generally arrived at via one of three methods:

• a logical chain of decision rules, i.e.: a decision tree
• an algebraic equation
• a "checklist"

Because indexing is still an imperfectly-developed methodology for fully describing the enviro-social impacts of pesticides, and also because the demand for tools to measure pesticide impacts is coming fast and furiously from numerous constituencies, the field is dynamic--new techniques are emerging, being refined and synthesized. Thus this "how to" primer does not cover all possible permutations and combinations of methods. Its broad brush is intended to help readers detect and dissect the inner workings of assessment tools that use an indexing approach. Hopefully this will enable readers to better evaluate which enviro-social factors are considered--and how--by any particular assessment system. By reading this more abstract discussion of methods in conjunction with the descriptions of specific assessment tools presented later, both may be more comprehensible. This section also continues in the trajectory of the previous section by highlighting some of the technical challenges and limitations of this methodology, particularly with regard to scoring, weighting and integrating multiattribute criteria. An awareness of these "sticky points" is key to improving upon hazard indexing tools.

One method for deriving a composite environmental impact index rating for pesticides is to use a set of decision rules. This approach is somewhat akin to following a flow chart or using a dichotomous identification key. The logical chain of decision rules creates a knowledge-based system that can solve complex problems. It is a series of algorithms that can be written on paper as a decision tree, or is amenable to conversion to an interactive, computer based "expert" system. In either case, knowledge is represented in a series of if-then-else rules (Plant and Stone 1991). Answers to a first tier of questions direct the decision-maker to a next tier of questions, which will not be the same for all respondents. Or the response to an algorithm generates a score, while a different response generates a different score (Figure 3).

Advantages to this approach are that it can utilize qualitative as well as quantitative information from different sources. Unlike algebraic models, decision tree models do not necessarily depend upon a complete array of comparable data for each pest control evaluated. With decision rules, detailed input data may be required only when deemed necessary by responses to previous questions. Decision trees can therefore utilize descriptive and anecdotal accounts of impacts. For example, there is mushrooming interest, but limited information, about potential endocrine-disrupting impacts of pest controls. Certainly no rigorously compiled datasets exist which contain results from controlled tests which all follow the same protocols for all pest controls. Because of the many possible modes of action of endocrine disruption, and the likelihood that effects are synergistic at low levels of long term exposure, such protocols may be difficult to develop and such tests may be impossible to conduct. Therefore, it is not facile to include this important environmental variable into an algebraic model, but neither is it intellectually or perhaps morally honest to ignore these sublethal impacts in deriving new assessments of pesticide environmental impacts. Existing information could be incorporated into a decision tree assessment model via a decision rule of the format: "If evidence exists which suggests that x pest control may have endocrine disrupting potential, then this pest control is to be classified as a potential endocrine disrupter and treated as..." Pest controls in the suspect category could then be assigned a certain negative hazard score. The amount and type of supporting evidence required could be specified, but need not all be from the same source or in the same form (the O’Bryan and Ross 1988 screening model includes criteria of this type).

Decision trees, used alone or in conjunction with algebraic equations, can also flexibly treat situation-specific conditions, such as those which affect exposure potential. A person knowledgeable about field conditions (e.g.: the farmer, the local farm advisor or agricultural researcher) can input situation-specific data, so that the system uses appropriate decision rules for the situation. This type of interactive assessment system can generate better estimates for exposure and other situation-specific variables than can a non-interactive system. Again drawing on honey bees for an example: The impact of a pest control on honey bees is a function of toxicity of the pesticide to the bee and the likelihood of exposure. Honey bees fly only in daylight, however, so that exposure to pesticides is minimal if application is at night. An additional complicating factor is that some very toxic pesticides repel bees before killing them, so the bees are not effectively exposed. Conversely, some less acutely toxic pesticides do not kill the adult worker bees and nor are they repellent, so the chemicals are carried from the field back to the hives where they are stored and fed to the brood. These pesticides can, therefore, affect hive survival and the honey bee population more profoundly than would be expected from a rank order of pesticides determined solely from acute dermal toxicity to adult bees. It is well nigh impossible to capture the complexity of these ecotoxicological effects within the constraints of an algebraic model.

A logical chain of decision rules can segment these interactive effects into a set of ‘if-then-else’ algorithms which assign appropriate value to the x_{HONEY BEE EXPOSURE} variable. These decision rule-generated values could be linked to an algebraic model by a workbook exercise or simple computerized routine. The flexibility of the decision tree approach makes it well-suited for farm-scale decision tools and other impact assessments which should be sensitive to situation-specific conditions. However, this format creates an unbreakable link between the process of decision-making and the results of the assessment, making it somewhat cumbersome and perhaps unsuitable for the policy arena, where a terse summary of impacts may be preferred.

A number of pesticide ranking systems are based on simple algebraic equations, using a format similar to plant breeding selection indexes (Cotterill and Dean 1990) and multiattribute indicators used in the social sciences (Putnam 1993). A generalized form of these equations is:

Environmental Impact Index Value_COMPOSITE = f(b₁x₁ + b₂x₂ + b₃x₃... + b_ix_i).

Each x_i can represent a single test measure, or can be a function of several variables. For example, an x_i estimate of potential pesticide risk to birds could be based solely upon acute lethality (LD₅₀) to adults of one species, or could be a composite of acute lethality (LD₅₀) scores for several species (t, for toxicity), the pesticide’s effects on bird reproduction (r), its persistence in the environment (p), and drift potential (d). In this latter multiattribute assessment of impacts on birds, the first two factors are both indicators of toxicity to birds and the last two are indicators of potential exposure to birds. The model could be structured such that f(x_i) is an additive function equal to (t_j + r_j + p_j + d_j); a multiplicative function, equal to (t_j x r_j x p_j x d_j); or the toxicity scores could be summed and multiplied by the sum of scores estimating bird exposure: (t_j + r_j) x (p_j x d_j). These are several of the simpler functional forms used to estimate impacts on an environmental parameter, but the conceptual possibilities are almost limitless.

Ratios are another commonly used algebraic format:

Generally with these models, the numerator (y_i) is the concentration of the chemical that is measured or predicted to be found in the environment, while the denominator (x_j) is the measurable endpoint for a toxic concentration (e.g., the EC₅₀ or the dosage at which 50% of the organisms of a species cease to function effectively). Exposure potential is therefore treated as an integral part of the Index Value for each variable.

The validity of most algebraic models depends upon complete sets of quantitative input data for all pest controls evaluated by the system. Data gaps have been a limiting factor in the number and types of environmental variables that have been included in this type of assessment model. Algebraic models may be preferred over the logical chain of decision rules when the objective of an assessment system is to generate a tabular ranking of pest controls that can be viewed and utilized separately from the evaluative process. These models are more suitable when the parameters considered are measurable indicators of (relatively) inherent properties, such as LD₅₀ toxicity values. They are less suitable for assessments which are intended to reflect situation-specific conditions and to be sensitive to such variability.

The complexities inherent to deriving a single composite index value, as well as other problems with the structure and input data, limit the utility and reliability of both of these pioneering algebraic models for assessing pesticide impact. Later in this section, improved methods for scoring and structuring algebraic models will be shown, but first an alternative approach, suitable for meeting some assessment objectives, is suggested.

Some multiattribute assessment systems avoid the complications inherent to deriving a single composite index value. These assessments do not attempt to integrate the scores, rank positions or hazard categories for all of the environmental variables considered by the system. Rather the evaluation of each variable stands alone, to be checked off on a list of criteria taken into account by the assessment. Thus no overall rank order or "best choice" among pesticide options is produced. The "checklist" can be used when a composite assessment is not needed. It has been the method of choice for many ecolabeling and other accreditation systems, especially when disparate criteria are used to judge products and/or processes. It has also been used for chemical screening tools where the objective is to prioritize research and regulatory action for chemicals that are flagged as potentially hazardous in any parameter (see, for example, O’Bryan and Ross 1988). There is less need in such systems to generate a composite rank order of pesticides. A generic notation for this type of system is:

|Index Value₁ = f(b₁x₁)|Index Value₂ = f(b₂x₂) |Index Value₃ = f(b₃x₃)|...|Index Value_i = f(b_ix_i)|.

Index values for the set of variables can be presented in a table or spreadsheet array (Figure 4), or ranked by one parameter with "warning flags" to indicate potential hazards in other parameters. For example the Planetor system (1995) is organized primarily as an economic analysis of alternative farm management scenarios. Scenarios with potentially hazardous ramifications in any of several environmental parameters--such as erosion potential or the use of a highly toxic pesticide--are flagged. The checklist method is one solution to the difficult conceptual problem of ranking pest controls and products in more than one dimension--such as public health hazard as well as consumption of non-renewable resources, or environmental impacts and also costs of production.

Weighting of Variables

Weighting is an algebraic way of expressing the relative importance of the variables considered in an assessment. Weights can reflect either the greater/lesser importance of certain variables to the ecosystem (e.g.: impacts to aquatic organisms may be weighted heavily when farm fields are near bodies of surface water) or to the evaluation of the system (e.g.: human life may be valued more than impacts to aquatic organisms).

The most powerful means of assigning weight to a variable in an impact assessment model, however, is to include it in the algebraic equation. No matter what the apparent claims or uses to which an assessment is put, the output of an indexing system reflects no more than the sum of its parts--which are the environmental variables or indicators included in the model. Beyond this truism, a second important point to recognize about the weighting of variables is that all multiattribute assessments which generate a single index score have, effectively, assigned weights of relative importance to each environmental parameter. When no b coefficient is explicitly expressed, implicitly the value of b_i= 1. Thus in models without any b coefficients, all x variables have equal weight. Although the "checklist" method (described above) should obviate the problem of assigning relative importance weights to disparate enviro-social parameters, practitioners need guard against the temptation to mentally create a composite index value while scanning across a spreadsheet of scores for independent indicators.

The EIQ (Kovach et al. 1992) and PestDecide© (Penrose et al. 1995b) models both use weighted b coefficients. However, variables are sometimes also inadvertently weighted by other means--such as using different units or scales of measurement for different variables, or applying inconsistent standards in setting criteria for classification into hazard categories or for assigning scores to the hazard categories. Examples are: using acute mortality (LD₅₀) as the test endpoint for one group of organisms while using effective concentration (EC₅₀) for a temporary disorder as the input data for another group of biota. Or comparing toxicity data from field tests measuring mortality at recommended rates of application--and thus implicitly including a field exposure factor--with toxicity input data from standardized laboratory dosage, unmodified by potential field exposure. Unequal weighting occurs if both sets of toxicity data are multiplied by an indicator of exposure, so that the exposure factor is doubly weighted for the first group.

Weighting factors can be assigned by the system developers, by a regulating or accrediting body, or by stakeholders in a production system (e.g.: producers and/or consumers). Some systems--particularly those which use a workbook procedure for calculating impact ratings--may enable end-users (i.e.: farmers, farm advisors, accrediting groups) to adjust or modify weighting factors to better reflect local conditions or personal considerations. For example, potential impacts from pesticide drift would logically be weighted quite differently when assessing greenhouse production than open field conditions.

Weighting is sometimes criticized because it involves "value judgments." It should not be inferred, however, that these judgments are necessarily prejudicial or illogical. To illustrate, the choice and weighting of factors in a ecolabeling system designed to affect consumer purchasing may logically be quite different than in a system intended as a decision aid for farmers. The former might emphasize a set of global enviro-social concerns, whereas field-scale assessments may instead focus on the local agro-ecosystem, including environmental impacts from pesticides on soil microorganisms and the beneficial arthropods at work in a particular production scenario. As in most matters, there is no one objective reality in environmental impact assessment. System developers have a responsibility first, to acknowledge this and secondly, to insure that the value judgments implicit in their systems reflect the expert judgments of scientists and other stakeholders.

Rigorous decision-making procedures have been developed which formalize and justify parameter selection and weighting of multiple criteria in assessment models. Theories underlying these procedures draw from optimization and risk analysis, game theory, social judgment theory, and multiattribute utility theory (see for example Saaty, 1977; Watson and Buede, 1987; Spires, 1991; Keeney and Raiffa, 1993). These methods have not been stringently applied to many of the pesticide impact assessment indexes--nor are they necessarily to be recommended--because consensus on the relative importance of different variables or modules may not be the primary objective of the weighting system.

Dealing with Data Gaps

Any assessment of pesticide environmental impacts will undoubtedly confront the problem of data gaps and also, in many cases, the opposite situation of more than one datum for an indicator. The protocols chosen to deal with these situations can influence both the scores for individual pest controls and the rank order of pesticides by risk potential. Missing data for some variables may be available from published Quantitative Structure Activity Relationship (QSAR) values. QSARs are multiple regression equations, similar to economic production functions, which estimate environmental effects of chemicals on the basis of known effects from chemicals with similar chemical structures and activity. There is a large literature debating their reliability and proffering alternative QSAR equations for different chemical groups and effects (for a sampling, see various issues of the journals Chemosphere, Environmental Science and Technology and The Science of the Total Environment). In certain situations data gaps can be filled by a procedure that is less formal than using QSARs, but similar in concept: Where gaps result from exemptions from testing requirements because certain pest controls are presumed to have no negative impact, then scores reflecting this assumption can be used to fill these data gaps. Alternatively, some assessment models fill data gaps with the mean scores for similar class of chemicals (this has been as broad a group as "herbicides" but can also be a more narrowly defined group of chemicals, with similar mode of action or similar molecular structure). Caution should be exercised in imputing values to fill data gaps because biological activity of a chemical may be due to small structural details, rather than the primary structure of the chemical--on which the QSAR is generally based.

Several methods are available for choosing from among, or utilizing, multiple data points. If data quality are thought to differ, then the most reliable data from the most appropriate tests should be chosen. If multiple data of similar quality are available, then the most conservative data should be used. This is often called the "most-sensitive species" approach (see discussion of the Ipest in Section 5). Alternatively, multiple data points can be averaged by taking the geometric mean (Watkin and Stelljes 1993). These authors also give EPA extrapolation factors to use when data are available for related species, or related test endpoints. The uncertainty factor increases with increased taxonomic difference. A factor of 5 is used between species in a genus; 10 within families; 20 within an order, with adjustments for greater knowledge on a case-by-case basis. To adjust for acute to chronic exposure a factor of 10 is used; also to adjust for a most sensitive or endangered species. Note that these extrapolation factors provide a buffer for uncertainty, but are not appropriate to use in filling data gaps.

Assigning Scores to x_i

In deriving an index value for potential pesticide risk, environmental endpoint data are first compiled in the units of the measurable parameter (e.g.: LD₅₀s are in units of mg kg^-1; pesticide soil half-life in units of hours, days or weeks; and solubility in ppb). These raw data do not convey meaningful information about potential hazard or impact, however, unless presented in a context that provides some interpretation of significance (e.g.: At what solubility is there a high risk of pesticide leaching to groundwater? How does the lethal dose correspond to the dosage applied?). Moreover, the disparate units of measure cannot be combined into a composite index value, if that is required. Thus, for indexing purposes, environmental and public health data are generally transformed either to numerical scores or to hazard categories (which may also be scored). The following discussion highlights points to consider when developing, reviewing or applying scoring systems. Much of it is relevant both to scoring each x_i, as well as to deriving a composite index value.

US EPA criteria for classifying pesticides into Toxicity Categories to meet pesticide labeling requirements (40 CFR 156.10h) are shown in Column A. Categories are numbered (Col. B) and associated with a warning label (Col. C). The decision rule for assigning test values to hazard categories should be read as follows: If acute oral LD₅₀ is less than or equal to 50 mg kg^-1, then use the EPA Human Hazard Signal Word "Danger" on the pesticide label, and classify the pesticide in Toxicity Category I. Columns D-H are fabricated to illustrate how several scoring functions would assign scores (numerical or descriptive) to test values.

As shown in Figure 5, criteria are set (Column A) to evaluate the raw impact data so that the array of pesticides can be grouped into hazard categories and described (Columns C and D), and/or transformed to numerical scores (as in Columns E-H). Column A shows the doses of pesticide (mg kg^-1) that are considered to have high, moderate, low or no lethal effect on 50% of test organisms. These are the criteria or decision rules for assigning pesticides to hazard categories and/or assigning scores. Column B gives the "numerical name of the category." Several assessment models have made the mistake of confounding these numerical names or category labels with quantitative scores which are manipulated mathematically. The EPA use of Roman numerals diminishes this temptation (Column B), but the effect is shown by the sequential scores in Column E. As a result of using this arithmetic progression, the range of scores in Column E is limited to a 4-fold difference between the most hazardous chemicals and the most benign methods of pest control. This is far less than the >100-fold range of raw scores (Column A).

Moreover, the numerical intervals between categories do not correlate with measured differences in toxic effect between benign and dangerous pest controls. The result is a compressed index, which does not reflect real differences among pesticides. A second ramification of inadvertently using category labels as scores is that benign impacts receive a positive score greater than zero. The scores for benign effects thus contribute to the hazard index value even though the impact does not add to hazard potential. The distortions caused by rating neutral effects as 1 rather than 0 are compounded in systems where exposure and toxicity terms are multiplied to derive an index value. At issue is the anomaly that a high dose of a relatively harmless input, which would not be hazardous at any reasonable exposure, can receive a rating comparable to a highly toxic input used at a lower dose, at which it might still be problematic (i.e.: 1 x 4 = 4 x 1).

Column C gives the US EPA human hazard signal word and Column D is simply another way to verbally describe these categories, using words that are more generally applicable to hazard indicators. Columns E-H are possible ways to score these hazard categories. The four sets of scores in Figure 5 are assigned to the same set of impact categories, yet mathematically they carry different weight. The problems with the scores in Column E are discussed above, and the scores in Column F also rate neutral effects with a positive number. However these scores have an 8:1 ratio between maximum and minimum effects. Is this a sufficient ratio to reflect the range of impacts? It is far less than the range of raw scores (Column A), but may be preferable to the 100:1 range of scores in Column H, which is closer to the range of raw test values. While this range may permit greater sensitivity to differences among chemicals, the 100:1 range of scores would produce composite index values of an unwieldy magnitude. The composite index value could also easily be swayed by a single high scoring variable.

Ideally, the range of scores should parallel the range in magnitude of impacts. An important factor to consider is that the range of impacts may not have a linear correlation with the range of test results. The magnitude of impacts may instead have a threshold dose-response relationship. Unfortunately, relationships between indicator values and meaningful breakpoints in biological, ecological or other impacts are rarely known or are ignored by assessment procedures, and arbitrary scoring criteria are common. Note, for example, that even the criteria for establishing hazard categories (Column A) are based upon magnitudes of order, not on biophysical or ecological response to threshold doses.

A "criteria and indicator matrix" can provide a useful framework for compiling and organizing relevant information about each environmental indicator proposed for inclusion in the assessment system. Each row represents an environmental variable. Columns contain information about the variable, including the criteria for each hazard category, and the score for that category, if it is to be scored. Additional columns contain information about sources of data for the variable, and citations for the sources of expert judgment used in setting scoring and category thresholds (Levitan and Merwin, in progress).

Scoring functions

It is sobering to note that even when the same environmental variable is considered (e.g.: pesticide persistence), and the same indicator is used to measure impact (e.g.: soil half life), different scoring methods can generate radically different results which may affect the ordinal hazard ranking of pesticides. This is illustrated by the Environmental Health Policy Program (University of California at Berkeley) comparison of three methods for scoring their proposed pesticide impact assessment model (Pease et al. 1996 is described in more detail in Section 5). The rank order of pesticides generated by categorical, non-continuous scoring functions is compared with rank orders derived from linear and non-linear continuous scoring functions. The five most hazardous pesticides, as calculated by each scoring method, are listed in Figure 6. The disparity in results is unnerving! Which rank order is correct?

Categorical or "step" functions are used to score the UC Berkeley Environmental Health Policy Program Ranking System. Adjacent columns show the rank order for the same pesticides, using the same assessment system, scored by linear and non-linear functions (Pease et al. 1996).