advertisement

Integrated Environmental Assessment and Management — Volume 7, Number 4—pp. vi–viii ß 2011 SETAC vi Editorial Well Past Time to Stop Using NOELs and LOELs God give me strength to face a fact though it slay me. — Thomas H. Huxley (1825–1895) In this editorial, we convey the fundamental error of the use of no-observed-effect levels (NOELs) and lowestobserved-effect levels (LOELs) (these terms include concentrations as well as levels [e.g., LOECs and NOECs]), alert the ecotoxicology community to the flaws inherent in this practice, and push for this error to cease. We maintain that the time has come, and indeed is past due, to put environmental toxicology, risk assessment and the management decisions that they inform on a scientifically consistent footing. Many researchers have warned that use of NOELs and LOELs reflects a poor application of environmental statistics and laboratory testing (e.g., Skalski 1981; Stephan and Rodgers 1985; Leisenring and Ryan 1992; Hoekstra and Van Ewijk 1993; Laskowski 1995; Chapman et al. 1996; Kooijman 1996; Suter 1996; Moore and Caux 1997; Crane and Newman 2000; Pires et al. 2002). Two recent textbooks point out the lack of technical defensibility of NOELS and LOELS (Newman 2010; Landis et al. 2011). Despite this accumulating evidence, NOELs and LOELs continue to be widely used in the field of ecotoxicology. The problem with NOELs and LOELs is basic. The fundamental model of environmental toxicology is the exposure–response (or concentration–response or dose– response) curve that describes the relationship between exposure and effect. It is a given that the best possible description of this relationship must be the keystone of the field of toxicology. NOEL and LOELs do not meet the criterion of adequately describing the exposure–response curve. The procedure for the derivation of an NOEL and LOEL is the following. First, an analysis of variance is performed to see if differences exist in the groups, and then a multiple comparisons test is done to determine differences among treatments. In the derivation of the NOEL, the lowest exposure that is not different from the control or no-exposure treatment is reported. In other words, what is reported is the treatment for which there is no evidence of an effect given, no a priori model, and no context of what is occurring at the higher and lower exposures. In fact, there can be ample evidence given the entire data set that an effect does occur, but the effect is hidden because only a small subset of the data is used. The LOEL is similarly flawed because only a slice of the exposure–response relationship is used in the analysis. In essence, these types of analyses ignore the fundamental model of toxicology, ignore critical data (at the other exposures), and use a lack of evidence as no-effect. Furthermore, the LOEL and NOEL are merely exposures selected by those doing the testing and are inconsistent between studies. Detailed critiques can be found in Suter (1996), Nelder (1999), and Fox (2008). Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/ieam.249 Given these fatal flaws, NOELs and LOELs should be recognized as extremely poor tools to use as the basis for data interpretation and decision making. After all, NOELs and LOELs are not measurements with an associated standard error or deviation. They are not data, nor are they direct observations, but are simple labels for experimental treatments. Yet, these labels of treatment groups are often used in the calculation of species sensitivity distributions and other data compilations. These compilations appear scientific but sacrifice the exposure–response underpinning of toxicology. A clear alternative exists. Curve-fitting can use all of the data obtained from a toxicity experiment, can express the variability and uncertainty of the data and of the model, and provides information on the slope of the response. Because replication at each exposure is not required, a broader range of exposure–response interactions can be observed at the same level of effort, better describing the exposure–response. Scientists using curve-fitting have long demonstrated the power of this approach (Litchfield and Wilcoxon 1949). Stephan (1977) reported methods for calculating median lethal concentrations (LC50s) that involved both curve-fitting and the calculation of confidence limits. More than 25 y ago, Stephan and Rodgers (1985) showed the power of the regression approach in the analysis of chronic toxicity data. Moore and Caux (1997) demonstrated the ability to model exposure curves for a wide variety of data sets. The important feature of the data set was to have a wide range of exposures so that the curve-fitting program would have information at both the highest response possible and the lowest exposures that could be tested. Two examples of experimental design that optimizes the derivation of exposure–response are publications by Rider and LeBlanc (2005) and Olmstead and LeBlanc (2005). In each example, the data from the experiments are shown, and the exposure–response curve is plotted, illustrating the relationship between the curve fit and the data. Measurements are also taken over a range of responses, so that the maximum response and the inflection of the curve at the lower exposures, which reflects the threshold of effect, can be ascertained. A comparison between results obtained via curve-fitting and the determination of NOELs and/or LOELSs for the same data sets has been done. Moore and Caux (1997) demonstrated that NOELs typically correspond to an EC10 to EC30 on an exposure–response curve. This means that NOELs incorporate a 10% to 30% effect, a confidence interval surrounds each estimate, and these values do not correspond to a no-effect level. Curve-fitting is a more transparent and accurate approach than an NOEL for estimating no-effect exposures, and the confidence interval provides a measure of the uncertainty of the estimate. There are no computational reasons why exposure– response curves are uncommon in the ecotoxicological literature. Caux and Moore (1997) published a spreadsheet tool for estimating exposure–response curves. GraphPad, SPSS, and other software can perform the calculations. Stephenson et al. (2000) present a number of regression techniques for estimating exposure–response curves for plant Editorial—Integr Environ Assess Manag 7, 2011 toxicity. A newer and perhaps more useful method has been derived. Fox (2006, 2008, 2010a, 2010b) developed a Bayesian approach for determining a no-effect exposure from exposure–response data sets. A Bayesian approach does require a prior, which is an initial model. The model to be used as a prior can be derived by the investigator observing the data. The program written in the WinBUGS code then uses an iterative process to determine the curve and estimate the exposure that does not result in the effect being measured. Similar to confidence intervals, credibility limits provide an indication of the uncertainty of the estimate. Although new to most environmental toxicologists, the derivation of the exposure–response curve using a Bayesian approach appears powerful. In the laboratory of WG Landis, estimating exposure–response relationships on trial data sets appears to be a straightforward and relatively rapid exercise, through use of WinBUGS. The open-access version of WinBUGS, OpenBUGS (http://www.openbugs.info/w/) is free, and the program is available. Although still early in its development, the only substantial obstacle to the use of such an approach seems to be the training of personnel to perform the analysis. The preceding paragraphs have illustrated the flaws in the NOEL and/or LOEL approach to the interpretation and use of toxicity data. Data analysis approaches that describe the exposure–response curve are technically defensible and consistent. As shown above, there are no scientific or technological reasons for not using exposure–response curves as the basis for describing toxicological effects. Given these facts, we advocate adoption of curve-fitting as the standard interpretation of laboratory test data and urge rejection of the technically indefensible use of the NOEL and/or LOEL approach. Unfortunately, NOELs and LOELs have historically been widely used in peer-reviewed articles. For instance, a search on 23 May 2011 for the term ‘‘NOEC’’ in the 2 journals of the Society for Environmental Toxicology and Chemistry (SETAC), Environmental Toxicology and Chemistry and Integrated Environmental Assessment and Management, found 588 and 73 articles, respectively. We offer a series of approaches for countering the historical norm. We propose that toxicologists, in particular SETAC members, follow 3 guidelines: 1) In the field of (eco)toxicology, curve-fitting is the preferred approach for establishing exposure–response relationships; 2) This approach should include the calculation of confidence or credibility intervals with supporting raw data archived as supplemental data associated with the conclusions derived from such an exercise. We note in this regard that the journals of the Entomological Society of America will not publish exposure–response data without confidence interval or slope data; 3) Work that treats NOELs and LOELs as data for further analysis, for example, the derivation of species sensitivity distribution curves, should be subject to intense statistical and scientific scrutiny. We understand that, for many toxicity tests, NOELs and LOELs are the only results that remain. However, the demonstrated uncertainties in the accurate representation of the exposure–response curves need to be acknowledged. The onus now has to be on the users of such information to ensure that any conclusions inferred from this flawed, technically indefensible approach are appropriately presented and caveated. vii To hasten the adoption of curve-fitting, we call on the Editors-in-Chief of the 2 SETAC journals to ban statistical hypothesis tests for the reporting of exposure–response from their journals. There is precedence for this; when the renowned epidemiologist Kenneth Rothman was Editor-inChief of the American Journal of Epidemiology, he banned statistical hypothesis tests from that journal. We also call on regulatory agencies across the world to ban statistical hypothesis tests for the reporting of exposure– response from their guidance documents. We note that the US Environmental Protection Agency is already moving in this direction, using its Benchmark Dose software (Crump 1995) to derive exposure–response curves, confidence limits, and supporting statistics in their toxicological summaries for human health assessments. Such curves are fit to the toxicity data when possible, even if the original authors used hypothesis testing. However, the decades-old test guidelines still encourage NOELs and LOELs. These technically indefensible practices must cease. Acknowledgment—We thank 2 anonymous and independent reviewers for their prompt and thoughtful review of this editorial. We also thank the Editors-in-Chief of the 2 SETAC journals (Rick Wenning, Herb Ward and Alan Burton) for reviewing this editorial and allowing it to be published. Wayne G Landis Western Washington University Bellingham, Washington, USA Peter M Chapman Golder Associates Ltd. Burnaby, BC, Canada REFERENCES Caux P-Y, Moore DRJ. 1997. A spreadsheet program for estimating low toxic effects. Environ Toxicol Chem 16:802–806. Chapman PM, Caldwell RS, Chapman PF. 1996. A warning: NOECs are inappropriate for regulatory use. Environ Toxicol Chem 15:77–79. Crane N, Newman MC. 2000. What level of effect is a no observed effect? Environ Toxicol Chem 19:516–519. Crump KS. . Calculation of benchmark doses from continuous data. Risk Anal 15:79–89. Fox DR. 2006. Statistical issues in ecological risk assessment. Human Ecol Risk Assess 12:120–129. Fox DR. 2008. NECS, NOECS and the ECX. Australas J Ecotox 14:7–9. Fox DR. 2010a. Statistics and ecotoxicology: shotgun marriage or enduring partnership? Integr Environ Assess Manag 6:499–506. Fox DR. 2010b. A Bayesian approach for determining the no effect concentration and hazardous concentration in ecotoxicology. Ecotoxicol Environ Saf 73:123– 131. Hoekstra JA, Van Ewijk PH. 1993. Alternatives for the no-observed-effect level. Environ Toxicol Chem 12:187–194. Kooijman SAL. 1996. An alternative for the NOEC exists, but the standard model has to be abandoned first. Oikos 75:310–316. Landis WG, Sofield RM, Yu M-H. 2011. Introduction to environmental toxicology: Molecular substructures to ecological landscapes 4th ed. Boca Raton (FL): Lewis/CRC. 524 p. Laskowski R. 1995. Some good reasons to ban the use of the NOEC, LOEC and related concepts in ecotoxicology. Oikos 73:140–144. Leisenring W, Ryan LM. 1992. Statistical properties of the NOAEL. Regul Toxicol Pharmacol 8:161–171. viii Integr Environ Assess Manag 7, 2011—WG Landis and PM Chapman Litchfield JT, Wilcoxon F. 1949. A simplified method of evaluating dose-effect Special Technical Publication 737. Philadelphia (PA): American Society for experiments. J Pharmacol Exp Ther 96:99–113. Moore DRJ, Caux P. 1997. Estimating low toxic effects. Environ Toxicol Chem Testing and Materials. p 337-387. Stephan CE. 1977. Methods for calculating an LC50. In: Mayer FL, Hamelink JL, eds. 16:794–801. Aquatic toxicology and hazard evaluation, 1st annual symposium; 1976 Nelder JA. 1999. Statistics for the millennium: From statistics to statistical science. The Statistician 48:257–269. October 25-26; Nashville, Tennessee. ASTM Special Technical Publication 634. Philadelphia (PA): American Society for Testing and Materials. Newman MC. 2010. Fundamentals of toxicology 3rd ed. Boca Raton (FL): Lewis/ CRC. 480 p. p 65–84. Stephan CE, Rodgers JR. 1985. Advantages of using regression analysis to calculate Olmstead AW, LeBlanc GA. 2005. Toxicity assessment of environmentally relevant pollutant mixtures using a heuristic model. Integr Environ Assess Manage results of chronic toxicity tests. In: Bahner RC, Hansen DJH, eds. Aquatic toxicology and hazard assessment, 8th symposium; 1984 April 15-17; Fort 1:114–122. Pires AM, Branco AJ, Picado A, Mendonça E. 2002. Models for the estimation of a Mitchell, Kentucky. ASTM Special Technical Publication 891. Philadelphia (PA): American Society for Testing and Materials. p 328–339. ‘no effect’ concentration. Environmetrics 13:15–27. Rider CV, LeBlanc GA. 2005. An integrated addition and interaction model for Stephenson GLN, Koper N, Atkinson GF, Solomon KR, Scroggins RP. 2000. Use of nonlinear regression techniques for describing concentration-response assessing toxicity of chemical mixtures. Toxicol Sci 87:520–528. Skalski JR. 1981. Statistical inconsistencies in the use of no-observed-effect-levels in relationships of plant species exposed to contaminated site soils. Environ Toxicol Chem 19:2968–2981. toxicity testing. In: Branson DR, Dickson KL, eds. Aquatic toxicology and hazard evaluation, 4th Conference; 1979 October 16-17; Chicago, Illinois. ASTM Suter GW II. 1996. Abuse of hypothesis testing statistics in ecological risk assessment. Hum Ecol Risk Assess 2:331–347.