Well past time to stops using NOELs and LOELs

Integrated Environmental Assessment and Management — Volume 7, Number 4—pp. vi–viii
ß 2011 SETAC
vi
Editorial
Well Past Time to Stop Using NOELs and LOELs
God give me strength to face a fact though it slay me. —
Thomas H. Huxley (1825–1895)
In this editorial, we convey the fundamental error of the
use of no-observed-effect levels (NOELs) and lowestobserved-effect levels (LOELs) (these terms include concentrations as well as levels [e.g., LOECs and NOECs]), alert the
ecotoxicology community to the flaws inherent in this
practice, and push for this error to cease. We maintain that
the time has come, and indeed is past due, to put environmental toxicology, risk assessment and the management
decisions that they inform on a scientifically consistent
footing.
Many researchers have warned that use of NOELs and
LOELs reflects a poor application of environmental statistics
and laboratory testing (e.g., Skalski 1981; Stephan and
Rodgers 1985; Leisenring and Ryan 1992; Hoekstra and Van
Ewijk 1993; Laskowski 1995; Chapman et al. 1996; Kooijman
1996; Suter 1996; Moore and Caux 1997; Crane and
Newman 2000; Pires et al. 2002). Two recent textbooks
point out the lack of technical defensibility of NOELS and
LOELS (Newman 2010; Landis et al. 2011). Despite this
accumulating evidence, NOELs and LOELs continue to be
widely used in the field of ecotoxicology.
The problem with NOELs and LOELs is basic. The
fundamental model of environmental toxicology is the
exposure–response (or concentration–response or dose–
response) curve that describes the relationship between
exposure and effect. It is a given that the best possible
description of this relationship must be the keystone of
the field of toxicology. NOEL and LOELs do not meet the
criterion of adequately describing the exposure–response
curve.
The procedure for the derivation of an NOEL and LOEL is
the following. First, an analysis of variance is performed to see
if differences exist in the groups, and then a multiple
comparisons test is done to determine differences among
treatments. In the derivation of the NOEL, the lowest
exposure that is not different from the control or no-exposure
treatment is reported. In other words, what is reported is the
treatment for which there is no evidence of an effect given, no
a priori model, and no context of what is occurring at the
higher and lower exposures. In fact, there can be ample
evidence given the entire data set that an effect does occur,
but the effect is hidden because only a small subset of the data
is used. The LOEL is similarly flawed because only a slice of
the exposure–response relationship is used in the analysis. In
essence, these types of analyses ignore the fundamental model
of toxicology, ignore critical data (at the other exposures),
and use a lack of evidence as no-effect. Furthermore, the
LOEL and NOEL are merely exposures selected by those
doing the testing and are inconsistent between studies.
Detailed critiques can be found in Suter (1996), Nelder
(1999), and Fox (2008).
Published online in Wiley Online Library
(wileyonlinelibrary.com).
DOI: 10.1002/ieam.249
Given these fatal flaws, NOELs and LOELs should be
recognized as extremely poor tools to use as the basis for data
interpretation and decision making. After all, NOELs and
LOELs are not measurements with an associated standard
error or deviation. They are not data, nor are they direct
observations, but are simple labels for experimental treatments. Yet, these labels of treatment groups are often used in
the calculation of species sensitivity distributions and other
data compilations. These compilations appear scientific but
sacrifice the exposure–response underpinning of toxicology.
A clear alternative exists. Curve-fitting can use all of the
data obtained from a toxicity experiment, can express the
variability and uncertainty of the data and of the model, and
provides information on the slope of the response. Because
replication at each exposure is not required, a broader range
of exposure–response interactions can be observed at the
same level of effort, better describing the exposure–response.
Scientists using curve-fitting have long demonstrated the
power of this approach (Litchfield and Wilcoxon 1949).
Stephan (1977) reported methods for calculating median
lethal concentrations (LC50s) that involved both curve-fitting
and the calculation of confidence limits. More than 25 y ago,
Stephan and Rodgers (1985) showed the power of the
regression approach in the analysis of chronic toxicity data.
Moore and Caux (1997) demonstrated the ability to model
exposure curves for a wide variety of data sets. The important
feature of the data set was to have a wide range of exposures
so that the curve-fitting program would have information at
both the highest response possible and the lowest exposures
that could be tested.
Two examples of experimental design that optimizes the
derivation of exposure–response are publications by Rider and
LeBlanc (2005) and Olmstead and LeBlanc (2005). In each
example, the data from the experiments are shown, and the
exposure–response curve is plotted, illustrating the relationship between the curve fit and the data. Measurements are
also taken over a range of responses, so that the maximum
response and the inflection of the curve at the lower
exposures, which reflects the threshold of effect, can be
ascertained.
A comparison between results obtained via curve-fitting
and the determination of NOELs and/or LOELSs for the
same data sets has been done. Moore and Caux (1997)
demonstrated that NOELs typically correspond to an EC10 to
EC30 on an exposure–response curve. This means that
NOELs incorporate a 10% to 30% effect, a confidence
interval surrounds each estimate, and these values do not
correspond to a no-effect level. Curve-fitting is a more
transparent and accurate approach than an NOEL for
estimating no-effect exposures, and the confidence interval
provides a measure of the uncertainty of the estimate.
There are no computational reasons why exposure–
response curves are uncommon in the ecotoxicological
literature. Caux and Moore (1997) published a spreadsheet
tool for estimating exposure–response curves. GraphPad,
SPSS, and other software can perform the calculations.
Stephenson et al. (2000) present a number of regression
techniques for estimating exposure–response curves for plant
Editorial—Integr Environ Assess Manag 7, 2011
toxicity. A newer and perhaps more useful method has been
derived.
Fox (2006, 2008, 2010a, 2010b) developed a Bayesian
approach for determining a no-effect exposure from exposure–response data sets. A Bayesian approach does require a
prior, which is an initial model. The model to be used as a
prior can be derived by the investigator observing the data.
The program written in the WinBUGS code then uses an
iterative process to determine the curve and estimate the
exposure that does not result in the effect being measured.
Similar to confidence intervals, credibility limits provide an
indication of the uncertainty of the estimate.
Although new to most environmental toxicologists, the
derivation of the exposure–response curve using a Bayesian
approach appears powerful. In the laboratory of WG Landis,
estimating exposure–response relationships on trial data sets
appears to be a straightforward and relatively rapid exercise,
through use of WinBUGS. The open-access version of
WinBUGS, OpenBUGS (http://www.openbugs.info/w/) is
free, and the program is available. Although still early in its
development, the only substantial obstacle to the use of such
an approach seems to be the training of personnel to perform
the analysis.
The preceding paragraphs have illustrated the flaws in the
NOEL and/or LOEL approach to the interpretation and use
of toxicity data. Data analysis approaches that describe
the exposure–response curve are technically defensible and
consistent. As shown above, there are no scientific or
technological reasons for not using exposure–response curves
as the basis for describing toxicological effects. Given these
facts, we advocate adoption of curve-fitting as the standard
interpretation of laboratory test data and urge rejection of the
technically indefensible use of the NOEL and/or LOEL
approach.
Unfortunately, NOELs and LOELs have historically been
widely used in peer-reviewed articles. For instance, a search
on 23 May 2011 for the term ‘‘NOEC’’ in the 2 journals of
the Society for Environmental Toxicology and Chemistry
(SETAC), Environmental Toxicology and Chemistry and
Integrated Environmental Assessment and Management, found
588 and 73 articles, respectively. We offer a series of
approaches for countering the historical norm.
We propose that toxicologists, in particular SETAC
members, follow 3 guidelines: 1) In the field of (eco)toxicology, curve-fitting is the preferred approach for establishing
exposure–response relationships; 2) This approach should
include the calculation of confidence or credibility intervals
with supporting raw data archived as supplemental data
associated with the conclusions derived from such an exercise.
We note in this regard that the journals of the Entomological
Society of America will not publish exposure–response data
without confidence interval or slope data; 3) Work that
treats NOELs and LOELs as data for further analysis, for
example, the derivation of species sensitivity distribution
curves, should be subject to intense statistical and scientific
scrutiny. We understand that, for many toxicity tests, NOELs
and LOELs are the only results that remain. However, the
demonstrated uncertainties in the accurate representation of
the exposure–response curves need to be acknowledged. The
onus now has to be on the users of such information to ensure
that any conclusions inferred from this flawed, technically
indefensible approach are appropriately presented and
caveated.
vii
To hasten the adoption of curve-fitting, we call on the
Editors-in-Chief of the 2 SETAC journals to ban statistical
hypothesis tests for the reporting of exposure–response from
their journals. There is precedence for this; when the
renowned epidemiologist Kenneth Rothman was Editor-inChief of the American Journal of Epidemiology, he banned
statistical hypothesis tests from that journal.
We also call on regulatory agencies across the world to ban
statistical hypothesis tests for the reporting of exposure–
response from their guidance documents. We note that the
US Environmental Protection Agency is already moving in
this direction, using its Benchmark Dose software (Crump
1995) to derive exposure–response curves, confidence limits,
and supporting statistics in their toxicological summaries for
human health assessments. Such curves are fit to the toxicity
data when possible, even if the original authors used
hypothesis testing. However, the decades-old test guidelines
still encourage NOELs and LOELs. These technically
indefensible practices must cease.
Acknowledgment—We thank 2 anonymous and independent reviewers for their prompt and thoughtful review of
this editorial. We also thank the Editors-in-Chief of the
2 SETAC journals (Rick Wenning, Herb Ward and Alan
Burton) for reviewing this editorial and allowing it to be
published.
Wayne G Landis
Western Washington University
Bellingham, Washington, USA
Peter M Chapman
Golder Associates Ltd.
Burnaby, BC, Canada
REFERENCES
Caux P-Y, Moore DRJ. 1997. A spreadsheet program for estimating low toxic effects.
Environ Toxicol Chem 16:802–806.
Chapman PM, Caldwell RS, Chapman PF. 1996. A warning: NOECs are
inappropriate for regulatory use. Environ Toxicol Chem 15:77–79.
Crane N, Newman MC. 2000. What level of effect is a no observed effect? Environ
Toxicol Chem 19:516–519.
Crump KS. . Calculation of benchmark doses from continuous data. Risk Anal
15:79–89.
Fox DR. 2006. Statistical issues in ecological risk assessment. Human Ecol Risk
Assess 12:120–129.
Fox DR. 2008. NECS, NOECS and the ECX. Australas J Ecotox 14:7–9.
Fox DR. 2010a. Statistics and ecotoxicology: shotgun marriage or enduring
partnership? Integr Environ Assess Manag 6:499–506.
Fox DR. 2010b. A Bayesian approach for determining the no effect concentration
and hazardous concentration in ecotoxicology. Ecotoxicol Environ Saf 73:123–
131.
Hoekstra JA, Van Ewijk PH. 1993. Alternatives for the no-observed-effect level.
Environ Toxicol Chem 12:187–194.
Kooijman SAL. 1996. An alternative for the NOEC exists, but the standard model
has to be abandoned first. Oikos 75:310–316.
Landis WG, Sofield RM, Yu M-H. 2011. Introduction to environmental toxicology:
Molecular substructures to ecological landscapes 4th ed. Boca Raton (FL):
Lewis/CRC. 524 p.
Laskowski R. 1995. Some good reasons to ban the use of the NOEC, LOEC and
related concepts in ecotoxicology. Oikos 73:140–144.
Leisenring W, Ryan LM. 1992. Statistical properties of the NOAEL. Regul Toxicol
Pharmacol 8:161–171.
viii
Integr Environ Assess Manag 7, 2011—WG Landis and PM Chapman
Litchfield JT, Wilcoxon F. 1949. A simplified method of evaluating dose-effect
Special Technical Publication 737. Philadelphia (PA): American Society for
experiments. J Pharmacol Exp Ther 96:99–113.
Moore DRJ, Caux P. 1997. Estimating low toxic effects. Environ Toxicol Chem
Testing and Materials. p 337-387.
Stephan CE. 1977. Methods for calculating an LC50. In: Mayer FL, Hamelink JL, eds.
16:794–801.
Aquatic toxicology and hazard evaluation, 1st annual symposium; 1976
Nelder JA. 1999. Statistics for the millennium: From statistics to statistical science.
The Statistician 48:257–269.
October 25-26; Nashville, Tennessee. ASTM Special Technical Publication
634. Philadelphia (PA): American Society for Testing and Materials.
Newman MC. 2010. Fundamentals of toxicology 3rd ed. Boca Raton (FL): Lewis/
CRC. 480 p.
p 65–84.
Stephan CE, Rodgers JR. 1985. Advantages of using regression analysis to calculate
Olmstead AW, LeBlanc GA. 2005. Toxicity assessment of environmentally relevant
pollutant mixtures using a heuristic model. Integr Environ Assess Manage
results of chronic toxicity tests. In: Bahner RC, Hansen DJH, eds. Aquatic
toxicology and hazard assessment, 8th symposium; 1984 April 15-17; Fort
1:114–122.
Pires AM, Branco AJ, Picado A, Mendonça E. 2002. Models for the estimation of a
Mitchell, Kentucky. ASTM Special Technical Publication 891. Philadelphia (PA):
American Society for Testing and Materials. p 328–339.
‘no effect’ concentration. Environmetrics 13:15–27.
Rider CV, LeBlanc GA. 2005. An integrated addition and interaction model for
Stephenson GLN, Koper N, Atkinson GF, Solomon KR, Scroggins RP. 2000. Use of
nonlinear regression techniques for describing concentration-response
assessing toxicity of chemical mixtures. Toxicol Sci 87:520–528.
Skalski JR. 1981. Statistical inconsistencies in the use of no-observed-effect-levels in
relationships of plant species exposed to contaminated site soils. Environ
Toxicol Chem 19:2968–2981.
toxicity testing. In: Branson DR, Dickson KL, eds. Aquatic toxicology and hazard
evaluation, 4th Conference; 1979 October 16-17; Chicago, Illinois. ASTM
Suter GW II. 1996. Abuse of hypothesis testing statistics in ecological risk
assessment. Hum Ecol Risk Assess 2:331–347.