Some Practical Problems of Recent Nonparametric Procedures

Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
•
.
SI
1.
Some Practical Problems of Recent
Nonparametric Procédures: Testing, Estimation,
and Application.
Jorge Barrientos-Marin
Advisor: Stefan Sperlich
Quantitative Economies Doctorale
Departamento de Fundamentos del Análisis Económico
Universidad de Alicante
January 2007
mff' /',*.
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
i~
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
To my wife and my family.
1
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Agradecimientos
Los artículos que componen esta tesis es el resultado de cinco años de trabajo
continuo. Pero sin duda, esto no habría sido posible sin la colaboración de muchas
personas. Quiero expresar mi gratitud para con todas ellas. Sí alguno se queda sin
mencionar, lo más posible es que mi memoria, como es usual, me juegue una mala
pasada. Quiero entonces expresar mi reconocimiento a los miembros del departamento de Fundamentos del Análisis Económico, a mis profesores y especialmente a
mis condiscípulos, ellos hicieron estos cinco años soportables lejos de casa. Especial
reconocimiento entre profesores merecen Antonio Villar, quien confió en mi siempre
y fue consejero en momentos difíciles, a Juan Mora por proveerme ánimo y ratos
agradables discutiendo resultados y teoremas, al igual que Javier Alvarez y a Lola,
quienes con sus excelentes cursos me animaron a seguir el camino de la econometría.
Entre mis condiscípulos agradezco a Alicia quien siempre ha sido una amiga.
Agradezco a Ricardo su ayuda e innumerables favores (muchos de ellos pecuniarios)
y a Paco, Silvio y Szaby su compañía placida y su amistad sincera. A Fafael López,
su buen humor e intelegiencia fueron un reto para mi. Agradezco a José Maria su
gran aprecio para conmigo, algo que es mutuo, y su generosidad, estos años habrían
sido menos divertidos y algunas navidade tristes sin su amistad.
No puedo dejar de mencionar al personal administrativo (Mercedes, Mariló, Julio,
Carlos y Lourdes) siempre estuvieron atentos a ayudarme y tuvieron paciencia para
mis innumerables solicitudes.
Agradezco también a Frédéric Ferraty y a Philippe Vieu su dedicación, ellos me
proveyeron la mejor atmósfera para hacer uno de los capítulos que componen esta
tesis. Aquí merece mención Juan y Mónica, quienes me acogieron en su casa y
siempre fueron compañía, además de introducirme, en modo nada superficial, en los
aspectos de la vida francesa.
Agradezco a mi familia, en especial a Patricia, mi esposa, quien me ha poyado
todos estos años de semi-soledad a la espera de que esto acabara, siempre con paciencia y optimismo. A mi madre, quien sé que mi ausencia siempre la entristeció.
A mis tías, para quienes soy un orgullo. A José y Leticia Restrepo por ayudar a
Patricia a llevar la carga de la soledad.
Agradezco a mis amigos en Colombia, a Mauricio Alviar y a Pedro, quienes
desde un comienzo creyeron que esto era posible de alcanzar. Menciono también a
Alejandro Gaviria, quien continua enseñándome a pensar como un economista, me
dio además consejos acertados en el momento justo.
Finalmente, un reconocimiento especial merece Stefan Sperlich, quien me ha enseñado mucho de econometría semi y noparametrica. Estaré siempre agradecido con
él, porque se preocupó de que esto terminara bien y ha sido un director excepcional
aún desde la distancia.
2
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Contents
Agradecimientos
Introduction and Summary
Introducción y Resumen en Español
1
2
5
8
The Size Problem of Kernel Based Bootstrap Tests when the Nuil
is Nonparametric
12
1.1 Introduction
12
1.2 Statistical Methods: Estimators and Test Statistics
14
1.2.1 Estimators
14
1.2.2 Test Statistics
15
1.3 Resampling and Choice of Parameters
17
1.3.1 Bootstrap Tests
18
1.3.2 The Choice of Bandwidths h
19
1.3.3 The Choice of Bandwidths k
19
1.3.4 The Choice of Bootstrap Residuals
20
1.3.5 An Alternative: Subsampling
21
1.3.6 The Choice of Bootstrap Bandwidth/i;,
23
1.4 Simulation Results
23
1.5 Conclusions
27
Références
29
2 Estimating and Testing An Additive Partially Linear Model in a
System of Engel Curves
37
2.1 Introduction
37
2.2 Additive Partially Linear Model and Testing Hypothesis
40
2.3 The Shape of Engel Curves and Spécification Testing
45
2.3.1 Data Used in this Application
48
2.3.2 Some Pictures of the Expenditure expenditure-Log Total Expenditure Relationship
49
2.3.3 Spécification Testing
56
2.4 Conclusions and Future Research
59
Références
61
3
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
3 Locally Modelled Régression and Functional Data
3.1 Introduction
3.2 Position of the Problem
3.3 Functional locally modeled régression
3.3.1 The p-dimensional case
3.3.2 The infinite-dimensional case: the functional setting
3.4 FFLM kernel-type estimator: asymptotic behavior
3.5 FFLM régression in action
3.6 Conclusions
Références
4
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
64
64
67
68
68
69
73
77
80
88
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Introduction and Summary
This thesis is composed of three chapters, in which we focus on three related, but
différent, issues regarding testing, estimation and theoretical developments1. More
precisely, in Chapter 1, "The Size Problem of Kernel Based Bootstrap Tests when
the Nuil is Nonparametric", we are interested in clioosing an appropriate smoothing parameter, a problem that is fundamental for the reasonable use of non- and
semiparametric methods. In particular for testing, we make note the this problem
is not équivalent to the one in régression. At least from a theoretical point of view,
the optimal smoothing parameter for testing has différent rates from those which
are optimal for estimation.
While there exists an increasing literature on how to find a proper smoothing
parameter for the nonparametric alternative, almost nothing is known on how to
choose a smoothing parameter in practice for the nuil hypothesis if it is also semi- or
nonparametric. We do know that at least asymptotically oversmoothing is necessary
in the pre-estimation of the nuil model for generating the bootstrap samples, see
Hardie and Marron (1990,1991). However, in practice this knowledge is of little
help. The same can be said about various parameters and procédures to be chosen
in practice when performing such tests. In this Chapter, we discuss ail thèse choice
questions. In particular we study the problem of bandwidth choice for the preestimation to genérate bootstrap samples. As an alternative, we also discuss briefly
the possibility of subsampling. 2 .
In Chapter 2, "Estimating and Testing An Additive Partially Linear Model in
a System of Engel Curves ", we focuses on an application of additive partial linear
model and some ideas extracted from applications on Chapter 1. Our main goal is
to make an application to consumer theory. More exactly, to Engel curves Systems.
The form of the Engel curve has long been a subject of discussion in applied econoChapter 1 is a joint work with Stefan Sperlich and Chapter 3 is a joint work with Frédéric
Ferraty and Philippe Vieu.
2
The authors gratefully acknowledge financial support from the Spanish DG de Investigación
del Ministerio de Ciencia y Tecnología. SEJ2004-04583/ECON.
5
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
metrics and until now there has no been definitive conclusion about its form. In this
Chapter an additive partially linear model is used to estimate semiparametrically
the effect of total expenditure in this context. Additionally, we consider the nonparametric inclusion of some regressors which traditionally have a non linear effect
such as age and schooling. To that end we compare an additive partially linear
model with the fully nonparametric one using recent popular test statistics. Because of inference in nonparametric regression can take place in a number of ways,
the most natural is to use nonparametric regression as an alternative against a fully
parametric or semiparametric null hypothesis. Then, for investigating purpose we
check whether an additive PLM provides a reasonable adjustment to our data using
different resampling schemes to obtain critical (p-values) computed by bootstrap
and subsampling schemes for the proposed test statistics.
Additionally, in this Chapter, we dealing with a well-known problem very common in the context of Engel curves, it is that total expenditure may well be jointly determined with expenditure on different goods. Therefore, endogeneity problem may
arise. In order to solve this problem we are interested in applying nonparammetric
constructed regressors as instrumental variables. In particular, we use the nonparametric two step with generated regressors and constructed variables (NP2SCV) due
to Sperlich (2005). Our feeling is that a generated variables approach in combination
with additive PLM can help us to overcome to some extent any possible endogeneity
problem and that is exactly the procedure implemented in this Chapter.
In Chapter 3, "Locally Modelled Regression and Functional Data"3, we are interested in extend nonparametric methods when the regressors are functions (i.e.
one observation could be curve, surface or any other object lying into an infinite
dimensional space). From a statistical pint of view, this corresponds to a functional
regression setting because on wishes to predict a response Y from an explanatory
functional variable X. In addition, only regularity conditions on regression operator
J
A c k n o w l e d g e m e n t . The authors thank gratefully the members of the working group STAPH
(http : //www.lsp.ups — tlse.fr/staph)
for their helpful comments and discussions. In addition, the
first author acknowledges financial support from the Spanish Ministry of Education and Science,
under project BEC2001-0535
6
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
are assumed. Then, this leads us to the nonparametric context. So, the problematic
of this work deal with the nonparametric functional regression. Recently, there are
several works dealing with the nonparametric functional regression (see for instance
Ferraty and Vieu (2002, 2005)). This nonparametric functional regression method is
essentially based on an extension of the well-known Nadaraya(1964)-Watson(1964)
kernel regression estimator of the regression, to the case of functional explanatory
variable. On the other hand, local linear ideas have been developed in the regression
context for univariate and multivariate explanatory variable, see Wand and Jones
(1995) for an overview of this topic. Therefore, our work can be considered as an
extension, which is a combination, of the nonparametric local constant method with
the ideas of functional variable. So, the aim of this setting does not make easy both
the asymptotic study and the implementation of a natural generalization of the multivariate local linear method. Therefore, one focuses on a simpler and faster local
approach. Asymptotic properties are stated, and a functional dataset illustrates the
good behavior of this fast functional local modelled regression method.
7
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Introducción y Resumen en Español
Esta tesis esta compuesta por tres capítulos, los cuales se centran en tres diferentes
problemas, aunque relacionados, estos van desde estimación y contrastes de hipótesis
hasta desarrollos teóricos. Más exactamente, en el Capítulo 1, " The Size Problem of
Kernel Based Bootstrap Tests when the Null is Nonparametric", nosotros estamos
interesados en la selección apropiada de un parámetro de suavización, un problema
que es fundamental para un razonable use de los métodos semi y noparamétricos.
En particular, para contrastes de hipótesis, nosotros notamos que este problema no
es equivalente aquel que se presenta en análisis de regresión, esto es en la simple
estimación. Al menos desde un punto de vista teórico, la selección del parámetro
para contrastes de hipótesis tiene tasas (de convergencia) diferents a las que se
supone debe tener los parámetros que son óptimos para la estimación.
Mientras que existe una creciente literatura sobre el modo de hallar un parámetro
apropiado para la hipótesis alternativa, casi nada es sabido sobre como elegir un
parámetro de suavización en la práctica para la hipótesis nula, si esta es también
semiparamétrica o incluso noparamétrica. Solo sabemos que asintóticamente una
parámetro sobresuavizado es necesario en la preestimación del modelo bajo la nula
para generar las muestras bootstrap, ver al respecto Hárdle and Marrón (1990,1991).
Sin embargo, en la práctica este conocimiento es de poca ayuda. Lo mismo puede
decirse acerca de varios parámetros y procedimientos a ser elegidos en la práctica
cuando hacemos un uso de un procedimiento de contraste. En este Capítulo entonces, nosotros discutimos estas cuestiones acerca de la selección. En particular,
nosotros estudiamos el problema de la selección del parámetro de suavizado en la
pre-estimación para generar las muestras bootstrap. Como alternativa, también
discutimos brevemente la posibilidad de submuestras.
En el Capítulo 2, "Estimation and Testing An Additive Partially Linear Model
in a System of Engel Curves", nosotros nos centramos en la aplicción de modelos
aditivos parcialemente lineales basados en algunas ideas del Capítulo 1. Nuestra
meta es hacer una aplicación en teoría del consumidor. Específicamente a sistemas
8
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
de curvas de Engel. La forma de la curva de Engel ha sido por mucho tiempo objeto
de investigación en econometría aplicada y hasta el momento no hay conclusiones
definitivas sobre su forma. En este capítulo un modelo parcialmente aditivo es usado para estimar semiparametricamente el efecto del gasto total. Adicionalmente,
consideramos la inclusión noparamétrica de algunos regresores que tradicionalmente
tienen un efecto no-lineal como la edad y la escolaridad. Para llevar a cabo este
trabajo, comparamos un modelo aditivo parcialmente lineal con un modelo plenamente noparamétrico usando algunos estadísticos de contraste recientemente desarrollados. Puesto que infererencia en regresión noparamétrica puede ser hecha de
varias maneras, lo más natural es usar la regresión noparamétrica como hipótesis
alternativa contra una hipótesis nula semiparametrica. Entonces, para propósito
de investigación nosotros chequeamos si un modelo PLM proporciona un razonable
ajuste a los datos usando diferentes métodos de reemuestreo para obtener valores
críticos calculados con bootstrap y submuestras de los mencionados estadísticos de
contraste.
En este capítulo, nosotros también tratamos un problema común el contexto
de las curvas de Engel, y es que el gasto total esta conjuntamente determinado
con el gasto en los diferentes bienes. Por ello existe una endogenidad potencial.
Para resolver este problema usamos regresores construidos como variables instrumentales, en adición a variables en otras bases de datos. En particular, nosotros
usamos el método desarrollado por Sperlich (2005) llamado regresores noparametricamente generados o construidos en dos pasos (NP2SCV). Nuestra sensación es que
ciertamente (NP2SCV) en combinación con modelos aditivos parcialmente lineales
ayudan a eliminar la endogeneidad en la estimación de las curvas de Engel.
En el Capítulo 3, "Locally Modelled Regression and Functional Data", nosotros
estamos interesados en extender los métodos noparametricos cuando los regresores
son funciones (i.e. una observación podría ser una curva, una superficie o cualquier
otro objeto perteneciente a un espacio de dimensión infinita).
Desde un punto
de vista estadístico, esto corresponde a una regresión funcional, porque deseamos
predecir un^ variable respuesta Y de una variable explicativa funcional X. Además,
9
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
solo condiciones de regularidad con impuestas al operador de regresión son asumidas.
Esto conduce entonces a un contexto noparamétrico. Así que la problemática de
este trabajo trata de la regresión funcional noparamétríca. Recientemente, varios
artículos tratan con la regresión funcional noparamétríca (ver por ejemplo Ferraty
and Vieu (2002, 2005)). Estos consiste esencialmente en la extensión de estimador
kernel Nadaraya(l964)-Watson(1964) a el caso de variable explicativa funcional. De
otro lado, ideas de regresión local han sido desarrolladas en el contexto de regresión
univariante y multivariante, ver Wand and Jones (1995). Por tanto, nuestro método
es una extensión, que es una combinación de los métodos de regresión locales con
las ideas actuales de variables funcionales. Así pues, la meta nos es fácil en cuanto
al estudio asintótico y la implementación de una más que natural generalización
del método lineal local multivariante. Por tanto, nos centramos en una más simple
y rápida aproximación local. Las propiedades asintóticas son establecidas y datos
funcionales ilustran el buen comportamiento de este método rápido de regresión
local.
10
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Ir al índice/Tornar a l'índex
REFERENCES
Ferraty, F and P. Vieu (2004). Nonparametric Models For Functional Data, with
Applications in Regression, Time Series Prediction and Curve Discrimination. NonParametric Statistics, 16, 1-2, 111-125.
Ferraty, F and P. Vieu (2006). Nonparametric Modelling for Functional Data Analysis. Theory and Practice. Springer, New York (In print).
Hardle, W and J.S Marrón (1990) Semiparametric Comparison of Regression Curves.
Annals of Statistics, 18, 63-89.
Hardle, W and J.S Marrón (1991) Bootstrap Simultaneous Bars For Nonparametric
Regression. Annals of Statistics, 19, 778-796.
Sperlich, S. (2005). A Note on Nonparametric Estimation with Constructed Variables and Generated Regressors. Working Paper. Universidad Carlos III.
Wand, M. P and M. C. Jones (1995) Kernel Smoothing. Monographs on Statistics
and Applied Probability, 60. Chapman & Hall.
Watson, G. S (1964) Smooth Regression Analysis. Sankhya Ser. A 26.
11
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based
Bootstrap Tests when t h e Null is
Nonpar ametric
1.1
Introduction
IN BOTH A P P L I E D AND MATHEMATICAL STATISTICS, N O N - AND S E M I P A R A M E T R I C
SPECIFICATION TESTING is still quite a popular research field. Any internet search
engine can find several hundred papers dealing with this topic even when looking
at the last five years only. Therefore, it is surprising that so few of them study
the problem of choosing an appropriate smoothing parameter, a problem that is
fundamental for the reasonable use of these methods. Unfortunately, for testing this
problem is not equivalent to the one in regression. It is well known that, at least
from a theoretical point of view, the optimal smoothing parameter for testing has
different rates from those which are optimal for estimation.
In the last couple of years there has been a growing amount of literature on
adaptive testing. In most cases, the adaptiveness refers to the smoothness of the
alternative and deals with the choice of smoothness parameter for the alternative, or
the test statistic, see e.g. Ledwina (1994), Spokoiny (1996,1998), Kallenberg & Ledwina (1995), Hardle et al (2001), Horowitz & Spokoiny (2001), Guerre & Lavergne
(2005). Even though these methods have so far had little direct impact in the sense
that we could not find published papers using these methods (in practice or in the12
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
ory), they have been useful in determining a better understanding of the problem.
However, to our knowledge, all these papers concentrate on testing problems where
the null hypothesis is fully parametric. It is not clear to what extend these methods help if the null hypothesis is semi- or nonparametric. This is not such a rare
situation, since additivity tests already belong to this family. When bootstrap is
used to determine the critical value, these tests entail at least one more parameter
choice problem: pre-estimating the model under the null hypothesis to later generate the bootstrap samples. This is necessary as in most cases the bandwidths
for the estimation and the bootstrap should have different rates, see e.g. Hardle &
Marrón (1990,1991). Although these authors have already mentioned the problem
of choosing an appropriate bandwidth, in practical applications this problem has
hardly been addressed. As a consequence, in most published procedures for testing or constructing confidence bands with a semi- or nonparametric null hypothesis,
there is no guarantee that the test holds the level, or the bands the nominal coverage
probability. This has recently been confirmed in the work of Dette et al. (2005)
and Rodríguez-Póo et al (2004). However, in the former it is not referred to as a
bandwidth problem but rather as a problem of correlated designs and dimensionality because the size distortion is much smaller for uncorrelated design. In the
latter paper the problem is avoided by using subsampling instead of bootstraps. It
should also be mentioned that in that simulation study, the authors face basically a
parametric bootstrap drawing the bootstrap errors from a distribution known up to
a certain parameter. Although that unknown parameter depends on nonparametric
nuisance parameters, knowledge of distribution greatly mitigates the impact of the
bandwidth on the critical value.
To study the problem outlined in more detail we concentrate on the problem
of testing additivity. We limit ourselves to test statistics proposed in Dette et al.
(2005) and Rodriguez-Poo et al. (2004) but we try different modifications, methods
of bandwidth choice, and subsampling. The aim is not to find the most efficient
additivity test or to propose new ones. Our focus is only directed at finding a
method that guarantees that the level will be held by non trivial power when the
13
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
null hypothesis and the resampling method are non- or semiparametric. So, after
a review of the additivity tests considered here, we study different procedures for
bandwidth choice. Unfortunately, we have not found a generally valid method. Our
conclusion is basically that further research is necessary.
The rest of the paper is organized as follows. In the next section we review the
estimation and testing procedures considered in this work. In Section 1.3 we discuss
the different, scenarios from which the practitioner has to make his choice, including
modifications of test statistics, and resampling methods. Section 1.4 summarizes
the main findings from our simulation results, and Section 1.5 concludes.
1.2
1.2.1
Statistical Methods: Estimators and Test Statistics
Estimators
We consider the following model:
Yi = m(Xi)+ui
'(1.1)
e Md xK i.i.d., m : Ud -» K the unknown function of interest,
with {{XtYl)}"=1
m[x) = E(Y\X
i = l,2:....n,
= x), and IÍ¿ i.i.d. random errors with E[u{] — 0 and finite variance.
The internalized Nadaraya-Watson estimator is defined as
n
-i
mk(x) = ] T vk{x, Xi)Yi, with vk{x, Xz) = (/ f c (X ; ))
where fk{Xj)
= ^ J^"=l Kk(Xj
Nadaraya-Watson, here Í fk{Xt)
Kk(x - X,)
(1.2)
— Xr) is a kernel density estimator (unlike standard
j
appears internally to the summation, see Jones
et.al (1994)), and Kfc(u) = \\da=l Kk (u) a product kernel with Kk{u) =
k^Kiuk"1).
Commonly, the kernel is assumed to be Lipschitz continuous with compact support
and / \K(x)\dx
< oo, / K{x)dx = 1. Furthermore, k is the bandwidth, assumed to
go to zero for sample size n going to infinity, but nk^ going to infinity. Let Vk be
the n xn matrix whose (j,i) element is vk{Xj,Xi)1
then rhk{x) = VkY, where Y
and mi; (•) are n x 1 vectors with rhk(Xj) and Yj is its jth entry respectively.
14
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
.Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
We are interested in the additive model, which we write in terms of
d
= x{; + J2m«
E (Y\X = x) = ms(x)
( x «) ,
(1-3)
a=l
where we set Exa {irLa(Xa)}
= J ma(x)fa(x)dx
= 0 Va for identification. Here,
ma, a = 1,. . ., d are the marginal impact functions for each regressor. Therefore,
^ is a constant equal to the unconditional expectation of Y.
ma(Xa)+m_a(X_a)
Xa, i.e. X^a
Writing m{X)
=
where X_a is the vector X of all explanatory variables without
= (Xii,...
,X¿(Q._i), Xi(a+i),...
,Xid) we can use the identification
condition directly to estimate ma. The so called marginal integration idea is based
on the fact that for xa fix we have
Ex-a [m {xa,X-a)\
= I m (xa, x_Q) /_„ (x_ Q ) dx^a = i< + ma (xa) .
Substituting for m(-) a nonparametric pre-estimator such as the one given in (1.2), a
sample average for the expectation, and for ip simply ip = - Y17-1 V* &ves (neglecting
the constant for a moment for the sake of simplicity):
n
fTla\%a)
/
t
l^ah
y^at
-A-ia) *i
j
¿=1
where
wh (xa, Xai) = Kh (xa - Xia) j - ^
Finally, we set rhs(Xj) = ip+^2a=1 rha(Xja)
l
sz£iL .
(1.4)
for each j = 1, 2,..., n. Note that defin-
ing Wh •= J2a=i Wah (xa) with Wah (xa) being the nxn
matrices with wah (Xj, X{)
as elements, one has rhs (x) — ip + Wh (x) Y. For more details see Dette el al (2005).
Some of the test statistics we will consider here are also introduced and discussed
there.
1.2.2
Test Statistics
As mentioned above, we do not introduce new testing procedures but rather study
two statistics which have already been studied in Dette et al (2005) together, with
15
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
other additivity tests, and which have turned out to perform best. We add a new
test statistic motivated by one that was introduced recently by Rodríguez-Poó et al
(2005), and which performed excellently in the study by Roca-Pardiñas & Sperlich
(2006). For more details on the test statistics readers are referred to these papers.
The null hypothesis of interest is Ho : rrc(-) = ms(-) versus Hi : m(-) ^
ms{-).
We consider the following two test statistics from Dette et al (2005) :
n
=
T2
=
(J m ( * i ) - m s ( * i ) ) M * í ) ,
nÍ /¿—
¿=i
1 "
nn
where é¿ = Y¿ — rhs(Xi),
Yi — m(Xi),
-S^ei{rh{Xi)-rns(Xl))w{Xl),
¿—•*
¿=1
i.e. the residuals under the null hypothesis, and ñ¿ =
the residuals without restrictions. Obviously, T\ calculates directly the
integrated squared difference between the null and alternative models. Alternatively,
T2 seeks to mitigate the bias problem inherited from the estimate m, which suffers
from the curse of dimensionality. In Dette et al (2005) it is proved that for all r¿,
the nkz (jj — /¿-) converge under the null to a normal variable with mean zero
and variances v\ for j — 1,2 with
= —^ / a2(x)w(x)dx
/ K2(x)dx
¡ix =
EH0{TI}
¡i2 =
EHo {r 2 } =-r-j a
<r2{x)w{x)dx K(0) + o
nk J
+
d
nk
and
v\
=
VarHo { n } = 2 I a4(x)w2(x)dx
/ (K * K ) 2 (x)dx ,
u|
-
Vor//,, {r 2 } = 2 / o-4(x)u;2(x)dx / K 2 (x)dx .
The statistic of Rodríguez-Poó et al (2005) is defined to mitigate the above
mentioned bias problem still further:
T3
2
K
lY, nk¿£ *(*-*;)to-^ra)
d
n
16
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
w(Xi) ,
(1.5)
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based .Bootstrap Tests when the Null is Nonparametric
where for ease of presentation and implementation K (•) is the same kernel function
as in the last subsection, and k again its bandwidth. It is straightforward to derive
from the above mentioned paper that nkz (T 3 — fi3) converges under the null to a
normal variable with mean zero and variance v\ for
¡d3 = EH0{TÍ}—
I (K * K) (x) dx I
a2(x)f2(x)w(x)dx
All tests have been proven to be consistent in the sense that under the alternative
they converge with n to infinity.
Finally let us mention that we have also studied other test statistics, e.g. those
given in Dette et al (2005) but not presented here. These, however, showed even
less satisfactory performance, so we have skipped them in our presentation.
1.3
Resampling and Choice of Parameters
As is well known, asymptotic expressions are of little help in practice, for calculating
the
exact critical value, for several reasons: bias and variance contain unknown expressions which have to be estimated nonparametrically, and the convergence rate
is quite slow for large d. For this reason it is common to use resampling methods
to approximate the critical value for the particular sample statistic. These can be
bootstrap methods or subsampling procedures. Unfortunately, unlike subsampling,
for the bootstrap it is not known how to choose the smoothing parameter in practice
for the pre-estimation of the model that is used to generate the bootstrap samples.
From theory it is known that one should somewhat oversmooth (see for instance
Hardle and Marrón (1991) and discussion below). For the choice of k (when estimating the alternative), some procedures are provided in the literature (see our brief
discussion of adaptive tests in the introduction). We will come back to this point
later in this section.
17
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
1.3.1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
Bootstrap
Tests
We give the general procedure first and then discuss some details:
1. With bandwidth ft-, calculate the estimate rhs under the null hypothesis of
additivity and its resulting residuals é¿, i = 1,.. ., n.
2. With bandwidth k, calculate the estimator m for the conditional expectation
without the additivity restriction, and the corresponding residuals ü¿, i =
1,.. ., n.
3. With the results from step 1 and 2 we can calculate our test statistics TI, T2,
and T3.
4. Repeat step 1 but now with a bandwidth hb which depends on h from step 1. We
call the outcome rhbs, respectively e¿ = Yi—rhbs(Xi), i = 1,. . . , n. Draw random
variables e* with E[(e*Y] = u\ (respectively e\ or e¿, see discussion below) for
j = 1,2,3 (respectively j = 1,2, see below again). Set Y* = rhbs(Xi) + e*,
i = 1 , . . . , n. Repeat this B times. This defines B different bootstrap samples
{{Xi,Y;fi)}Z=1,b=l,...,B.
5. For each bootstrap sample from step 4 calculate the test statistics r*' , j — 1, 2, 3,
b = 1,... ,B.
Then, for each test statistic r¿, j — 1,2,3, the critical value
is approximated by the corresponding quantiles of the distribution of the B
bootstrap analogues: F*(ü) = j¡ Ylb=i ^iT*j'
— ^ } - R-ecaH that they are
generated under the null hypothesis.
This procedure is well known, has proved to be consistent for many test statistics and has therefore been applied, certainly with slight modifications, to many
non- or semiparametric testing problems. However, several questions of practical
importance remain open: bandwidth choice h in step 1., bandwidth choice k in step
2., how to generate the bootstrap residuals e* in step 4. (see above), and how to
choose hb. Finally, how many bootstrap samples are necessary to get a reasonable
18
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
approximation of the distribution in step 5. In this paper we will discuss all these
questions except the last one.
1.3.2
The Choice of Bandwidths h
The problem of finding an optimal h is somewhat different from that of finding the
optimal smoothing parameter k which is directly linked to the optimal rate of the
test statistic. In that case it is clear that a theoretical optimal choice depends on the
optimal rate at which the test can detect a deviation from the null hypothesis. For
further details see the next subsection. In most cases, the estimator of the null model
can have faster convergence rates than that of the alternative, so the asymptotics
of the test statistics provide no theoretical guideline for an optimal choice of h. In
other words, we have to rely on practical issues.
As there are exist data adaptive methods for finding the optimal bandwidth k for
the alternative (compare next subsection) one could argue that h should be chosen
according to k. This way one could guarantee that the same smoothness is imposed
on the regression function regardless of whether it is estimated under the null hypothesis or not. However, it is not clear whether this is always wanted. Moreover,
we will see later that on the one hand the adaptive choice of k is computationally
intensive, and on the other hand /i¡, depends on h. For k one needs a grid search
which then has to be extended to the choice of h (as it then depends on k) and thus
to the choice of h¡,. Altogether we would get a procedure that is computationally
quite unattractive.
Intuitively, it seems to be desirable to look for a reasonable estimation of the null
model. This is only guaranteed with a reasonable bandwidth choice of h beforehand.
We therefore recommend cross validation or plug-in methods.
1.3.3
The Choice of Bandwidths k
It is known that a bandwidth k which is optimal for estimation is usually suboptimal
for testing. More specifically, for testing the optimal smoothing parameter has faster
convergence rates, i.e. we should undersmooth. As for regression, cross validation
19
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
bandwidths have a tendency to undersmooth in practice, and they are also quite
popular for nonparametric testing.
As an alternative, let us consider the adaptive testing approach introduced e.g.
in Spokoiny (1996,1998). It has been extended by Rodríguez-Poó et al (2004) to
nonparametric testing problems such as those we consider here. The method is the
same for each of our three test statistics, so we can skip the index j of Tj, j — 1,2, 3
in this subsection. Adapted to our problem it works as follows:
We consider simultaneously a family of tests {rfc, k 6 &}, where 8. — {fcj, ¿2,...., kp)
is a finite set of reasonable bandwidths. The theoretical maximal number P depends
on n but is of no practical relevance, for details see Horowitz & Spokoiny (2001).
Define
rk - Eo[rk}
Tmax
=
t
m a x
>
.
w h e r e
keK Varl/2[Tk]
EQ[] indicates the expectation under HQ. This studentizing under the null is only
to correct for the deviations in distribution caused by the different bandwidths k.
Therefore, instead of Varl^2[rk] we could take something proportional to it without
loosing consistency, as long as it corrects for the standard deviation caused by the
different k — k\,...,
kp.
A particularity of the bootstrap analogues of rmax is that one first needs to calculate the bootstrap statistics (rfc)*'6 for all k E 8. to afterwards get ( T max )*. 6 . Note
that for each k, the empirical moment of the bootstrap statistics (jk)*'b (average,
respectively standard deviation) can be used as a substitute for EQ [rh\, respectively
Var1//2[Tk], in practice. This is what we do in our simulation study.
1.3.4
T h e C h o i c e of B o o t s t r a p R e s i d u a l s
From a theoretical point of view, wild bootstrap errors should be drawn from the
residuals of the alternative model, i.e. t¿¿ should be used in Subsection 1.3.4 instead
of e¿ or é¿. It is clear that this should maximize the power as the variance of e¿ (and
é¿) can increase greatly with increasing distance between HQ and the true model.
Arguments in favor of using e¿ exist only under practical aspects: often the size
20
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
distortion in bootstrap tests is worse when using ui or é¿; when using adaptive
procedures as described in Subsection 1.3.3, then it is not that clear which of the
Ui to use or whether the t¿¿ should even be estimated independently of the fc-choice
for the test; at least in the study of Dette et al (2005) to which our study comes
closest, the power loss is negligible so the size argument is decisive.
We conclude so far that if no adaptive choice of k is made, it would be desirable
to use m as long as one can control for the size distortion.
The second question is what kind of distribution for generating the random errors
should be used. In step 4 of the bootstrap procedure described in Subsection 1.3.4 a
distribution is often taken that gives e* with E[(e*y] = ej for j = 1 up to 3 (or even
more). The so called golden-cut wild bootstrap is also quite popular, see e.g. Hardle
& Mammen (1993). More recently, in the context of size distortion of bootstrap tests,
Davidson & Flachaire (2001) argue that for problems with moderate sample size
the disadvantages of the higher-order-moment adapting bootstraps outweigh their
(asymptotic) advantages. We therefore compare different methods in our simulations
(see Section 1.4).
1.3.5
An Alternative: Subsampling
A more and more popular alternative to bootstrapping is the subsampling procedure, see Politis et al (1999). To date, as subsampling is commonly believed to
converge slower in practice than bootstrapping, it has been used almost exclusively
when the bootstrap fails, i.e. has been proven not to converge. See Neumeyer &
Sperlich (2006) as an example in a purely nonparametric testing context. However,
Rodríguez-Poó et al (2004) introduce subsampling in the context that we discuss
here, although the bootstrap is consistent, because of the size distortion their bootstrap test suffered from (until the sample size was huge). In both papers subsampling
works well. The former also studies the automatic choice of subsample size m (with
m < n) which turns out to work in their simulations. As this method might be
remodeled to serve as a procedure for finding hb, we briefly introduce subsampling
and the automatic choice of the subsample size m:
21
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kerne] Based Bootstrap Tests when the Null is Nonparametric
Let y = {{X.¿, Y¡) \i — 1,..,, n) be the original sample, and denoted by r (y) the
original statistic calculated from this sample, leaving aside index j = 1,2,3 for a
moment. To determine the critical values we need to approximate
Q{z)
= P (nVi?T Q>) < z\ .
(1.6)
Recall that under HQ this distribution converges to an iV(/¿,t>2), for ¡i and v see
Subsection 1.2.2. For finite sample size n, drawing B subsamples y¡, - each of size
m - we can approximate Q under HQ by
¿W
:==
B
1
QT,I{myñ^Tkm^m)<z)
•
(1.7)
¡>=1
Note that the awkward notation comes from the fact that we have to adjust all
bandwidths for the new sample size m. For example, imagine k = ko • n's for fco
being constant. Then, Tfcm is calculated like T but with bandwidth km — konsm'6.
Certainly, under the alternative Hi, not only nVk^T (y) but also m^/k^jkm
(ym)
converges to infinity. When demanding m/n —> 0 guarantees that ny/k^r (y) converges (much) faster to infinity than the subsample analogues. Then, Q underestimates the quantiles of Q which yields the rejection of HQThe problem here is to find a proper subsample size m. Actually, the optimal
m is a function of the level a. Again we apply resampling methods: Draw some
pseudo sequences y*>1, i = 1 , . . . , L of y of size n with the same distribution as JA
For the desired level a, test HQ : m(x) ~ ms{x) = rh{x) — rhs{x) the same way as
you want to test HQ : m(x) = m-s(x), i.e. applying your particular test statistic to
HQ and using subsampling. From the L repetitions you can determine the empirical
rejection level (estimated size) for your given a.
Now find an m such that this
empirical rejection level is ^ a. In practice, you choose from a grid of possible m
the one whose estimated rejection level for HQ is closest to a from below. Note that
HQ is always true up to an estimation error that should be almost the same as in your
original test. The only drawback of this procedure is the enormous computational
effort. For further details and examples see Politís et al (1999), Delgado et al (2001),
and Neumeyer & Sperlich (20P6).
22
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
1.3.6
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
T h e C h o i c e of B o o t s t r a p B a n d w i d t h hi
In general, for many test statistics one could repeat the arguments outlined in Hardle
& Marrón (1990,1991): For the mean of fhh(x) — m(x) under the conditional distribution of Yi,..., Yn\Xi, ...,Xn, respectively of rh*h(x) — rhhb(x) under the conditional
distribution of Yf, ...,Y*\X\, ...,Xn, we know from Rosenblatt (1969) that
EY\x{mh(x)-m{x))
Er(m-h{x)-mg(x))
where fj,(K) = J u2K(u)du.
«
h2^-m"{x)
«
h^^-m^x)
,
(1.8)
,
(1.9)
Obviously, we need that vnl'h (x) — m"(x) •—> 0. The
optimal bandwidth /i6 for estimating the second derivative is known to be much
larger (in rates) than the optimal h for estimating the function itself. We can even
give the optimal rate. For example, the optimal rate to estimate ras" is of the order
n - 1 ' 9 (instead of n~ 1//5 ), an observation we make use of in our simulation studies in
Section 1.4.
As will be seen once more in Section 1.4, the typical comment that /ib has to be
oversmoothing, is unhelpful in practice. We therefore try the following automatic
bandwidth choice: apply the same procedure used for the automatic choice of a
proper subsample size m (last subsection) to find an adequate hi, for a given level a.
This is what we explain in more detail and afterwards try in our simulation study.
1.4
Simulation Results
To study all the points listed in the last section, we perform a huge number of
simulations. We give here only a summary of them; for example, limiting the presentation to Tj, j = 1, 2, 3, one particular model, one specific (random) design, and
sample size n = 100.
The model considered is as follows: We consider the same data generating process
23
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
as Dette et al (2005). That is, we draw i.i.d. three dimensional explanatory variables
Xi ~ N(0, Ex)
/ 1 0.2 0.4 \
with Ex = 0 . 2 1 0.6
\ 0.4 0.6 1 /
and i.i.d. error terms e¿ ~ iV(0, al) to generate
Yi = Xhi + Xli + 2 sin(7rX3ii) + aX2,iX3s + eu
i=
l,...,n
with a = 0 to generate an additive separable model, or a = 2 for the alternative.
Recall that the target is a test for additivity. Unless otherwise indicated, we set
ae — 1. Dette et al (2005) show that for the rather unrealistic situation that if
Ex is the identity matrix (i.e. with an uncorrelated design), then the problem is
greatly simplified, whereas a (much) stronger correlated design than ours leads to
identification problems for moderate sample sizes.
All results in the tables are calculated from 250 replications using 200 bootstrap
samples (or subsamples respectively). For real data applications 200- bootstrap samples are certainly very few; but in our simulations the results differed little when we
increased the number to 500. We used the (multiplicative) quartic kernel throughout.
In all three test statistics we use the weighting function w(-) for different trimming: we cut the outer 10%, 5% or 0% of the sample, where "outer" refers to the
tails of the explanatory variables. This is done to get rid of the boundary effects in
the statistics. The tables only give results for 5% and 0% as the boundary effects
turned out not to be a major problem.
To further speed up our simulation studies, we first looked for an average cross
validation bandwidth k, which turned out to be kopt = 0.78. Then we did all our
simulations for the non-adaptive tests (compare Subsection 1.3.3) with kopt. This
was done not only for computational reasons but also because otherwise the size of
the tests would also depend on the randomness induced by the estimation of k. For
the adaptive test procedure, k ran over a grid of 10 bandwidths placed around kopt.
We verified that in most cases Tmax did not refer to the boundary, i.e. to kmin or
h
^rnax •
24
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Nul) is Nonparametric
As discussed above, the bandwidth choice problem is different for h. Here, the
parameter responsible for the size, /ifr, depends on both a (the level) and h. Altogether, it is no problem here that h is chosen by cross validation in each simulation
run as recommended in Subsection 1.3.2. For the internalized marginal integration
estimator, cross validation bandwidths were introduced by Kim et ai (1999). For
the nuisance directions X-a (see equation (1.4) in Section 1.2.1) we used h_ a = 6 • h
as recommended in Dette et al (2005) and Hengartner & Sperlich (2005).
We tried different bootstrap residuals (compare Subsection 1.3.4). Our simulations mainly seem to confirm the findings discussed above. Therefore, below we
report only results referring to e* = £¿e¿, where the e¿ are i.i.d., drawn either from
the golden-cut distribution
€i
f - ( \ / 5 + l)/2
~ \ (\/5 + l ) / 2
with probability
with probability
p = (>/5 + l ) / ( 2 v 5 )
1- p
or from the Gaussian normal N(0,1). However, we admit that it may be interesting
to try more, different automatic choice procedures for h¡,, in order to study again
what effect the choice of residuals taken has (ult ¿, or ¿ ; ).
Probably the most interesting and challenging point is the choice of h¡,. We first
give the results for several choices of /i¡, with different bootstrap generating methods,
/c-adaptive and non adaptive procedures. To have h¡, as a function of h, to take also
into account h/hb —* 0, and perhaps validate the rate n~ l//9 (motivated in Subsection
1.3.6) we set hb = /in 1 / 5 - 1 /" and try different tc < 9.
Table 1.1 shows the results for the non-adaptive golden-cut bootstrap test. These
results basically i) confirm the statements of Dette et al (2005) for our context;
and ii) show that the problem is not solved simply by different smoothing in the
pre-estimation. Undersmoothing, as generally stated from a theoretical point of
view, seems to go in the wrong direction. In particular, the hope that the results
of Rosenblatt (1969) (see equations (1.8) and (1.9)) might give us a hint or even
provide a rule of thumb for the choice of h^, is not confirmed here. T 3 , introduced
by Rodríguez-Poó et al (2004) clearly outperforms the others in this study (as it
does in the following).
25
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
The results for the fc-adaptive analogues, see Table 1.2, show hardly any improvement. In particular, the problem of choosing h^ or, in other words, the size
problem is only mitigated for r%.
Following to some extent the findings of Davidson & Flachaire (2001) we then
repeated these two studies but with the Gaussian bootstrap, see above. Though
there is some improvement in both, size and power, the results in Table 1.3 and 1.4
give us hope only for test statistic T%. Note that the observation that a slight undersmoothing is produces much better results than oversmoothing has not changed
over the four different trials.
Next, for comparison we also provide a small simulation study where the critical
values are approximated by subsampling, trying several subsample sizes m. The
results are given in Table 1.5 for non-adaptive tests, and in Table 1.6 for fc-adaptive
tests. We tried more sizes m for the non-adaptive test but got reasonable results
only for T 3 . In contrast, looking at the A;-adaptive versions, ryax,
T™ax seem to
work, too - though with a rather weak power. Table 1.6 unfortunately is misleading
concerning r™ax\ one needs a much smaller m to get reasonable results here. A small
simulation study evaluating the automatic choice of m seems to indicate that this
procedure might work and therefore should be tried for what is our main focus: the
automatic choice of hbTherefore, we adjusted the automatic choice of the subsample size to find an
adequate hb (see Subsection 1.3.6). This was done as follows, described here in
detail for r 3 . Let {Y*, £*}™=1 := 3^* be a member of the pseudo sequence introduced
in Subsection 1.3.5. Then, for testing HQ : m{x) — ms(x) = rh(x) — rhs(x) with
sample 3^*, an analogue to T3 would be
1
^ = IT,
2
-K^X'-XAiYj-msiXj)}
26
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
w{X¡) .
(1.10)
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
Other statistics are thinkable certainly, e.g.
2
- ¿ K , ( X t - X*){Y* ~ rhs(X;)} - ¥Lh{Xt - X3){Y0- - ms(X3)} w(X-
n /—1 nkd
but they should all be asymptotically equivalent to (1.10). The procedure was
performed with only L = 100 pseudo samples 3^*- As the results varied widely
we were forced either to enlarge L considerably or to reduce ae considerably. For
computational reasons we decided on the second option and repeated the study with
a e = 0.1.
Some results are summarized in Table 1.7. As can be seen, this time we emphasize
the possibility of undersmoothing much more. You first have to look at T\ to find
the K giving the rejection level closest to a = 5% from below. Here, this is always
K = 3. Note that this might also change depending on the trimming, a, sample size,
etc. It is important to understand that the lines of T^ can always be calculated, i.e.
without knowing the true data generating process. Therefore we call this method
fully automatic. Now look at the lines for T%, the test of interest. Obviously, K — 3
is indeed the best possible choice; it holds the level and has strongest power of
any K respecting the level. This could be taken as indicating that our suggestion
for selecting /ib works. Unfortunately, this method does not work that well for all
possible a; specifically, it becomes quite incorrect for a > 10%. We repeated this
study also for j \ and T-I- The results were always somewhat worse than for T3 so
they do not change our conclusion that this procedure seems to be an interesting
and promising approach but further research is necessary.
1.5
Conclusions
We discuss the choices of all "parameters" a practitioner has to use when facing a
kernel based specification test where the null hypothesis is non- or semiparametric.
We have set parameters in quotation marks because we refer here also to questions
such as how to generate bootstrap errors, etc. However, our main focus is the bootstrap and its size distortion in practice when the sample size is small or moderate.
27
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
These points are illustrated along the popular problem of additivity testing. Naturally, one looks for an optimal trade-off between controlling for size under the null
hypothesis HQ and maximizing power. Even though these problems have already
been discussed and studied in theory, as yet, it is unclear how to set these parameters in practice. We show that theory is not just unhelpful here; at present, a
reasonable application of tests of these kinds is questionable.
We try and compare many modifications that can be found in the literature
without finding any clue to an optimal - or even a reasonable - parameter choice.
While there are different suggestions for singular problems such us which residuals
to take for the bootstrap or an adaptive choice of k, combining them gives puzzling
results. Sometimes, in practice, combining these suggestions, the power goes down
where it should increase or size becomes less precise where it should come closer to
the level.
Altogether, we have recommend certain procedures for particular test statistics.
However, the main open question seems to be how to find an automatic choice of
lib- We suggest a new procedure, taken from subsampling theory, that seems to
be a good way to go. However, further research is necessary to provide reliable
procedures for the nonparametric testing problems considered here.
28
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
REFERENCES
Davidson, R. and Flachaire, E. (2001) The Wild Bootstrap, Tamed at Last, Working
Papers 1000, Queen's University, Department of Economics.
Delgado, M. A., Rodriguez-Poó, J. M. & Wolf, M. (2001). Subsampling Cube Root
Asymptotics with an Application to Manski's MSE. economics letters, 73, 241-250.
Dette, H., von Lieres und Wilkau, C , and Sperlich, S. (2005) A Comparison of Different Nonparametric Method for Inference on Additive Models. J. Nonparametric
Statistics, 17, 57-81.
Guerre, E. and Lavergne, P. (2005). Data-driven rate-optimal specification testing
in regression models. Annals of Statistics, 33(2), 840-870.
Hardle, W and J.S Marrón (1990) Semiparametric Comparison of Regression Curves.
Annals of Statistics, 18, 63-89.
Hardle, W and J.S Marrón (1991) Bootstrap Simultaneous Bars For Nonparametric
Regression. Annals of Statistics, 19, 778-796.
Hardle, W. and E. Mammen (1993) Comparing Nonparametric Versus Parametric
Regression Fits. Annals of Statistics, 21, 1926-1947.
Hardle, W., Sperlich, S., and Spokoiny, V. (2001) Structural tests in additive regression. J. Am. Statist. Assoc, 96, 1333-1347.
Hengartner, N.W. and Sperlich, S. (2005) Rate Optimal Estimation with the Integration Method in the Presence of Many Covariates. Journal of Multivariate Analysis,
95, 246-272.
Horowitz, J.L. and Spokoiny, V. (2001) An Adaptive, Rate-optimal Test of Parametric Mean-Regression Model Against A Nonparametric Alternative.
Econometrica,
69, No. 3, 599-631.
Jones, M., C , Davies, S.,J and B. U. Park. (1994) Versions of Kernel-Type Regression Estimators. Journal of the American Statistical Association, Vol 89, 825-832.
29
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
Kallenberg, W.C.M. and Ledwina, T. (1995), Consistency and Monte-Carlo simulatioins of a data driven version of smooth goodness-of-fit tests, Annals of Statistics,
23, 1594-1608.
Kim, W., Linton, O.B., and Hengartner, N. (1999) A computationally efficient oracle
estimator of additive nonparametric regression with bootstrap confidence intervals.
The J. of Computational and Graphical Statistics, 8, 278-297
Ledwina, T. (1994), "Data-driven version of Neyman's smooth test of fit," J. Amer.
Stat. Ass., 89, 1000-1005
Neumeyer, N. and S. Sperlich, S. (2006) Comparison of Separable Components in
Different Samples. Forthcoming in the Scandinavian Journal of Statistics
Politis, D.N., Romano, J.P., and Wolf, M. (1999) Sub sampling. Springer Series in
Statistics. Springer.
Roca-Pardiñas, J. and Sperlich, S. (2006) Testing the link when the index is semiparamtric - A comparison study. Working Paper Universidad de Vigo, Spain.
Rodriguez-Póo, J.M., Sperlich, S., and Vieu, P. (2004) And Adaptive Specification
Test For Semiparametric Models. Working Paper Carlos III de Madrid, Spain.
Rosenblatt, M. (1969) Conditional Probability Density and Regression estimators.
Multivariate Analysis II, 25-31.
Spokoiny, V. (1996) Adaptive hypothesis testing using wavelets. Annals of Statistics,
24, 2477-2498.
Spokoiny, V. (1998) Adaptive and spatially adaptive testing of a nonparametric
hypothesis. Math. Methods of Statist, 7, 245-273
30
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
Trimming
a{%)
K
0%
5
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
7
8
9
10
5%
5
10
under HQ a=0.0
T\
T2
r3
.000
.040
.068
.128
.176
.256
.024
.068
.120
.184
.272
.344
.012
.060
.108
.172
.284
.340
.040
.084
.168
.288
.364
.440
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.008
-008
.008
.012
.012
.012
.032
.024
.024
.024
.020
.020
.008
.008
.008
.012
.012
.012
.024
-020
.024
.024
.020
.020
under H}
a=2.0
T\
T2
T3
.000
.004
.012
.016
.028
.028
.004
.008
.020
.032
.036
.056
.016
.020
.028
.040
.064
.080
.024
.036
.044
.064
.076
.116
.032
.012
.012
.012
.012
.024
.060
.028
.020
.024
.028
.032
.052
.032
.028
.028
.032
.032
.112
.076
.052
.048
.052
.056
.248
.184
.184
.196
.228
.252
.448
.312
.292
.300
.304
.340
.248
.192
.184
.184
.228
.244
.448
.308
.284
.292
.308
.332
Table 1.1: Rejection levels of the three original test statistics with and without
trimming. Critical values are determined with golden-cut wild bootstrap, using
hb = hn1^"1^ for the pre-estimation.
31
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kerne] Based Bootstrap Tests when the Null is Nonparametric
Trimming
a(%)
K
0%
5
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
7
8
9
10
5%
5
10
under Ho a—0.0
T\
T2
r3
.004
.004
.000
.000
.000
.000
.016
.012
.008
.000
.000
.000
.008
.000
.000
.000
.000
.000
.020
.008
.004
.000
.000
.000
.004
.004
.000
.000
.000
.000
.012
.008
.004
.000
.000
.000
.004
.000
.000
.000
.000
.000
.012
.004
.000
.000
.000
.000
.028
.020
.012
.000
.000
.000
.076
.072
.056
.028
.008
.008
.016
.016
.008
.004
.004
.004
.080
.064
.040
.024
.008
.008
under Hi
T\
r2
.044
.064
.048
.036
.036
.032
.096
.140
.132
.104
.072
.064
.080
.068
.036
.016
.008
.008
.136
.120
.116
.100
.084
.056
.032
.056
.036
.032
.012
.008
.072
.120
.092
.052
.044
.036
.052
.024
.016
.012
.008
.004
.120
.096
.060
.036
.024
.016
a=2.0
r3
.176
.204
.204
.196
.196
.188
.316
.308
.296
.316
.296
.284
.196
.184
.188
.184
.200
.192
.328
.296
.296
.292
.284
.288
Table 1.2: Rejection levels of the three ¿-adaptive test statistics with and without
trimming. Critical values are determined with golden-cut wild bootstrap, using
h¡, — hn1^5"1^ for the pre-estimation.
32
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
Trimming
a(%)
K
0%
5
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
7
8
9
10
5%
5
10
under HQ a=0.0
T\
r2
r3
.004
.036
.080
.132
.188
.260
.020
.072
.116
.196
.276
.352
.012
.052
.116
.176
.268
.352
.028
.088
.164
.252
.380
.436
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.008
.012
.012
.012
.012
.012
.044
.044
.032
.028
.016
.020
.008
.012
.012
.012
.012
.012
.048
.032
.024
.020
.016
.020
under H]
a=2.0
T\
T2
r3
.000
.004
.008
.016
.028
.032
.012
.012
.020
.036
.044
.068
.008
.008
.028
.040
.064
.096
.036
.036
.048
.060
.092
.120
.036
.024
.012
.016
.012
.016
.064
.036
.024
.032
.032
.032
.080
.036
.036
.028
.028
.032
.172
.092
.072
.056
.048
.064
.340
.236
.216
.224
.240
.248
.560
.380
.336
.332
.344
.372
.324
.236
.212
.220
.244
.260
.556
.372
.332
.328
.340
.376
Table 1.3: Rejection levels of the three original test statistics with and without
trimming. Critical values are determined with Gaussian bootstrap, using hi, =
^ n i/5-i/K £ or t k e p r e-estimation.
33
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
Trimming
a (%)
K,
0%
5
4
5
6
7
8
9
4
5
6
7
8
9
4
5
6
10
5%
5
7
10
8
9
4
5
6
7
8
9
under i í 0 c1=0.0
Tl
r3
T2
.000
.000
.000
.000
.000
.000
.028
.020
.004
.004
.004
.000
.004
.000
.000
.000
.000
.000
.012
.004
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.008
.004
.000
.000
.000
.000
.004
.000
.000
.000
.000
.000
.012
.000
.000
.000
.000
.000
.036
.028
.020
.008
.008
.008
.096
.088
.056
.032
.024
.016
.024
.024
.012
.008
.008
.008
.096
.072
.048
.040
.020
.012
under Hi
T\
.048
.048
.052
.032
.024
.016
.124
.184
.172
.116
.092
.076
.064
.048
.032
.016
.016
.016
• 136
.124
.100
.072
.052
.040
T7
a=2.0
T3
.028 .220
.048 .204
.032 .216
.016 .200
.008 .204
.008 .200
.096 .364
.156 .340
.124 .328
.072 .324
.048 .296
.032 .304
.036 .220
.020 .200
.012 .196
.004 .200
.004 .200
.004 .204
.100 .368
.092 .332
.048 .300
.032 .312
.012 .292
.012 .296
Table 1.4: Rejection levels of the three fc-adaptive test statistics with and without
trimming. Critical values are determined with Gaussian bootstrap, using h>, =
hnl/5~1/K for the pre-estimation.
34
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 1
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
under Ho a =0.0
Trimming
0%
i
a{%)
m
5
50
40
50
40
50
40
50
40
10
5%
5
10
.000
.000
.004
.028
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.000
under Hi a =2.0
7-3
T\
T2
.000
.040
.020
.224
.000
.052
.028
.240
.004
-004
.004
,004
.000
.000
.000
.000
.000
.004
.004
.004
.000
.000
.000
.000
T
3
.028
.212
.248
.744
.032
.202
.232
.732
Table 1.5: Rejection levels of the three original test statistics with and without
trimming. Critical values are determined with subsampling, using subsamples of
sizes m.
under HQ a=0.0
under Hx a==2.0
Trimming
a(%)
m
T\
T2
r3
T¡
T2
0%
5
90
80
70
60
90
80
70
60
90
80
70
60
90
80
70
60
.000
.000
.056
.244
.000
.028
.208
.584
.000
.000
.008
.060
.000
.008
.048
.196
.000
.000
.088
.336
.000
.072
,328
.680
.000
.000
.016
.084
.000
.012
.096
.304
.000
.000
.000
.000
.000
.000
,000
.000
.000
.000
.000
.000
.000
.000
.000
.000
.140
.148
-156
.196
.192
.192
.276
.416
.080
.080
.076
,064
.128
.140
.132
.136
148
160
168
236
196
208
308
484
104
104
088
076
152
148
160
168
10
5%
5
10
000
000
000
000
000
000
000
000
000
000
000
000
000
000
000
000
Table 1.6: Rejection levels of the three ¿--adaptive test statistics with and without
trimming. Critical values are determined with subsampling, using subsamples of
sizes m.
35
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Ir al índice/Tornar a l'índex
Chapter ]
The Size Problem of Kernel Based Bootstrap Tests when the Null is Nonparametric
K
Trimming
#0
0%
(a = 0)
T
3
T3
5%
T3
Hi
( a = 2)
0%
5%
T
3
T3
1
2
3
4
5
6
7
.012
.680
.012
.676
.001
.972
.001
.968
.063
.392
.062
.380
.019
.932
.019
.936
.028
.032
.028
.024
.042
.632
.042
.620
030
012
030
012
022
380
023
368
.032
.012
.032
.012
.015
.272
.015
.260
.031
.012
.031
.012
.011
.260
.011
.252
.029
.016
.029
.020
.009
.264
.010
.264
Table 1.7: Rejection levels of T?, and T\ for a = 5%, with and without trimming,
using Gaussian bootstrap with hb = /in 1 / 5 - 1 /* for the pre-estimation.
36
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 2
Estimating and Testing An
Additive Partially Linear Model in
a System of Engel Curves
2.1
Introduction
T H E SPECIFICATION OF E N G E L CURVES IN EMPIRICAL MICROECONOMICS
has
been an important problem since the early studies of Working (1943) and Leser
(1963) and the well-known work of Deaton and Muellbauer (1980a), in which they
developed parametric structures such as the Almost Ideal and Translog demand
model. Many Microeconomic examples are provided in Deaton and Muellbauer
(1980b) in which a separable structure is convenient for analysis and important
for interpretability. However, there is increasing empirical evidence pointing to the
conclusion that a sort of nonlinearity is present in the specification of Engel curves.
An alternative way of investigating nonlinear effects is to model consumer behavior by means of semi- and nonparametric additive structures. Moreover, non and
semiparametrie regression provides an alternative to standard parametric regression,
allowing the data to determine the local shape of the conditional mean.
From an economic point of view there are many reasons why it is interesting to
recover a correct specification of Engel curves. Firstly, a correct specification allows
us to examine the nature of the effect of changes in indirect tax reforms. Secondly,
it is important to specify the response of consumers in the face of changes in total
37
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Mode! in a System of Engel Curves
income. Changes of this kind allow us to assess the impact on consumers' welfare.
Consumer demand has become a very important field for applying non and semiparametric methods. An interesting analysis of the cross-sectional behavior of consumers in the context of a fully nonparametric model can be found in Bierens and
Pott-Buter (1990). Papers which consider the implementation of semiparametric
methods in empirical analysis of consumer demand include Banks, Blundell and
Lewbel (1997) and Blundell, Duncan and Pendakur (1998). This latter paper is of
special interest because its analysis regression is based on semi- and nonparametric
specifications of Engel curves. It also tests Working-Leser and Piglog's null hypothesis against the well-known partial linear model in which budget expenditures are
linear in the log of total expenditure. In this paper we estimate the Engel curves
directly as in Lyssiotou, Pashardes and Stengos (2003) among others.
We estimate an additive partially linear model (PLM) in order to investigate
consumer behavior using individual household data drawn from the Spanish Expenditure Survey (SES) and use the result obtained from semiparametric analysis to
examine the modelling-of age, schooling and expenditure in a system of Engel curves.
The importance of using an additive PLM models lies in the fact that in the context
of this model the effects of expenditure, the age and schooling on consumer demand
can be investigated simultaneously in the semiparametric context 1 . There are several
ways to get estimations of nonparametric additive structure, and we mention only
the most important: smooth backfitting, series estimators and marginal integration.
In this paper we use internalized marginal integration to estimate nonparametric
components in the additive PLM mainly because at the present time there is no
applied or theoretical study on the testing procedure using smooth backfitting.
Most of the papers that investigate consumer behavior in a nonparametric context are focused on the appropriate way of modeling the form of the Engel curves.
Those focused on the unidimensional nonparametric effect of log total expenditure on
budget expenditures, taking in to account some parametric indexes to reflect demo1
Analysis of consumer behavior can he carried out with fully nonparametric models. However,
for sake of interpretability and implementation, additive models overcome the well-known problems
coming from multidimensional Nadaraya-Watson and Local Polynomial regression estimators.
38
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Mode! in a System of Engel Curves
graphic composition include Blundeli, Browning and Crawford (2003) and references
therein. In this paper we investigate consumer behavior in semi and -nonparametric
terms focused on the nonparametric effect of total expenditure the age and the
schooling. In this study, unless stated otherwise, the effect of age and schooling
refer to the age and schooling of the household head. There is evidence suggesting
that these have deeper effect than generally assumed in parametric demand analysis
(see Lyssiotou, Pashardes and Stengos (2001)). In fact, it is common practice to include the square of age and/or schooling as well as their higher terms in parametric
models to capture possible nonlinear effects.
Inference in nonparametric regression can take place in a number of ways. The
most natural is to use nonparametric regression as an alternative against a fully
parametric or semiparametric null hypothesis. With this in mind, we investigate
whether an additive PLM provides a reasonable adjustment to our data using different resampling schemes to obtain critical values of the test statistics. In this paper
we are interested in applying some recently developed test statistics which are very
popular in the literature about testing semiparametric hypotheses against nonparametric alternatives. These test statistics are in the spirit of Hardle and Mammen
(1993) and Gózalo and Linton (2001), among others. On the other hand there is a
growing interest in the so called adaptive testing methods, in which the test statistics are adaptive to the unknown smoothness of the alternative, see among others
Horowitz and Sponkoiny (2001) and Rodrigue2-Poo, Sperlich and Vieu (2005). in
this paper we adapt their ideas with some differences, where are considered kernel
smoother for our problem.
It should be remarked that a problem that we may well have to consider is the
endogeneity of regressors. Note that in the context of Engel curves total expenditure
may well be jointly determined with expenditure on different goods. The approach
used to solve this problem is instrumental variable estimation.
We remark two
recently developed procedures in the context of nonparametric regression to tackle
the problem of endogenous regressors. The so called nonparametric two step least
square (NP2SLS) due to Newey and Powell (2003), and the nonparametric two
39
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
step with generated regressors and constructed variables (NP2SCV) due to Sperlich
(2005). Newey and Powell (2003) 's approach is a cumbersome procedure involving
the choice of basis expansion in the first step. However, Sperlich's approach only
requires a non, semi or even parametric construction of regressors of interest in the
first step. Our feeling is that a generated variables approach in combination with
additive PLM can help us to overcome to some extent any possible endogeneity
problem and that is exactly the procedure implemented in this paper.
The contribution of this work can be summarized as follows. Firstly, we are the
first (to our knowledge) to carry out an exploratory analysis of consumer behavior
with data drawn from the Family Expenditure Survey for Spain using semiparametric models. Second, we apply recently developed methods to estimate, test (various model specifications) and correct for possible endogeneity of total expenditure.
Third, our estimations of the additive model are accompanied by a reasonable measurement of discrepancy between the fully nonparametric model and the additive
estimation. An adequate model check is necessary whenever estimations of additive
models are carried out (Dette, von Lieres and Sperlich (2004)). Additionally, our
measure of discrepancy adapts to the unknown smoothness of the non-parametric
model and this constitutes a novelty in empirical economics.
The rest of the paper is organized as follows. In Section 2 we provide some background to understand both the estimating and the testing procedures. In Section
3, we discuss the shape of Engel curves and report empirical results obtained from
the application of additive PLM. We also provide the results of testing the additive
specification as well as the linearity of each nonparametric component in additive
PLM regression. In Section 4 concludes.
2.2
Additive Partially Linear Model and Testing
Hypothesis
There are many fields of empirical economics in which explanatory variables and
their second power are included in regression analysis to capture nonlinear effects;
40
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
In order to estimate the functions ma (xQ) we first estimate the function m (x) with
a multidimensional local smoother and then integrate out the variables different
from Xa. This method can be applied to estimate all the components, and finally
the regression function m(-) is estimated by summing an estimator ifi of tp, so we
get that:
ms(X3) = 4, + ¿
for j=l,...,n.
¿
Kh [X3a - Xia)
filM-Yi
¡4}
The expression to get the estimation of each component rna (•) defined
in [4], is called the internalized marginal integration estimator (IMIE) because of
the joint density that appears under the summation sign. For a detailed explanation
see Dette, von Lieres and Sperlich (2004) and references therein. Note that IMIE
does not provide exactly the orthogonal projection onto the space of additive functions. In other words, the sum of the estimated nonparametric components does
not necessarily recover the complete conditional mean because the interaction terms
are excluded from the regression. So, it is very interesting to establish whether the
sum of additive components is the conditional mean. Therefore, it is necessary to
carry out a specification test. With this in mind, we are concerned with testing the
validity of the additive specification of the regression function m (x) in eq.[l]. Thus,
the null hypothesis to be tested can be formulated as:
HQ :m(x)=
ms (x)
[5]
against a general alternative that Ho is false. An adaptive test statistics is implemented by Horowitz and Spokoiny (2001) (among others) in the context of parametric models against nonparametric alternatives; and by Rodriguez-Poo, Sperlich
and Vieu (2005) in the context of semi and nonparametric against a nonparametric
alternative. However, it should be remarked that the first implementation in the
context of nonparametric additive separable models against a fully nonparametric
alternative adaptive test was by Barrientos and Sperlich (2005).
The first test statistic is defined as the square of the differences between the
semiparametric fit and the fully nonparametric estimator, extending the concept
42
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
introduced by Hardle and Mammen (1993). In order to test the validity of our
hypothesis, we also consider the test statistics introduced by Gózalo and Linton
(2001) and Rodriguez-Poo, Sperlich and Vieu (2005) defined as:
J
n z—
t=i
1 "
n ¿—'
t=i
n
n
12
w (Xi)
[8]
2=1
where é,: = Yx — rhs(Xi) are the residuals under the additive model and ú¿ = Yt —
rn[(Xi) denote the corresponding residuals of the unrestricted model. In this study
we use the well-known Nadaraya (l964)-Watson (1964) estimator for the unrestricted
model. These test statistics can be used not only in specification testing defined by
(5) but also to test the linearity of individual nonparametric components, see Hardle,
Huet Mammen and Sperlich (2004). More exactly, we can test the null hypothesis
HQ : ma (x a ) = 8xa for all a and for some 6
Now we discuss the procedure for computing the critical values. Note that our
idea is based on a combination of adaptive test statistics with both bootstrap 2 and
subsampling 3 schemes. For the former case see Horowitz and Spokoiny (2001) and
2
To obtain bootstrap critical values we consider the following steps. 1) To obtain the bandwidth
from cross-validation, hc». 2) To estimate rhs (as) = 4> + !C a eA ^ai^a)
-%) To use the bootstrap
scheme to get e* for each i = 1, ...,7i. 4) For each i = 1, ...n generate Y* = ms(Xl) + e ' : where e*
is sampled randomly and we use the data {Y*, Xt}^=1 to estimate J7i s (i) under Ho- 5) Repeat the
process 2-4 B times to obtain {T"hj and use these B values to construct the empirical bootstrap
distribution.
The bootstrap errors e* are generated by multiplying the original estimated residuals from the
semipaiametric model, e, = Y¡; - rhs {Xi): by a random variable with standard distribution. This
procedure provides exactly the same first and second moments for c\ and for s*.
3
In the subsampling case one takes all subsamples of size b from the original sample {Xt. Y¿}.
The problem in selectitig the subsample size b is similar to the problem in selecting the bandwidth
in nonparametric regression analysis: the assumptions on the param<3ier b to require that b/n —> 0
and b —-» oo as n —> oo. Unfortunately, such asymptotic conditions are no help m solving the
43
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
for the latter one see Rodriguez-Poo, Sperlich and Vieu (2005). It is remarkable
that using subsampling to get an estimator of the variance of the restricted errors
guarantees consistency under Hi. Having estimated semiparametric and nonparametric models, rhs (•) and m(-j respectively, we construct the origina! test statistics
denoted by Tjk- As the distribution of Tjk varies with k (the bandwidth for the
alternative) we define the standard test statistic denoted by
I9|
f* - M i
where /¿- and -0? are the estimated mean and variance of the test Tjk for j = 1,2,3
(where j denotes test statistics [6]-[8]). Then we compute the test statistics based
on the resampling data (boootstrap and subsampling data), denoted by:
J
T-k - A;
v*
[10]
This creates a family of test statistics {T^, k G Kn} where the choice of k makes the
difference between the null and global alternative hypotheses. In order to maximize
power we take the maximum of f*k over a finite set of bandwidth values Kn with
cardinality L. Then we define the final test statistics by means of:
** = %*£**
Where Kn = {k = a^n'^5
I 11 )
i = l , . . . , £ } , a {I) = [l + (cx {I - l)" 1 )] n" 1 / 5 and
cx — 7 (max (Xi) - min (Xi)) with 7 e (0,1).
The testing procedure rejects Ho if at least one of the k e Kn the original
test statistic is significantly larger than the bootstrap analogues. In Horowitz and
Spokoiny (2001) the estimators for variance and bias are asked to be consistent under
alternative hypothesis. Note that this is only necessary for efficiency; for consistency
of the test, it is sufficient for the difference between real variance and estimate to
block size choice problem in finite samples. Instead, it is possible to use an algorithm to estimate
a "good" subsample size. This method has been applied in practice in another contexts with
good results, see for instance Rodriguez-Poo. Sperlich and Vieu (2004) and Neumeyer and Sperlich
(2005).
44
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
be bounded. Nevertheless, Rodriguez -Poo, Sperlich and Vieu (2005) suggest using
a subsampling scheme in order to get a consistent estimator of variance under Hi
and thus to have optimal power. They also discuss size problems of bootstrap tests
when the null model is non or semiparametric and show that the subsampling based
analogue suffers less from this problem.
2.3
The Shape of Engel Curves and Specification
Testing
The most usual structure in consumer behavior analysis is the so-called WorkingLeser specification. In this model each expenditure expenditure is defined over the
logarithm of total expenditure. Thus the model has a simple structure given by:
Wl
= f{\nXt)+£j
[12]
where W{ is the budget expenditure, In Xi is the log total expenditure and el is an
error term satisfying E (ex\ \nXi) = 0. Empirical analysis using parametric specification in eq.[12] can be found in the literature on consumer behavior, see Deaton and
Muellbauer (1980a, 1980b). For empirical unidimensional nonparametric analysis
see Blundell, Browning and Crawford (2003) and references therein. Instead of a
Working-Leser specification we can assume that consumer demand could be modelled by means of an additive structure as in eq[2], such that:
Wi = ijj + mx (InXii) + m2(X2i)
+m3(X3i)
+ £i
i = l,...,n
[13]
where In Xit is the log total expenditure, X2i and X3i are the age and schooling and
€i is assumed to satisfy E (e^Xi) ~ 0. Consider the augmented model:
Wi = il} + Z$k + mi (In Xu) + m2 [X2r) + m 3 (X3l) + ez
i — 1, ...,n
[14]
where Z¿ is a set of discrete or continuous variables of dimension K,
P is a K x 1 vector of parameters and e, is assumed to satisfy E(et\Zl,
Xi) = 0.
The models given by [13] and [14] are motivated because they allow us to include
45
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
other regressors with nonlinear effects, and at the same time to reduce the curse of
dimensionality; which may be the main weakness of nonparametric techniques. To
estimate the model [14] we follow the treatment of Hengartner and Sperlich (2005).
There are many ways to get a Vn-consistent estimator of ¡5: we use Robinson's
(1988) method. Let $ be an estimator of/?. Eq.[14j can be written as:
w, = ^ + mi (In Xu) + m2 (X2l) + m 3 (X3t) + &
[15]
where w¿ = wi — Zifik and ^t = e¿ + Zi l/3k — /3k) is the new composite error term.
The intercept term ip can be v^-consistently estimated by ip = Y — ZTJ3 where Y
and X are the sample mean. As in eq.[14] we can apply to eq.[15] the procedure
described in Section 2 to obtain estimates of mi (lnXi), m,2 {X2) and 777.3 (X3).
Now we turn to the problem mentioned in the Introduction about constructing
regressors to overcome the endogeneity problem. For a detailed explanation, see
Sperlich (2005). Let x¿ be an unobservable or endogenous variable and let Xi be a
generated regressor4, it is then possible to write x,: = x + b (x) + a (x), where b(-) is
the bias term such that &(-) —> 0 as n —» 00 and a (•) is the variance term. In order
to obtain consistent estimates of density and conditional mean and thus construct
¿i, with the help of instruments or even with help from different data sets it is
possible to estimate the reduced regression form, semi-, non- or even parametrically
(first step) and then use it in the structural regression (second step), instead of the
original regressor.
The procedure can be described as follows. Let {W} be the set of exogenous
variables, including the log of total income. Note that we are worried about endogeneity due to jointly determination of total expenditure, lnXi, and expenditure on
different categories of goods (endogeneity due to simultaneity). Suppose that l n ^ i
is endogenous such that:
ln(X!) =g{W)
+U
[16]
'with bias and variance of order O (g 2 ) and O ( - ^ j )
Theorem 2 in Sperlich (2005)
46
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
in order to fulfil the assumptions of
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
where E (£\W) = E{U\W)
= 0 for W = (Z,X2iX3),
but £ ( £ | m X : ) ¿ 0. Putting
[15] and [16] together we get
UJ
= rf> +
mi
(g (W) + U) + m2 (X 2 ) + m3 (X3) + (
applying the modeling mi (g (W) + U) = mg (g (W)) + A {(/), what somehow means
in the end that you assume additive in the exogenous impact of the explanatory
variables what is possible as assuming [16]. The we get:
w =
u
ip + mg(g(W))
= i> + mg{g(W))
+ m2(X2)
+ m3{X3)
+ m2{X2)+m3{X3)
+ X{U)+£
+l
[17]
[18]
where E [A {U) \W] = 0, E \^\W\ = 0 and E [£\g {W), X2, X3) = 0. The expression
in eq[l8] is the model that we have estimated only with the additional burden of
a pre-estimation g (W) consistently. Note that this methodology certainly involve
less difficulties (and is faster) than Newey and Powell's (2003) approach.
Household expenditures typically display variation respect to demographic composition. Then, we can use additive specification to pool across household types.
However, Blundell et.al (2003) suggested modifications to take into account integrability conditions (integrability is related to the problem of recovering a consumer's
utility function from his demand functions). Note that in eq[14], the Z matrix represent a household composition variables for each household observation i. This
means that we imposed a restriction on the way in which demographics affect expenditures (if j index is referred to specific category of good then we are interested
in imposing the restriction Z¿ = Zij, that is demographic composition affects in the
same way the consumption of different goods). Thus, under stated restriction on Z
matrix, our empirical researching did not provide evidence of linearity of mj (•) in
our system (see Section 3 and Table 4).
Blundell et al.
(2003) agrees that an alternative specification that does not
impose restriction on the form of m¡ (In _X\) is a straightforward extension of additive
PLM: Wi = ^ + Ztak + m-j (In Xu - £ {Z[9)) + m2 {X2l) + m3 (X3t) in which £ {Z¡9)
47
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
is some known function5 of a finite set of parameters 9 (otherwise mj (•) might be
linear in In X\ whenever Slutsky symmetry conditions are satisfied).
2.3.1
D a t a U s e d in this Application
In our application we consider mainly four broad categories of goods, Food (including
alcohol and tobacco), Clothing (including shoes), Transport (personal and public)
and Leisure (recreational activities, publications and general teaching). We draw
data from the 1990-1991 Spanish Expenditure Survey (SES) and for the purposes
of our study we select only houses with three children or less. Total income, total
expenditure and expenditure categories are measured in pesetas (yearly) at constant
1983 prices. In order to preserve a degree of homogeneity in most of aspects we
use a subset of married (or cohabiting) couples of household in the Madrid regional
community. This leaves us with 757 observations, 12.4% comprising couples without
children, 20.02% couples with one child, 47% couples with two children and 20.03%
couples with three children. Table 1 gives brief descriptive statistics for the main
variables used in the empirical analysis. '
5
As they suggested (. (Z[9) can be interpreted as the log of a general equivalence scale for
household i.
48
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
Table 1. Descriptive statistics for budget expenditure data
Variables
Mean
Std.dev
Min
Max
Food expenditure
709216
348565
344776
3307304
2254260
Clothing expenditure
304160
335535
7200
Transport expenditure
413226
486898
3640
2426126
Leisure expenditure
231988
265513
999
2128000
Total Expenditure
3162401
1397284
1039319
9304396
14.87
0.429
13.85
16.04
2052240
2289599
282504
42000000
Log total income
14.37
0.50
12.5
17.5
HHAge
40.6
10.6
21
80
HH Schooling
5.1
2.47
1
10
HNAD
2.1
0.5
1
4
HHSEX
0.90
0
1
Child_0
0.124
0
1
Child_l
0.202
0
1
Child_2
0.470
0
1
Child 3
0.203
0
1
Log total Expenditure
Total income
2.3.2
S o m e P i c t u r e s of t h e E x p e n d i t u r e
Total E x p e n d i t u r e Relationship
expenditure-Log
In this section we present the estimated additive partially linear regression of the Engel curves for the four budget expenditures in our SES sample. Each figure presents
the estimated marginal effect together with 90% bootstrap pointwise confidence
bands (dashed lines). In all cases we present kernel regression for the quartic kernel
| | (1 — u2) I (\u\ < 1) where I (•) is the indicator function, using the leave-one-out
cross-validation method to automatic bandwidth choice, hcv = 0.72 in the direction
of interest and b = 6hcv in the nuisance direction as in Dette von Lieres and Sperlich
(2004). In order to estimate the parametric part of the model [13] we have used a
set of discrete variables such as number of adults, sex and dummies for number of
children; this kind of regressor traditionally enters into the regression function in
the parametric part.
As usually, it is assumed that income is partially correlated with expenditure and
49
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
we can suppose that it is not correlated with the errors in model [13], therefore log
total income is a natural instrument to the log total expenditure. Then, based on
generated regressor and constructed variable methods we adjust the estimations for
any possible endogeneity of log total expenditure with the existing data as described
in Section 3.1. The set of exogenous variables includes the log income and its power
(up to the fourth one), age and schooling6.
Figures 1-4 show the estimated marginal effect of log total expenditure, age and
schooling on budget expenditures, controlled parametrically by the sex, number of
adults in households and dummies for number of children and corrected for any
possible endogeneity. It is clear from the plots that the effect of total expenditure
on the different budget expenditures is nonlinear. We can see in the case of transport
and leisure expenditures that this effect is increasing and monotone, whereas in the
case of food and clothing it is also increasing, but less stable for different levels of
total expenditure.
Note that the effect of schooling on expenditure on different goods is nonlinear.
In the cases of leisure, clothing and even transport it is interesting to observe the
pronounced effect for values of schooling close to the average (at which point the
greatest expenditure expenditure is reached). Note that leisure and clothing are
necessary goods (but not basics like food), so this behavior could be related to low
returns on education (whenever there is a strong correlation between income and
schooling, such a relation is generally observed in practice), so that consumers prefer
to dedicate their budget to basic goods. We remark that food expenditure does not
include food outside the household. It might be assumed that head of the household
might take some meals (e.g lunch) outside the house. On the other hand, in the
case of transport expenditure, we note an increasing effect up to values close to
the average for schooling, but from that point onwards the expenditure becomes
stabilized.
Another possible explanation for the behavior of leisure expenditure with respect
''Estimations for the model given by [13] with no endogeneity correction are available from the
author on request.
54
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
to schooling, is that high levels of schooling in couples that have many children are
accompanied by high income levels, and more hours of work per week, so that they
have no time for leisure. This idea is not so absurd if we consider that more than
half of households (67.03%) have two or three children to support.
According to our results, in the households with the oldest heads there is a
tendency to spend less money than in the households with younger heads, this
effect is notable at least for a range of ages between 30 and 40. It is explained,
at least in part, because the households with the oldest heads have less children to
support. Unlike leisure and transport expenditures, in the cases of food and clothing
expenditures this decreasing effect is considerable but not dramatic. Note that we
include the number of children parametrically, so this explanation makes sense if we
keep other effects unchanged. However, except for food and clothing expenditures,
the estimated parameters have no major impact.
Another question to take in to account is that 90% of household heads in our
SES sample are men: from the sociological point of view they pay less attention
to fashion, so this may explain, partly, the decrease in spending on clothes for
household heads of 40 and over. In the case of leisure and transport, the effect by
ages is dramatically decreasing: of course older heads have less recreational activities
and spend less time outside the household, so the use of transportation (private and
public) diminishes with age.
In regard to variables included parametrically, we remark that the number of
adults has no effect on consumer demand; the estimated parameter in each regression
is not statistically significant. On the other hand, the effect of sex is important
and different depending on the expenditure considered (except for transport). The
results tell us that men spend less money on food than on clothing and leisure. Table
2 displays estimated parameter in the additive PLM.
If the model is chosen correctly, the results quantify the extent to which each
variable affects consumer behavior. Clearly, the findings of the estimated additive
PLM have to be checked: this can be done by considering the test statistics described
in Section 2.
55
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
Table 2. Estimation of linear part in APLM
Variable
Estimated parameter
t-Statistics
Food Expenditure
|
Constant
13.490
73.78
Sex HH
-0.384
-2.535
Number of Adults
0.063
0.614
One child
0.407
2.441
Two children
0.594
3.584
Three children
1.151
3.612
Clothing Expenditure
Constant
11.03
26.938
Sex HH
1.196
3.535
Numbers of adults
-0.305
-1.328
One child
-0.986
Two Children
-0.967
-2.564
Three Children
-2.525
-3.532
(
-2.606
Leisure Expenditure
Constant
|
10.369
j
30.295
Sex HH
0.710
2.507
Numbers of adults
0.202
1.050
One child
-0.234
-0.739
Two children
-0.001
-0.003
Three children
-0.752
-1.256
Transportation Expenditure
2.3.3
Constant
-0.279
33.964
Sex HH
-0.279
-0.924
Number of adults
-0.008
One child
0.592
1.787
Two children
0.692
2.099
Three children
0.854
1.347
|
-0.043
¡
Specification Testing
Table 3 reports the p-values for testing additivity adjusted for any possible endogeneity problem. Since the choice of bandwidth is a crucial point, especially for the
56
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
bootstrap needed for the test procedure, we present results for different smoothing
parameters gr € {0.75,0.85,0.95} r—1,2,3. In order to apply the procedure described in Section 2 we implement 500 bootstrap replications; and we use 100 subsamples each of 70% and 60% of the size of the original sample n for our subsampling
scheme. To estimate the model under alternative hypotheses (fully nonparametric
model) we define a set Kn (with cardinality ~L=10) of bandwidths A; in a range from
0.3 to 2.
Note that the percentage of rejection is not so large for leisure and transport
expenditures with the bootstrap scheme. However, this situation is partly corrected
with the subsampling scheme where the percentage of rejection is increases, especially in the case of leisure. On the other hand, we find that test statistics r, and
T 3 give us a strong evidence of additive separable specification. Similar results are
obtained with the subsampling scheme in the sense that we are able to reject the
null hypothesis for all test statistics for each expenditure categories. In summary,
the null hypothesis is not rejected for the household types considered for all test
statistics with both resampling schemes and for all bandwidths.
Table 3. Testing Additive Specification
Bootstrap
Band
Clothing
Food
T~2
^
.66
9i
.81
.65] .84
92
93 "~ .63
1
.85
7"3
.95
.99 ^ 9 4
.99
JEL.
~6TH
.66
.55
b2
r2
.93
Transport
T2J^ T3
.22^.99 | -13_[ -99^ .97 | .12 .95
24j_,99 -91 , ,14_[ ,99J .95 | .14 j .95
T2J_T3
.99
Food
Block
Tl
Leisure
Ti
T2
T 3 J T\
.15
.99
92
.25 .99 .89
Subsampling
Clothing
Leisure
.94
.15
.95
Transport
T ^ Tl j T 2 J T 3
?3
Ti j T 2
T 3 | Ty
T2 j
.88
.99
.99
.99
.99
.99
.94
1.0
.82
.13
.90
1.0
.99
.99
1.0
.92
.83
1.0
.54
.18 | 1.0
.95
Certainly, the results from Table 3 need to be interpreted carefully, since the
test is telling us that model is clearly separable. We do not know whether one of
the regressors in the nonparametric part has a linear effect. Note that the results
57
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Model in a System of Engel Curves
of testing additivity in Table 3 tell us that the model is additive separable in its
nonparametric part, but they tell us nothing about the linearity of each component.
In other words, it is possible to accept the nonparametric additive (separability) hypothesis even if one of those regressors has a linear effect on expenditure on different
goods. The computed p-values concerned with testing linearity of each nonparametric component from model [13] are shown in Table 4. For this testing hypothesis
procedure we use a bootstrap scheme for two bandwidth g\=l and g<z=l-2. Again,
we define a set Kn (with cardinality h—10) of bandwidths k in a range from 0.35
to 2.
Note that for the clothing expenditure we are able to reject linearity of schooling
at 10% for both bandwidths, and for the food expenditure we reject linearity of
schooling at 7.9% (7.6% for £2); in both cases with test statistic T-¡. For clothing
expenditure, similar results on linearity of schooling are obtained with r 2 . With test
r 3 the percentage of rejection of linearity of schooling decreases to 6%. For the food
expenditure, we are only able to reject linearity of age at 10% for both bandwidths.
In the rest of the cases, we reject the linear effect hypothesis of age, schooling and
expenditure at a < 5% for all test statistics and for all bandwidths.
Table 4. Testing Individual linearity
Age
Schooling
1
Expenditure |
T3
Tl
T2
M
.095
.10
.062
.060
.018
.016
.05
.05
0
.024 "1)791 .008
.020 .076 .008
.020
.020
0
0
0
0 1
0
01
.030
.030
0
0
0
0
0
0
0
0
.002
.004
.002
.002
0
0
0
0
0
•
ri
T2
Tz
T\
gi
g2
.018
.016
.014
,014
.034
.028
.10
.10
gi
g2
.10
.10
.002
.004
Leisure
gi
g2
.020
.010
0
0
.004
.004
.008
.008
Transp
gi
g2
0
0
.018
.026
0
0
.004
.004
Clothing
Food
0 I
1
0
Note that in general, linearity of age and schooling is rejected for every expenditure type. Moreover, for all test statistics and for all bandwidths the linear effect of
total expenditure on expenditures categories is strongly rejected. From Tables 3-4
58
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Est ¡mating and Testing An Additive Partially Linear Model in a System of Engel Curves
we conclude that the results are coherent with the shape of the curves estimated in
Figures 1-4. This gives us an idea of the robustness and reliability of our methods.
2.4
Conclusions and Future Research
This paper applies semiparametric additive PLM regression techniques for studying
the relationship between consumption and household characteristics based on the
Spanish Expenditure Survey. On the one hand, in the case of clothing and leisure,
the additive specification for nonparametric components is (weakly) supported for
test statistics based on errors of the additive PLM model and non,-semiparametric
estimators, with the bootstrap scheme. However, with Tj and r 3 test statistics we
are unable to reject the null hypothesis of additivity for different resampling schemes.
On the other hand, additive separable nonlinear effects are completely supported by
the results on specification testing. In general terms, there is no evidence to assert
that any linear effect of regressors of interest on the different expenditure categories
is observed in the subsample SES data used in this analysis. In conclusion, the
results from Tables 3-4 allow us to assert that the joint effect of total expenditure,
age and schooling on expenditures categories is nonlinear additive separable.
The general results obtained from the estimation and testing of Engel curves
show that modelling the effects of total expenditure on the different expenditure
types simultaneously with other regressors such as those included here certainly
deserves better treatment than usually found in one-dimensional semiparametric
analysis.
In particular we observe that households with younger heads tend to
behave differently from other households, and clearly this fact is not captured in an
Engel curve system in which only linear and quadratic age effects are included in the
empirical specification. Of course, it could be reasonable testing that nonparametric
effect of the age is quadratic rather than linear. However, we think that a perfect
quadratic specification maybe fitted only for household head in the 40 and 50 age
range in some cases and in the 35 and 55 age range in other ones. Note that
nonparametric estimation gives us the possibility for observing the effect of whole
59
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating ant! Testing An Additive Partially Linear Model in a System of Engei Curves
age range on expenditure commodities.
Note that in this paper we only take into account a partial household composition
(we only control for number of children, sex and number of adults). Therefore, a
reasonable extension of empirical analysis with additive PLM (simple additivity
does not allow such analysis) could be carried out by introducing more demographic
variation to obtain variety in behavior (more regions, labor market, temporal dummy
to capture price effects, etc.). Moreover, we could be interested in allowing Z%j vary
in any way vvith j and Stlusky symmetry, then would be necessary to impose a
function to get general equivalence scale in order to fulfill conditions of proposition
5 in Blundell et al (2003).
Another interesting point to investigate is whether changes in consumer preferences take place over time and then to make an extension to dynamic models.
One can take data from 1980 and 1990, for instance, and to make a comparison of
consumer behavior. This would be an interesting question for the future together
the inclusion of more categories of goods (health, furniture house, rent, etc).
60
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partiaiiy Linear Mode! in a System of Engei Curves
REFERENCES
Arevalo, R., Cardelus, M. T., and Ruiz-Casti])o, J. (1998) La Encuesta de Presupuestos Familiares de 1990-91. http://www.eco.uc3m.es/epf90-91.html.
Banks, J. Blundell, R., and Lewbel, A. (1997). Quadratic Engel Curves and Consumer Demand. The Review of Economics and Statistics, 79, No 4, 527-539.
Barrientos-Marin, J and S. Sperlich (2006). The Size Problem of Kernel Based
Bootstrap Test When the Null Is Nonparametric. Working in progress. University
of Alicante
Bierens, H and H. Pott-Buter (1990) Specification of Household Engel Curves by
Nonparametric Regression. Econometric Reviews, 9, 123-184.
Blundell, R., Duncan, A., and Pendakur, K. (1998). Seminparametric Estimation
and Consumer Demand. Journal of Applied Econometrics, 13, No 5, 435-461.
Blundell, Richard; Browning, Martin and Ian A. Crawford (2003). Nonparametric
Engel Curve and Revelead Preference. Econometrica, 71, No 1, 205-240.
Deaton, A and J. Muellbauer (1980a). An Almost Ideal Demand System. American
Economic Review, 70, 321-326.
Deaton, A and J. Muellbauer (1980b). Economic and Consumer Behavior. Cambridge University Press, Cambridge.
Dette Holger, C. Von Lieres and S. Sperlich (2004) A Comparison of Different Nonparametric Method for Inference on Additive Models. Nonparametric Statistics, 00,
1-25.
Gózalo, P. L. and O. B. Linton (2001) Testing Additivity in Generalized Nonparametric regression models with Estimated Parameters.
Journal of Econometrics,
104: 1-48.
Hardk, W. and E. Mammen (1993) Comparing Nonparametric Versus Parametric
Regression Fits. Annals of Statistics, 21, No. 4, 1926-1947.
61
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter Estimating and Testing An Additive Partially Linear Mode! in a System of Engel Curves
Hardle, W., Huet, S., Mammen, E., and Sperlich, S (2004) Semiparametric Additive
Indices for Binary Response and Generalized Additive Models. Econometric Theory,
20, 265-300.
Hardle, W., Miiller, M., Sperlich, S., and Axel Werwatz. Nonparametric and Semiparametric Models. Springer Series in Statistics. Springer-Verlag, 2004.
Hengartner, N and S. Sperlich (2005) Rate Optimal Estimation with the Integration
Method in the Presence of Many Covariates. Journal of Multivariate Analysis, 95,
Issue 2, 246-272.
Horowitz. J, L and V. Spokoiny (2002) An Adaptive, Rate-optimal Test of Parametric Mean-Regression Model Against A Nonparametric Alternative.
Econometrica,
69, No. 3, 599-631.
Leser, C. E. V (1963). Form of Engel Functions, 31, No 4, 694-703.
Linton, O. B., and J. P. Nielsen. (1995). A Kernel Method of Estimating Structured
Nonparametric regression Based on Marginal Integration. Biometrika, 82, 93-101.
Lyssiotou, P; Pashardes, P and Stengos, Thanasis (2001). Age Effects on Consumer
Demand: An Additive Partially Linear Regression Model, The Canadian Journal
of Economics, 35, No 1, 153-165.
Nadaraya, E. A. (1964). On Estimating Regression. Theory Probability Applied, 10.
Neumeyer, N and S. Sperlich (2005). Comparision of Separable Components in
Different Samples. Workin Paper, Universidad Carlos III.
Newey, W. K and J. Powell (2003) Instrumental Variables Estimation of Nonparametric Models. Econometrica, 71, 1565-1578,
Robinson, P (1988) Root N-Consistent Semiparametric Regression. Econometrica,
56, 931-54.
Rodriguez-Póo, J. M, S. Sperlich and P. Vieu (2005) And Adaptive Specification
Test For Semiparametric Models. Working Paper.
62
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Ir al índice/Tornar a l'índex
Chapter Bstimating and Testing An Additive Partially Linear Model in a System of Engel Curves
Sperlich, S. (2005). A Note on Nonparametric Estimation with Constructed Variables and Generated Regressors. Working Paper. Universidad Carlos III.
Stone, C J (1985). Additive Regression and Other Nonparametric Regression Models. Annals of Statistics 13: 689-705.
Stone, C J (1986). The Dimensionality Reduction Principle for Generalized Additive Models. Annals of Statistics 14, 592-506.
Watson, G. S (1964) Smooth Regression Analysis. Sankhya Ser. A 26.
Working, H. (1943) Statistical Laws of Family Expenditure. Journal of the American
Statistical Association, 38, 4-56.
63
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and
Functional Data
3.1
Introduction
NONPARAMETRIC STATISTICAL MODELS HAVE TAKEN AN IMPORTANT PLACE IN
STATISTICAL S C I E N C E .
At the same time, there is an increasing number of situation
coming from different fields of applied sciences in which the data are of functional,
nature (i.e. one observation can be a curve or a surface,...). A lot of data set
has been studied in the recent literature, dealing with chemometrics (see Frank
and Friedman (1993) ) and (Ferraty and Vieu (2003)), radar waveforms (see Hall
et.al (1991) and Dabo-Niang and Ferraty (2004)), biometrics (see Ramsay et.al
(1994) and Gasser et.al (1998)),...Many others examples can be found in (Ramsay
and Silverman (2002, 2005). Such kind of data are called functional data in the
sense that they come from observation of a functional variable. The combination
of the nonparametric models with the functional data leads to the problematic of
Nonparametric Functional Statistics which is a recent field of investigations (see
Ferraty and Vieu (2005) for both practical and theoretical aspects of this topics).
Several examples of Functional dataset could be found in Ferraty and Vieu
(2003), for instance, Electricity consumption data in U.S.A for time series prediction. We include another example about Inflation rate in Colombia from 1955 to
2004, provided by Colombia Central Bank (http://www.banrep.gov.co).
64
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
The idea is
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
Figure 3.1: 50 Continuous Time Series Curves
simple. Suppose that you have monthly Inflation rate during 50 years, then the set
of explanatory variables to be used in functional statistical methods is composed of
50 functional data which are 50 continuous time series curves, see Picture 3.1.
Now for introducing the main problematic of this paper, let us describe briefly a
functional data set which will illustration our purpose. Initially studied by (Borggaard
and Thodberg (1992)) this data set comes from a quality control problem in the food
industry an can be found in http://ltb.stat.cmu.edu/dataset/tecator.
It is concerns a
sample of finely chopped meat. For each piece of meat, one observes a spectrometric curve by means of the Tecator Infratec Food and Feed Analyzer Working in the
wavelength range 850-1050 nm by the near-infrared (NIR) transmission principle.
Figure 1 displays the 215 spectrometric curves X\,..., #235•
In addition, for each piece of meat, one knows the percentage of moisture (by
means of some chemical process), and one have at hand the 215 scalar responses
^li •••>^2i5- The spectrometric technology is generally faster and less expensive that
analytical chemical processes. Therefore, the spectrometric curves are more and
more used for identifying the component of any product instead of some chemical
65
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
Figure 3.2: The Spectrometric Curves
analytical processes. The aim is clear: can we predict the percentage of moisture in
a piece of meat from its spectrometric curve? In other words, the statistical question
is the following one: does a relationship between the A"s and Y's exists?.
From a statistical pint of view, this corresponds to a functional regression setting
because on wishes to predict a response Y from an explanatory functional variable
X. In addition, only regularity constraints will be assumed with respect to regression
operator (which model the relation between moisture and the spectrometric curves).
This leads us to the nonparametric context. So, the problematic of this work deal
with the nonparametric functional regression. On the one hand, some works dealing
with the nonparametric functional regression exist already in the recent literature
(see for instance Ferraty and Vieu (2002, 2005) for recent and deeper developments).
This nonparametric functional regression method is essentially based on an extension
of the well-known Nadaraya-Watson kernel regression estimator of the regression, see
Nadaraya (1964) and Watson (1964) to the case of functional explanatory variable.
On the other hand, local linear ideas have been developed in the regression context
66
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
for univariate and multivariate explanatory variable, see Wand and Jones (1995) for
an overview of this topic. This is a direct extension of the nonparametric regression
model which can be viewed as a local constant method.
An interesting investigation consist in proposing a locally modeling approach
when on regresses a scalar response on an explanatory functional variable. This is
the main aim of the paper. However the aim of this setting does not make easy
both the asymptotic study and the implementation of a natural generalization of
the multivariate local linear method. Therefore, one focuses on a simpler and faster
local approach. Section 2 described the position of our problem. Section 3 recalls
what is the multivariate local linear regression approach and introduces a functional
locally modeled regression method and a simpler a faster version. Some asymptotic
properties of the fast functional locally modeled regression are given in Section 4
whereas its behavior in practice is illustrated in Section 5 by means of spectrometric
dataset. Finally, the reader interested by theoretical developments will find detailed
proofs in Appendix.
3.2
Position of the Problem
This paper focuses on the nonparametric estimation of the regression operator defined by
Y = m(X) + e,
with
E[e\X} = 0.
where the explanatory variable X is valued in some infinite dimensional space Tí
and Y is scalar response. To this end we can use a functional kernel estimator (see
Ferraty and Vieu (2006) for a deep study), which is an extension to this functional
framework of the Nadaraya-Watson estimator. Based on n pairs (Xiy Yi)i=1
n
iden-
tically and independently distributed as (X,Y) the functional kernel estimator is
defined as follows:
.
M
_ Y^^K(h-^(X!Xz))Yt
67
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
where is a standard univariate kernel function, <5(-,-) locates one element of
TL with respect to another one, and h is the so-called bandwidth which plays the
role of a smoothing parameter. An interesting example for S (•, •) is to consider a
semimetric, which is an index of proximity well adapted to the spectrometric curves
as explained in Ferraty and Vieu (2002). Indeed as in the standard case (i.e. the
explanatory variable is a real r.v.), the kernel estimator rh (x)
ca
n be introduced as
the solution of the minimization problem
n
mm Y (Yi - mf K (/rM (x, *i))
¿=1
This way, rh can be called functional local constant regression estimator and
denoted by mpLC because it approximates locally the Y¿'s with a constant. One way
to define a new functional nonparametric regression estimator is to introduce it as
the solution of a more sophisticated minimization problem, which is the aim of the
next section.
3.3
Functional locally modeled regression
The main idea developed here is inspired by the local linear approach. Therefore
we recall briefly what is the guideline of the local linear regression in the finite
dimensional space case (i.e. a p-dimensional explanatory variable is considered).
Starting from the expression of the local linear estimator in the multivariate case, one
introduces extensions of the local regression able to take into account an explanatory
functional variable.
3.3.1
The p-dimensional case
The idea of local linear regression has been developed by Fan and Gibéis (1992)
and the minimax properties of the corresponding smoother has been stated by Fan
(1993). A good overview on this topic can be found in Wand and Jones (1995)
and Fan and Gibéis (1996), and we refer to García-Soidán et.al (2003), Hertgartner
68
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Locally Modelled Regression and Functional Data
Chapter 3
et.al (2002) and Cheng et.al (2002) for more recent advances. Here we focus on
the multivariate approach, that is given p > 1 (i.e. we consider a p-dimensional
explanatory variable X). Given n independent real random vectors {X¿,Yj}¿=1
n
distributed as {X, Y} and according to Rupper and Wand (1994), a simpler way to
introduce the multivariate local linear estimator vn-^n, of m — E ( Y | X ) at x 6 R p
consist in solving the following minimization problem (Vo) '•
min
Y
fa
- a - (b,Xi - x)p)
Kp (H~l (Xf - x))
where (•. •) is some inner product defined onto W, H is generally a diagonal matrix
containing p bandwidths (one per dimension) and Kp is a standard p-varíate kernel.
The multivariate local linear estimator of m (x) is given by a (x) which is the solution
for a to the minimization problem (T-'o)-
3.3.2
T h e i n f i n i t e - d i m e n s i o n a l case: t h e f u n c t i o n a l s e t t i n g
There are various options for extending the local linear idea when the explanatory
variable is valued into some infinite-dimensional space H. The firts one is a generalization of the minimization problem (Vo) to the functional setting. The second
one proposes an interesting simplified version in a fast computational way. Before
going on, let us note that, in our functional regression setting, the p-dimensional
explanatory variable X (valued in some infinite-dimensional space H), whereas the
scalar response Y unchanged.
The functional locally modeled regression
A natural way for extending (Vo) to the functional case leads us to the following
functional minimization problem (V\):
n
min
T(Yt-
a- ^ (^)^x)f
i=l
69
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
K
(h-ló(x,^))
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
where T c Tí, S(•, -) locates one element of Tí with respect to another one, K is
a standard univariate kernel and $ a known operator from T x H2 into R such
that V( <= H # (-,£,£) = 0. In particular, for the given operator ^{rp,x,x')
=
w
(i/>, x — X')-H here (•, -)n is some inner product defined onto Ti, it is easy to see (V-¡)
as a direct extension to the functional case of the local linear regression problem.
Note that 5 (-, •) is a signed index of proximity between two elements of Tí, which
means that <5(-, •) is not constrained to be metric or a semimetric and can take
negative values. From a statistical point of view, we consider that a + Í» (ip, -, v) is
a good candidate for modelling the regression operator around v. In other words,
one suppose that m ( ) can be well approximated by a + ^ (ip, , x ) and as direct
consequences of the properties of ^ , a is a good candidate for fitting m(x) since
^(tptX-.x)
~ 0. Therefore, the solution a(x) for a to the minimization problem
(Pj) defines what we call the functional locally modelled (FLM) estimator of . The
main difference and also the main difficulty when we compare (PQ) with (T7;) resides
in the fact that the minimization acts is acting over the functional space T'. One
way of overcoming this problem is to suppose T of finite dimensional (D) and thus
one can replace the unknown function xjj with its finite basis expansion
D
^(•) = ][> d ed(-)
d=l
In addition, if we assume the operator '$' linear with respect to its first argument,
that leads us to the following minimization problem (Vi):
Clearly, from a practical pint of view, this problem is easier to solve and one
get the following expression for the locally modelled estimator mFLM (x) of m (x)
1&(Y\X — x) (which is the solution for a):
m(x)FLM^u'}(Q'KQy1Q'KY
70
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
=
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
where O is the n x D matrix such that
1 *K*!,x)
1
*{eD,XuX)
*(eD,X2,X)
*(ei,A' 2 ! x)
1 #(ei,*n,x)
•••
*(eD>^n,x)
K = d i a g ( # ( / ^ (x, Xi)),..., if (h-l5 ( Xl *n))), Y ' = \YU .., Yn] and u', = [1,0,.., 0] e
R D + 1 . From a theoretical point of view, the statement of asymptotic properties remain an heavy challenge because the matrix expression of rh {X)FLM
ma
kes unusable
the current theoretical advances in the functional setting. Therefore, we prefer to
focus on a simpler version
TU^FM
which will allow us to exploit the knowledge of
the kernel type estimator in the functional regression setting (see Ferraty and Vieu
(2006) for more details and relevant and extensive bibliography on this topic). This
is the aim of the next section.
A fast functional locally modelled regression
Instead of studying {{V\) or (V2)) o n e focuses on the following minimization problem
n
^ n
¿
Ty(Yl-a~bP(Xl,x)fK{h'15(X,Xl))
(a.b)&E
where /?(-,•) is a known operator from H2 into R such that V£ e H ¡3 (•, £, £) = 0.
As 5 (-, •), P (-, -) locates also one element of H with respect another one which means
that /?(-,•) can take also negative values. The fast functional locally modelled
(FFLM) kernel estimator mpFLM
of m is the solution for a of problem (V3) and we
have
™(x)FFLM
= u 'i (Q'pKQey1
G'/JKY
where
1
0(XltX)
P(Xn,X)
71
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
One of the main advantage is that one get the explicit expression:
™
uti Er=3 Wirt
n
(X)FFLM
^
Wi U
E n¿=1 V2-ij=l
with
Wij
= P% {Pi - Pj) KiKj
where Kl = K [h~15 ( x , ^ ) ) and Pi = P (Xi,\)-
It is clear that this is a faster
way for computing a locally modelled kernel-type estimator in a functional approach.
It is suffices to replace YLd=i ^d^ (ed-, •, •) with b@ (Xit x) in order to see (P3) as a
particular case of of (V2)- From statistical point of view, our FFLM approach
assumes that a — 6/3 (-, x) is a good way of approximation m (x) around x- In other
words, a is a good approximation of m (\) and hence TTIFFLU should be a. good
estimator of m. In addition, the explicit expression of TTIFFLM allows us to state
asymptotic properties based on the current knowledge on nonparametric functional
data analysis (see Ferraty and Vieu (2006)). Finally, this kernel type estimator
appears as a weighted average since it can be rewritten as follow:
n
mpFML (x) = J ]
W Y
ii
with j — 1,..., n and
W5
E
n
TT-^n
1 = 1 LJÍ=\
Wt i j
To complete this section, let us remark that we could take P {•,•) = 6(-, •)• But
P and 6 do not plays similar role. Indeed, P refers to the local behavior of the
regression whereas 5 concerns the local waiting. Therefore, as we will see in Section
5, authorizing two ways for locating one element with respect to another one (i.e.
P ^ S) can allow fit better the data from a practical point of view, in that sense,
considering P =¡¿ 5 can lead us to a more adaptive (or flexible) method.
72
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
3.4
Locally Modelled Regression and Functional Data
FFLM kernel-type estimator: asymptotic behavior
Our goal is to study the pointwise asymptotic behavior of TUFFLM (X) °f the regression operator m (x) = E (Y\X = x)i X being a fixed element of 7i. The main results
state the almost complete convergence which implies the almost sure convergence.
Before giving the theorems, let us introduce some assumption and terminologies.
First of all, let us star with a crucial hypothesis concerning the distribution of the
functional r . v ^ ,
(HI) (px (ui,u2) := P^
< 5(X,x)
< «2) and \fu > 0, ^px(-u,u)
> 0
In the following we will use the simpler notation
<px{u)
=<px{-u,u)
A soon as 5(-,-) is a metric or, more generally, a semimetric, <p^(u) can be
interpreted as a ball of H centered at x and radius u. When u becomes smail,
the terminology "small ball probabilities" is commonly used, which is a field of the
probability theory intensively investigated, see for instance Li and Shao (2001) for
an overview on this topic in relation with Gaussian process. Actually, the function
ipx (u) (which is a direct extension of the small ball probability notion) plays a
similar role in the functional case as the density function in the finite dimensional
setting. Indeed, it is usual in the multivariate nonparametric setting to estimate a
quantity at a point for which one has all around number a number of observations
large enough.
A common way for assuming that is to assume that the density
function valued at this point is strictly positive. In the infinite dimensional setting,
there is not a reference measured as the Lebesgue one in the finite dimensional
context. However, one has to make a similar assumption without density notion.
This is the goal of Hypothesis H I which translates in the functional context the fact
that we have at hand sufficiently observation around x and hence it makes sense to
estimate the regression operator at this point.
73
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
As it is standard in the nonparametric modelling, one has to consider regularity
assumptions. The first one concerns the unknown regression operator m which will
be supposed to verify one of the following constsraints:
(H2 C ): m e j / . - W - ^ R ,
lim
/ ( X ') =
f(x)\
and
(H2L): me{f:H-+U,
C\d{XlX')\v}
3(7 € R + , X' € 7i, !/(*') - f(x)\ <
Clearly, the first hypothesis is a continuity-type constraint which will aiiow us
to get pointwise convergence. Moreover, as soon as one wishes to state rate of
convergence, one have to introduce more restrictive constraint, which is the role
played by the second Lipschitz-type hypothesis.
An another regularity-type constraints is also necessary in order to control the
shape of the local functional (3 which can be expressed as follows:
(H3) 30 < M < M2, VX' £ H, Mx \d(XlX')\
< l/?(x,x')¡ <
^2¡d(X,X%
Now we focus on assumptions concerning the kernel estimator mppLM,
Let us
first introduce assumption on the kernel function K:
(H4) K is a differentiable function K : R —> R+, such that its support is ( - 1 , lj
These kind of kernels contain the standard symmetrical kernels used in the literature (uniform, triangle, quadratic, Epanechnikov,...) Once this class of kernels
defined, one can propose additional hypothesis acting on the estimator (and also on
the distribution of the functional r.v. X)
(H5) h is a positive sequence such that:
lim h = 0 and lim
n—too
(H6) 3n0, V?i > n0> (H?)
h
^
/^ ^
og
yA = 0.
n—•ocn'Px\
(zh, h) £ (z2K (z)) dz > C > 0
¡BixJl) P («• X) dP («) = o (jB{xh)
f? (ti, x) dP (u))
74
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
'
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally .Modelled Regression and Functional Data
One ends the listing of hypothesis by focusing on the scalar response Y through
its successive conditional moments:
(H8) VA; = 2,... 0fc : x -•* E (Yk\X — x) is
a
continuous operator with respect
to*Hypothesis (H2)-(H5) and (H8) are standard in the nonparametric functional
regression setting and extends what is usually assumed in the classical p-dimensional
nonparametric literature (see Ferraty and Vieu (2006) for a large discussion). Hypothesis (H6) precise the behavior of the bandwidth h in relation with the small
ball probabilities and the kernel function K. The new hypothesis (H7) about local
behavior of the operator (3 which models the local shape of the regression. Actually,
if ¡3 = d this assumption means that the local expectation of ¡3 is small enough
with respect to its moment of second order. In order to fix the ideas and to see the
unrestrictive feature of (H7), note that if the r.r.v. (3 (X, x) admits a differentiable
density with respect to the Lebesgue measure then (H7) is satisfied.
Now, we are ready for giving the two main results. The first one states the
pointwise almost complete convergence whereas the second one precise the rate of
convergence.
Theorem 1. Under (Hi),
fH2c),
rnFFLM
T h e o r e m 2. Under (til),
{H3)-fH8),
(x) - m (x) = Oa.CQ (1)
(H.2^), (H3)-(H8),
rhpFLM (x) -
m
we have:
we have:
(x) = O {hv) + 0 Q ,
iogn
mpx (h)
As it is usual in nonparametric statistics, when we wish to get a more accurate
asymptotic result, we have to introduce more restrictive assumptions. Clearly, this
75
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
is the case here because Theorem 2 needs that the regression operator m satisfies
the Lipschitz-type consdition (instead of the continuity-type one for Theorem 1). In
the remainder of this section, we give a sketch of the proof of both theorems. Let
us first introduce the folloiving quantities
n
1
VA;
n
J J
n (n - 1) E (wn) j-f ¿-¡
and in such way that we have
with E(rñ 0 (x)) = 1- The proof of the aforementioned theorems is based on the
following descomposition
rhFFLM(x)-™(x)
=
r^{(^i(x)-E(m1(x)))-(m(x)-E(m1(x)))}
m0 [x)
——n '"oW - 1 )
m0{x)
and also on the following lemmas:
Lemma 1 Assume that ( H I ) , (H3)-(H5) are satisfied; and
(i) if (H2c) holds, we have:
m(x)-E(m,(x)) = 0(l)
(ii) if (H2 L ) holds, we have:
m(x)-E(m1(x))-0(n
Lemma 2 Suppose that assumptioms ( H I ) , (H2c), (H3)-(H7) are satisfied,
(i) we have
™o {x)"l
= O,
76
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
•mpx (h)
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
(li) in addition, if (H8) holds, we get:
m 1 ( x ) - E ( m 1 ( X ) ) = 0„.c '
'
^
rupx[h) J
Lemmas l-(i) and 2 lead us to statement of Theorem 1. Lemma l-(ii) in addition
with Lemma 2 allow us to get Theorem 2. Detailed proofs of these lemmas can be
found in the Appendix.
3.5
FFLM regression in action
The aim of this section is to give an idea on the behavior of the FFLM method
from a practical point of view. To do that, we consider the functional data set
(X*,Yi)i_1
2J5
where the vector X*' = (A¿ (Ai),..., A¿ (A10o)) is the ith discretized
spectrometric curves presented in the introduction ant Y¿ is the corresponding percentage of moisture. Before implementing our local regression and as most of statistical methods, we have to fix some quantities.
As a fist important practical aspect consist in fixing the local shape of the regression, that is the operator /?(-, •). To this end, let us consider the usual inner
product
{f,9)n = J f(t)g(t)dt
According to our experiment of such a spectrometric dataset, we propose to
define a family of local shapes as follow:
Where 0 is a fixed real-valued function and where x^
denotes the gth derivative
of the function x- ® is selected among the eigenfunctions of the covariates operator
^ ((xi
_
X (9) ) (#. Xj •. X2 ) ) • From practical point of view, we get a dicretized
version of this eigenfunctions by computing the eigenvectors of the empirical covariances operator
1
' ie£
77
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Locally Modelled Regression and Functional Data
Chapter 3
where £ is a subset of {1,.... 215} (£ will called learning sample). The idea of
choosing 9 through the spectral analysis of the covariance operator is driven by the
functional principal components analysis methodology (see
). Indeed, this is
a useful tool for exhibing functional directions which can reveal pertain information.
Here, for our current dataset we tried several parameters q and 6 and the best results
in terms of prediction are obtained for q = 1 and for the third eigenfunction (i.e.
the eigenfunction associated to the third larger eigenvalue).
A second crucial point concerns the choice for the operator 5(-, •) which allows
to locate a curve with respect to another one. According to our experiment of
spectrometric curves, as well adapted local tool is a semimetric based on the second
derivative
Some motivations for using this kind of the smoothing parameter can be found
in Ferraty and Vieu (2002, 2006). It is worth noting that considering also the second
derivative for ¡5 leads to worse MSE. So, allowing and index of proximity S to be
different of the local shape /? makes this FFLM method more flexible in the sense
that the fit is better.
The last point concerns to the choice of the smoothing parameter (i.e. the bandwidth h), which is very standard problem in nonparametric statistics. As described
in Chapter 7 of Ferraty and Vieu (2006), we did it by using Cross-Validation procedure over fcNN-type bandwidth.
In order to illustrate the pertinence of the FFLM method, one splits randomly
the initial sample into two subsamples.
The first one (Pdi, Yi)ieC
"Learning sample", allows to build the estimator (i.e. rnFFLM
usually called
(•)). the second
subsample (A¿, Yi)i&r usually called the "Testing sample" allows to predict percentage of moisture (i.e. rnFFLM
(Xi)ieT).
Of course £ and T are constructed such that
£ U T = {1, ...,215} and £ n T = <fi. Now, one way to evaluate the performance of
78
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
MEAN SQUARE
FLC
ERRORS
FFLM
AVERAGE
Figure 3.3: Distribution of the MSE' for each method
this method is to compute the mean square errors (MSE):
MSE
(A T) = rj-
£
{Yi ~ KFLM
(*)) 2
which is a usual index for evaluating the quality of fitting to the data. In order to
get more robust results, we constructed randomly 100 learnings and testing samples
\^
'
J
/i=l,...,100
which allows us to get 100 quantities MSE (£{s\T^)
for s = 1,...,100. In
addition, to show the good behavior of the FFLM method, we also implemented
the functional local constant regression estimate mFLC
(see Section 2) with the
similar bandwidth choice procedure. Figure 2, displays several box-and-whiskers
which summarizes the distribution of the mean square errors computed over the 100
experiments: the left one correspond to the FLC regression, the middle one gives
and idea of the performance of the FFLM method, and the right one correspond to
the "AVERAGE" method which predict moisture by using 0.5 {mpic + rnpFLM)79
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Locally Modelled Regression and Functional Data
Chapter 3
In this situation, the local modelled approach seems globally to fit better to
the data. Actually, looking at the AVERAGE method we see that, if we run both
functional nonparametric approaches, we improve the quality of fitting in terms of
MSE. In other words, the FFLM method has to be seen as an complementary
nonparametric tools of prediction, and not only as a competitive one with respect
to existing nonparametric functional method.
3.6
Conclusions
One can see that Fats-FLM method as a good alternative to the FLC approach. It
is worth noting that a low computational cost drive us to significant improvement
in terms of the quality of the prediction. This good behavior makes this functional
local modelled regression very attractive, at least for the kind of spectrometric data
set. as emphasized in the implementation, this is also complementary functional
prediction method in the sense that combined with the functional local constant
regression, the obtained mean square errors overpasses those coming from FLC and
Fats-FML.
In addition, this work offers very interesting perspectives of investigations. A
first direction concerns the statement of theoretical properties with respect to the
cross-validation methodology (see Benhenni, et.al 2006) for asymptotic dealing with
optimal local bandwidth choice for the functional local constant regression method).
A second track would consist in looking for a more pertinent functional direction (i.e.
9) by proposing an "optimal" linear combination of eigenfunctions. Of course, this
deserves a deeper investigations from both practical and theoretical point of view.
At a third time, one can focuses on the function locally modelled (FML) regression
explained in Section 3.2.2. Surely, the
computational cost will enlarge but we
can expect a significant improvement in therms of quality of prediction because it
would allow to construct a large family of operators (3 (-, •). At the same time, this
statistical functional method opens new theoretical challenges as soon as we wishes
to state asymptotic properties.
80
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
Finally, this works is a first step toward local statistical models taking into
account functional variables. This is also an encouragement to pursue further investigations in this topic.
81
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
Appendix
We start this section with a preliminary technical lemma. In the following C is
a generic constant (0 < C < oo).
L e m m a 3. Under ( H I ) , (H3)-(H6), we have:
(0 V(Jfc,0 GN* x N ' . E Í ^ I ^ l ' )
<Cti<fx(h)
(u) E (KiPl) > Ch2ipx (h)
Proof.
(i) Note that (H3) implies:
and because of kernel K is bounded on [—1,1] we get:
K^\^\lh~l<CK,t\5{Xl,x)\lh-ll[-l,í]{h-l\5{?¿ux)\)
and thus, we have
K*\f3,\lh-l)<apx(h)
which is the claimed result
(ii) By using (H3) we have:
E(K10l)>CE(6(X1,X)2K1
Moreover, we can write for
E K,
dU,X)
t2K{t)P~^{t)
2
h
•1
1 r rt
i L/-1
=
/ J
~-{u2K{u))+K{-l))du
du
K(-l)ipx(h)
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
'
dP*^
(t)
l[Uil]{t)dPd^{t)^u{u2K{u))
+
j - i
82
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
the last equation coming from the Fubini's theorem. In addition, it's easy to
check that:
i{x.X)
1 K 1 ) (i) dPS-
(t) <P(uh<5
(X, X) < h)
J -i
in order to write
Ki
K (-1) <px (h) + ^
h?
Lx
(uh, h) ~ {u2K (u))\ du
Finally, it comes:
h~2E {Ktfl)
> C<px (h) (K (-1) - ^ - ~ £
(px (uh, h) j - (u2K (u)f) du*j
It suffices to use (H6) for obtaining the desired lower bound, which ends the
proof of lemma 3-(M)
P r o o f of L e m m a 1.
On the one hand, we have
E(w 12 )
on the other hand, it hold
E (E (rm (x) \X2)) = —
1
rE (W12E (Y2\X2))
E(tüi 2 )
which allows us to write:
\m(x)E(m0(x))~E(m1(X))\
E(wl2)
<
sup
<{wn{m(x)
-m(X2)))
\m(x)-m(x')\
X£B(x,h)
Combining lemma 3-(n) with (H2c) in order to get lemma l-(i).
we use (H2¿) instead of (H2c), it is clear that
83
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
However, if
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
sup
\m(x) — m(x')\
— O (hv)
xeB(x,h)
which leads us to lemma 1-(M)
P r o o f of L e m m a 2.
(ii) Let us start the proof remarking that
1^
n2h2cpx
mi{x)
n(n
KJYA/I
(hy
s2
(1)
— l)E(u>i2)
n j ^ hyx {h)J yn ^
53
h<£x (/i)
¿4
which allows to write:
mi (x) - E (mi (x)) = Q [S& - E (S1S2) - (S3S4 - E (S3S4))}
At first, we have
SiS2-E(SiS2)
-
(5i-E(51))(52-E(52)) + (52-E(52))E(5i)
+(5i - E (5i))E (5 2 ) + E (5i) E (5 2 ) - E (S1S2)
(2)
It remains to study each term of expression (2):
(a) Let us write
s,
" E ( s , ) -;£i^£(S)—-h
In order to apply Corollary A.8-Ü in Ferraty and Vieu (2006), we focus on absolute moments of the r.r.v. ZU'E|Z£|
E\vAhTm{K3Y3-E{K3Y3))
=
J2 ck,m (KjYj)k
<
<Px (hym
(3)
(E (/i i y i )) m " f c ( - I ) " " "
\m—k
J2 ck,mE {K*ok (XJ) |E ( ^ m ( ^ ) ) |
fc=0
84
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
(4)
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Locally Modelled Regression and Functional Data
Chapter 3
the last inequality is obtained by conditioning on X\. In addittion, (H2C) implies
that m (XI) = m (x) + o (1) whereas we get ak (X\) = <7fc (x) + 1 as soon as (H8) is
checked. This combined with (4) allow to write
E|Z£|
=
O L^(/i)-
m
^ E « ) ( E ( ^ ) ) m—k
fc=0
=
O
max <px(h)
fc+1
\ K=0,...,m
= Ofaih)-"*1)
knowing that E (K^) = 0(<px(h))
(5)
(see Lemma 3-(i) with 1 = 0). Finally, it
suffices to apply Corollary A.8-Ü in Ferraty and Vieu (2006) with a2n = tpx (/i) _1 to
get
(b) In the same way, writing
is not difficult to see that:
m
E\Z%\ < ^ - ^ ( ^ — ^ c ^ E ^ f ) |E(/r3/3?)r~fc
(4)
fc=0
which, combined with Lemma 3-(i) when 1=2 implies that:
<\z*\ = o{<px(h)- m + l i
Once again, we apply Corollary A.8-Ü in Ferraty and Vieu (2006) with a.2
<Px (hy1
t0
get
B(a) =0
*-
7
~ tó> < '
85
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
_
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
[c] Note t h a t
E(Si)
=
ipxihr'EiKiYj
^x(hylE(Kim(Xl))
=
and because of mi (X\). — rn (x) + o (1) we get that E (Si) ~ O (1). Similarly,
we get that E(S,2) = 0(1).
E(SiS2).
Now it remains to study the quantityE (5"i) E (52) -
To this end, let us remark that
E(S1)E(S2)-E(S1S2)
n{n
=(l-
1)
n^
h'2<px
)
(h)-2E{K10l)B(K1Y1)
Using similar arguments outlined above, it is easy to see that
E (Si) E (S2) - E (SjSa) = O (n" 1 )
which is negligible with respect to
\/^~n^-
Finally, items (a), (b) and (c) states that
SiS2~E(S1S2)
= Oa.co
logn
(8)
n
^ V Vx (h) t
Similar arguments allows to conclude that
S3S4~E(S3S4)
= Oa.,
logn
xV
(9)
n<
Px (h) J
Now to achieve the proof, we have to study the upper bound of the quantity
Firstly, note that
h\E(pK1)\
(3(u,X)dP(u)
<Ch
B(xA)
and (H7) implies that
h\E(PK1)\
= o[ /
lB(x,h)
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
pz(u,X)dP(u)
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Locally Modelled Regression and Functional Data
Chapter 3
by applying Lemma 3-(i) with K — l[-i,i]> k = 1 and I = 2 we get that
/
p2(u,x)dP(u)<C'h2^(h)
JB(x,h)
which implies that
E(/31K1) = o(<px(h))
Now, Lemma 3-(M) and the last result allows us to write:
E (w12) = E (KiPfj E (Kij - E (KxPJ > Ch2^>x {h)
As a direct consequence we get that
Q =
o(i)
which ends the proof of Lemma 2-(ii).
(i) This result can be deduced directly from Lemma 2-(M) by taking Yi = 1. In
this case, Hypothesis (H8) is unnecessary.
87
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Chapter 3
Locally Modelled Regression and Functional Data
REFERENCES
Abraham, C , Cornillon, P., Matzner-Lober, E., N Molinari. (2003) Unsupervised
Curve Clustering Using B-spline. Scandinavian Journal of Statistics., 30, 581-595.
Benhenni, K., Ferraty, F., Rachdi, M., and P. Vieu. (2006). Nonparametric Regression for Functional Data: Optimal local Bandwidth Choice. Submitted.
Boente, C , and Fraiman, R. Kernel-based Functional Principal Components. Statistics and Probability Letters, 48, 335-345 (2000).
Borggaard, C and Thodberg, H. (1992). Optimal Minimal Neural Interpretation of
Spectra. Analytical Chemistry, 64, 545-551.
Cheng, M-Y and Hall, P. (2002). Error Dependent Smoothing Rules in Local Linear
Regression. Statistica Sinica, 12, 429-447.
Castro, P. E., Lawton, W. H., and Sylvestre, E. A. Principal Models of Variation
for Processes with Continuous Sample Curves. Technometrics, 28, 329-337.
Dabo-Niang, S., Ferraty, F., and Vieu, P. (2004) Nonparametric Unsupervised Classification of Satellite Wave Altimeter Form. In Proceedings in Computational Statistics, ed. J. Antoch, Physica-Verlag, Heidelberg, 878-886.
Dauxois, J., Pousee, A., Romain, Y. (1982) Asymptotic Theory for the Principipal
Component Analysis of a Random Vector function: Some application to Statistical
Inference. Journal of Multivariate Anlysis, 12, 136-154.
Fan, J (1993). Local Linear Regression Smoother and Their Minimax Efficiencies.
Annals of Statistics, 21, 1, 196-216.
Fan, J and Gijbels, I (1992). Variable Bandwidth and Local Linear Regression
Smoothers. Annals of Statistics, 20, 2008-2036.
Fan, J and Gijbels, I (1996). Local Polynomial Modelling and its Applications,
Chapman and Hall, London.
Ferraty, F and P. Vieu (2002) The Functional Nonparametric Model and Application
to Spectrometric Data. Comput. and Statistics, 17, 545-564
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Locally Modelled Regression and Functional Data
Chapter 3
Ferraty, F and P. Vieu (2003) Functional Nonparametric Statistics in Action. Quantification and Simulation
Processes, Humboldt-Universitat Zu Berlin. Discussion
Paper No 39.
Ferraty, F and P. Vieu (2004). Nonparametric Models For Functional Data, with
Applications in Regression, Time Series Prediction and Curve Discrimination. NonParametric Statistics, 16, 1-2, 111-125.
Ferraty, F and P. Vieu (2006). Nonparametric Modelling for Functional Data Analysis. Theory and Practice. Springer, New York (In print).
Frank, I, E., and Friedman, J, H. (1993). A Statistical View of Some Chemometric
Regression Tools (with discussion). Technometrics, 35, 109-148.
Gasser, T.,Hall, P and Presnell, B (1998). Nonparametric Estimation of the Mode
of a Distribution of Random Curves. J. R Statistics Society. Serie B, 60, 681-691.
García-Soidán, P., Gonzalez-Manteiga, W and Febrero-Bande, M (2003). Local
Linear Regression Estimation of the Variogram. Statist.Probab.Letters.,
64, 169-
179.
Hall, P., Poskitt, P and Presnell, D (2001). A Functional Data-Analytic Approach
to Signal Discrimination. Technometric, 35, 140-143.
Hertgarner, N., Wegkamp, M., and Matzner-Lober, E (2002) Bandwidth Selection
for Local Linear Regression Smoothers. Journal of the Royal Statistical Society.
Serie B Stat. Methodology., 64, 791-804.
Li, W. V., and Shao, Q. M. (2001). Gaussian Processes: Inequalyties, Small Ball
probabilities and Applications. In: C.R. Rao and D. Shanbhag (eds.)
Stochatics
processes, Theory and Methods. Handbook of Statistics, 19, North-Holland, Amsterdam.
Locantore, N., Marrón, J., Simpson,D., Tripoli, N., Zhang, J and Cohen, K (1999).
Robust Principal Componente Analysis for Functional Data (with discussion). Test,
8, 1-174.
89
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007
Some Practical Problems of Recent Nonparametric Procedures: Testing, Estimation and Application. Jorge Barrientos-Marín
Ir al índice/Tornar a l'índex
Chapter 3
Locally Modelled Regression and Functional Data
Muller, H-G (1987) Weighted Local Regression and Kernel Methods for Nonparametric Curve. Journal of the American Statistical Associations, 82, 397, 231-328.
Nadaraya, E. A. (1964). On Estimating Regression. Theory Probability Applied, 10.
Ramsay, J. O and C. J Dalzell (1991). Some Tools for Functional Data Analysis.
Journal of the Royal Statistics Society. Series B, 53, 3, 539-572.
Ramsay, J,. Altman, N., Bock, R (1994). Variation in Height Acceleration in the
Fels Growht Data. Canadian Journal of Statistics, 22, 89-102.
Ramsay, J. O and B. W. Silverman (2002). Applied Functional Data Anlysis: Methods and Case Studies. Springer-Verlang, New York.
Ramsay, J. O and B. W. Silverman (2005). Functional Data Analysis. 2nd Edition.
Springer-Verlang, New York.
Ruppert, D and M. P. Wand (1994) Multivariate Locally Weighted Least Squares
Regression. Annals of Statistics, 22, 3, 1346-1370.
Wand, M. P and M. C. Jones (1995) Kernel Smoothing. Monographs on Statistics
and Applied Probability, 60. Chapman & Hall.
Watson, G. S (1964) Smooth Regression Analysis. Sankhya Ser. A 26.
90
Tesis doctoral de la Universidad de Alicante. Tesi doctoral de la Universitat d'Alacant. 2007