Rate of Convergence of the Density Estimation of Regression Residual

Abstract

Consider the regression problem with a response variable $Y$ and with a $d$-dimensional feature vector $X$. For the regression function $m(x) = \mathbb{E}\{Y|X = x\}$, this paper investigates methods for estimating the density of the residual $Y -m(X)$ from independent and identically distributed data. If the density is twice differentiable and has compact support then we bound the rate of convergence of the kernel density estimate. It turns out that for $d\leq3$ and for partitioning regression estimates, the regression estimation error has no influence in the rate of convergence of the density estimate.