Strongly Consistent Density Estimation of Regression Redidual

Abstract

Consider the regression problem with a response variable $Y$ and with a $d$-dimensional feature vector $X$. For the regression function $m(x) = \mathbb{E}\{Y|X = x\}$, this paper investigates methods for estimating the density of the residual $Y -m(X)$ from independent and identically distributed data. For heteroscedastic regression, we prove the strong universal (density-free) $L_1$-consistency of a recursive and a nonrecursive kernel density estimate based on a regression estimate.