Orthogonalization and thresholding method for a nonparametric regression problem

摘要

本研究では，データ数と同程度の数の基底関数の線形和による回帰の方法を提案した．本方法では，基底関数の出力からなるベクトルを直交化し，直交化されたベクトルの係数に対して閾値処理を行うことで，その上で疎な表現を得る．これを元の基底関数上での表現に変換したとき，直交化の実装の方法に応じて，元の基底関数上での疎な表現を得ることや縮小推定量を得ることができる．本研究では，加法雑書が正規分布に従うという仮定の下で，閾値処理における理論的に妥当な閾値を提案し，適当な条件の下で，提案法の与える汎化誤差の上界を解析した．さらに，その実装方法を示すとともに，簡単な数値実験において，提案法の有効性を確かめた．%For a nonparametric regression problem, we proposed a training scheme based on orthogonalization and thresholding, in which a machine is assumed to be a weighted sum of many fixed basis functions. In the basis of the scheme, vectors of basis function outputs are orthogonalized and coefficients of the orthogonalized vectors are estimated instead of weights. The coefficients are set to zero if those are less than predetermined threshold levels which are assigned componentwisely to each coefficient. We then obtain a resulting weight vector by transforming the thresholded coefficients. In this article, we presented theoretically reasonable threshold levels under an assump­tion of Gaussian additive noise. For a simple situation, we also gave an upper bound of a generalization error of the training scheme. Further, we gave implementations of the training scheme and showed that performances of the proposed training methods on some datasets are comparable to those of alternative methods.