Gaussian Process Prior And LASSO-Based Semiparametric Regression Model

Botao Zhang; Jingrong Zhou; Wujian Rao

doi:10.54097/zjz7y433

Authors

Botao Zhang
Jingrong Zhou
Wujian Rao

DOI:

https://doi.org/10.54097/zjz7y433

Keywords:

Curse of dimensionality, random effects, Gaussian process, semiparametric regression.

Abstract

The curse of dimensionality and autocorrelation effects are two common challenges encountered in regression models. In response to these issues, this paper proposes a semiparametric model based on L1 regularization and Gaussian process priors. On the one hand, the proposed method addresses the linear part using LASSO, which alleviates the curse of dimensionality and facilitates variable selection. On the other hand, it assigns a Gaussian process prior to the random effects, and by using different kernel functions, the model can handle autocorrelation effects in the response data while enhancing model flexibility. Moreover, the proposed method quantifies the uncertainty of the predictions. Simulation experiments demonstrate the high predictive performance of the method. Finally, the method shows promising results in two real-world data tasks: the HP Financial Inclusion Index and Economic Development Levels.

Downloads

Download data is not yet available.

References

[1] Giraud C. Introduction to high-dimensional statistics [M]. Chapman and Hall/CRC, 2021.

[2] Zhou Y, Kantz H. Effect of the uncertainty of autocorrelation estimation on trend analysis[J]. Physical Review Research, 2024, 6(4): 043160.

[3] Brownlee J. Data preparation for machine learning: data cleaning, feature selection, and data transforms in Python[M]. Machine Learning Mastery, 2020.

[4] Reddy G T, Reddy M P K, Lakshmanna K, et al. Analysis of dimensionality reduction techniques on big data[J]. IEEE Access, 2020, 8: 54776-54788.

[5] Teresa N, Hogg D W, Villar S. Dimensionality reduction, regularization, and generalization in overparameterized regressions[J]. SIAM Journal on Mathematics of Data Science, 2022, 4(1): 126-152.

[6] Li L, Xuan M, Lin Q, et al. An evolutionary multitasking algorithm with multiple filtering for high-dimensional feature selection[J]. IEEE Transactions on Evolutionary Computation, 2023, 27(4): 802-816.

[7] [7] Duan Shuyong, Yang Jianhua, Han Xu, et al. Adaptive Dimensionality Reduction Method for High Dimensional Data [J]. Chinese Journal of Mechanical Engineering, 2019,60(17):283-296.

[8] Filzmoser P, Nordhausen K. Robust linear regression for high‐dimensional data: An overview[J]. Wiley Interdisciplinary Reviews: Computational Statistics, 2021, 13(4): e1524.

[9] Bertsimas D, Van Parys B. Sparse high-dimensional regression[J]. The Annals of Statistics, 2020, 48(1): 300-323.

[10] Bunea F, Strimas-Mackey S, Wegkamp M. Interpolating predictors in high-dimensional factor regression[J]. Journal of Machine Learning Research, 2022, 23(10): 1-60.

[11] Mahmoudvand R. Two New Estimators for the Autocorrelation Function Through Singular Spectrum Analysis[J]. Fluctuation and Noise Letters, 2024: 2450026.

[12] Akpan E, Moffat I. Modeling the autocorrelated errors in time series regression: a generalized least squares approach[J]. Journal of advances in mathematics and computer science, 2018, 26(4): 1-15.

[13] Kumar S, Kumar R. Forecasting of municipal solid waste generation using non-linear autoregressive (NAR) neural models[J]. Waste Management, 2021, 121: 206-214.

[14] Silva I, Fleming C H, Noonan M J, et al. Autocorrelation‐informed home range estimation: A review and practical guide[J]. Methods in Ecology and Evolution, 2022, 13(3): 534-544.

[15] Salazar J J, Garland L, Ochoa J, et al. Fair train-test split in machine learning: Mitigating spatial autocorrelation for improved prediction accuracy[J]. Journal of Petroleum Science and Engineering, 2022, 209: 109885.

[16] Mohamed J. Time series modeling and forecasting of Somaliland consumer price index: a comparison of ARIMA and regression with ARIMA errors[J]. American Journal of Theoretical and Applied Statistics, 2020, 9(4): 143-153.