Robust M Hybrid Estimation in Linear Regression Model Based on PCA – Ridge Estimator
DOI:
https://doi.org/10.33003/fjs-2026-1011-5350Keywords:
Linear Regression Model, Multicollinearity, Outliers, Biasing Parameter, PCA, M EstimatorAbstract
Two common data anomalies that significantly impair the performance of traditional regression estimators are multicollinearity and outliers in the predictor space. While a number of shrinkage and robust estimation strategies have been proposed separately, there is no much focus on creating estimators that can concurrently solve both issues inside a single framework. This study proposes a novel Robust M Principal Component – Ridge estimation approach that integrates robust M – estimation, principal component analysis (PCA) and ridge regression to enhance parameter estimation in contaminated regression models. Theoretical properties of the proposed estimator were derived through mean square error (MSE) analysis and compared with existing robust ridge, Liu, PCA and PCA – ridge estimators. A comprehensive Monte Carlo simulation experiment involving varying sample sizes, error variances, multicollinearity levels and 10% contamination (outliers) in the predictor space was conducted. Results indicate that the proposed estimators consistently achieved lower MSE values than competing estimators across most experimental settings. In particular, the MPCACK2MIN estimator emerged as the most efficient estimator, followed by MPCACK1 and MPCACK2. The proposed methodology provides a reliable alternative for regression modelling in the presence of simultaneous multicollinearity and predictor – space outliers.
References
Alabi, R. E., Alabi, O. O. and Ojo, O. O. (2025). Development of Hybrid Ridge–PCA Estimators for Addressing Multicollinearity in Gaussian Linear Regression Models. World Journal of Advanced Research and Reviews, 27(01), 942 – 957.
Alabi, R. E., Alabi, O. O. and Ojo, O. O. (2026). Performance of Proposed Ridge–PCA Estimators: Simulation Evidence and Real Data Applications, International Journal of Development Mathematics, 3(1), 135 – 149.
Alao, N. A., Ayinde, K. and Solomon G. S. (2019). A Comparative Study on Sensitivity of Multivariate Tests of Normality to Outliers. A SMSc J.,12, 65 – 71.
Arum, K. C. and Ugwuowo, F. I. (2022). Combining Principal Component and Robust Ridge Estimators in Linear Regression Model with Multicollinearity and Outlier. Concurrency Computat Pract Exper. 11,e6803.
Asar, Y., Erişoğlu, M. and Arashi, M. (2017). Developing a Restricted Two-Parameter Liu-Type Estimator: A Comparison of Restricted Estimators in the Binary Logistic Regression Model. Communications in Statistics - Theory and Methods, 46(14), 6864-6873.
Askin, G. R. and Montgomery, D. C. (1980). Augmented Robust Estimators. Technometrics, 22(3), 333-341.
Belsley, D. A. (1991). Conditioning Diagnostics: Collinearity and Weak Data in Regression. John Wiley and Sons.
Chand, S. and Kibria, B. M. G. (2024). A New Ridge Type Estimator and Its Performance for the Linear Regression Model: Simulation and Application. Hacettepe Journal of Mathematics and Statistics, 53(3), 837 – 850.
Croux, C. and Haesbroeck, G. (2000). Principal Component Analysis based on Robust Estimators of the Covariance or Correlation Matrix: Influence Functions and Efficiencies. Biometrika, 87(3), 603 – 618.
Enguraz, S. and Sakarya, N. (2020). Comparison of Robust PCA Methods. Communications in Statistics - Simulation and Computation, 49(10), 2715–2732.
Ertas, H. (2018). A Modified Ridge M-Estimator for Linear Regression Model with Multicollinearity and Outliers. Journal of Statistical Computation and Simulation, 88(12), 2267–2281.
Fayose, T. S. (2023). Development of Robust Weighted Ridge Estimators of Linear Regression Models, An Unpublished Ph.D. Thesis submitted to the Department of Statistics, Federal University of Technology Akure, Ondo State, Nigeria.
Fayose, T. S. and Ayinde, K. (2019). Different Forms Biasing Parameter for Generalized Ridge
Regression Estimator. International Journal of Computer Applications,181, 21–29.
Fayose, T. S., Ayinde, K. and Alabi, O. O. (2023a). M Robust Weighted Ridge Estimator in Linear Regression Model. African Scientific Reports, 2(123), 1 – 28.
Fayose, T. S., Ayinde, K., Alabi, O. O. and Bello, A. H. (2023b). Robust Weighted Ridge Regression Based on S – Estimator. African Scientific Reports, 2(126), 1–28.
Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. (1986). Robust Statistics: The Approach based on Influence Functions. John Wiley and Sons.
Hoerl, A. E. and Kennard, R. W. (1970). Ridge Regression: Biased Estimation for Non-Orthogonal Problems. Technometrics, 12(1), 55–67.
Hoerl, A. E. and Kennard, R.W. (1975). Ridge Regression: Iterative Estimation of the Biasing Parameter. Communications in Statistics - Theory and Methods, 4(1), 77 – 88.
Holland, P. W. (1973). Weighted Ridge Regression: Combining Ridge and Robust Regression Methods. Cambridge: MA: National Bureau of Economic Research. Doi:10.3386/w0011.
Huber, P. J. (1981). Robust Statistics. John Wiley and Sons.
Hubert, M., Rousseeuw, P. J. and Vanden Branden, K. (2005). ROBPCA: A New Approach to Robust Principal Component Analysis. Technometrics,47(1),64-79. https://doi.org/10.1198/004017004000000563
Hubert, M., Engelen, S. and Rousseeuw, P. J. (2021). ROBPCA: A New Approach to Robust Principal Component Analysis. Journal of the American Statistical Association, 116(533), 428–442.
Jegede, S., Lukman, A.F., Ayinde K. and Odeniyi, K.A (2022). Journal of the Nigerian Society
of Physical Sciences, 4(2): 251-264.
Jolliffe, I. T. (2002). Principal Component Analysis (2nd ed.). Springer.
Kibria, B. M. G. (2003). Performance of Some New Ridge Regression Estimators. Communications in Statistics - Simulation and Computation, 32(2), 419–435.
Kibria, B. M. G. and Lukman, A. F. (2020). A New Ridge-type Estimator for the Linear Regression Model: Simulations and Applications. Scientifica, 2020, 1–9.
Liu, K. (1993). A New Class of Biased Estimate in Linear Regression. Communications in Statistics – Theory and Methods. 22(2), 393–402,
Liu, K. (2003). Using Liu-type Estimator to combat Collinearity. Communications in Statistics - Theory and Methods, 32, 1009–1020.
Lukman, A. F., Osowole, O. I. and Ayinde, K. (2015). Two Stage Robust Ridge Method in a Linear Regression Model. Journal of Modern Applied Statistical Methods. 14(2), 53 – 67.
Lukman, A. F. and Ayinde, K. (2017). Review and Classifications of the Ridge Parameter Estimation Techniques. Haccetteppe Journal of Mathematics and Statistics, 46 (5), 953 – 967.
Lukman, A. F., Dawoud, I., Kibria, B. M. G., Algamal, Z. Y. and Aladeitan, B. (2021). A New Ridge-type Estimator for the Gamma Regression Model. Scientifica,1–8.
Lwando,D .,Chioneso,S.M and Lwando, O.K(2026). Performance Evaluation of the Robust Stein Estimator in the Presence of Multicollinearity and Outliers. Stat 2026,9(1),21; https://doi.org/ 10.3390/stats9010021
Maronna, R. A. (2011). Robust Ridge Regression for High-Dimensional Data. Technometrics, 53(1), 44–53.
Maronna, R. A., Martin, R. D., Yohai, V. J. and Salibián-Barrera, M. (2019). Robust Statistics: Theory and Methods (2nd ed.). John Wiley and Sons.
Midi, H. and Zahari, M. (2007). A Simulation Study on Ridge Regression Estimators in the Presence of Outliers and Multicollinearity. Journal Teknologi, 47, 59 – 74.
Montgomery, D. C., Peck, E. A. and Vining, G. G. (2021). Introduction to Linear Regression Analysis (6th ed.). John Wiley and Sons.
Naz, H. , Shah,I., Wasim,D and Ali, S. (2025). Robust Kibria Estimators for Mitigating Multicollinearity and Outliers in a Linear Regression Model. Stats 2025, 8, 119. https://doi.org/ 10.3390/stats8040119
Owolabi, A.T., Ayinde, K. and Alabi , O.O. (2022) A Modified Two Parameter Estimator and
Different Forms of Biasing Parameters in the Linear Regression Model. Nigerian
Society of Physical Sciences.
Özkale, M. R. (2009). A Stochastic Restricted Ridge Regression Estimator. Journal of Multivariate Analysis, 100(8), 1706–1716.
Özkale, M. R. and Kaçıranlar, S. (2007). The Restricted and Unrestricted Two-parameter Estimators. Communications in Statistics–Theory and Methods, 36(13), 2707–2725.
Rana, S., Rana, S. and Kumari, K. (2022). Robust Principal Component Regression using S– estimators for High-dimensional Data. Journal of Applied Statistics, 49(11), 2801 – 2820.
Rousseeuw, P. J. (1985). Multivariate Estimation with High Breakdown Point. In W. Grossmann, G. Pflug, I. Vincze, and W. Wertz (Eds.), Mathematical Statistics and Applications, B, 283 –297. D. Reidel.
Rousseeuw, P. J. and Leroy, A. M. (1987). Robust Regression and Outlier Detection. John Wiley and Sons.
Samkar, H. and Alpu, O. (2010). Ridge Regression based on Some Robust Estimators. Journal of Modern Applied Statistical Methods. 9(2), 17.
Silvapulle, M. J. (1991). Robust Ridge Regression based on an M-estimator. Australian Journal of Statistics, 33(3), 319–333.
Wasim, D., Suhail, M., Khan, S. A., Shabbir, M., Awwad, F. A., Ismail, E. A. A., Ahmad, H. and Ali, A. (2025). Quantile–based Robust Kibria–Lukman Estimator for Linear Regression Model to Combat Multicollinearity and Outliers: Real Life Applications using T20 Cricket Sports and Anthropometric Data. Kuwait Journal of Science, 52(1), 100336.
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2026 Remilekun Enitan Alabi, Olatayo Alabi, Oluwadare Ojo
This work is licensed under a Creative Commons Attribution 4.0 International License.
