ESTIMATING NONLINEAR REGRESSION PARAMETERS USING PARTICLE SWARM OPTIMIZATION AND GENETIC ALGORITHM
Abstract
Obtaining parameter estimates for nonlinear regression model using gauss-newton and gradient-based methods present some complex analytical challenges. In this paper we investigated the effectiveness and simplicity of Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) on five nonlinear regression models with varying level of complexities. We developed function in R-programming for each models and performed 30 independent runs for at least 100 iterations for both PSO and GA. We evaluated PSO and GA performance in view of computation time, residual error produced and compared our results with values published online. Based on the results obtained, PSO significantly outperform GA in view of computation time and quality of parameter estimates. Even so, GA required fewer iterations and produced fairly accurate results. Further investigation shows that PSO and GA are both competitive, effective, simple to implement, and can be considered reliable for obtaining the parameter estimates of different nonlinear regression tasks.
References
Adjad H., Baba YF., Mers A. A., Merron O., Bouatern A., Boutmmachte N. (2019). Particle swarm optimization for optimal-geometric optimization of linear Fresnel solar concentrations. Renewable Energy, 130, 992-1001.
Ajay S. & Ausif M. (2016). Improving Genetic Algorithm with fine-tuned Crossover and Scaled Architecture. Journal of Mathematics, 2016.
Wikipedia (2021). Automatic differentiation. Wikipedia, Retrieved September 4, 2022 from https://www.wikipedia.org/wiki/Automatic_differentiation.
Bates, D. M. & D. G. Watts. (2007). Nonlinear Regression and its Applications. John Wiley and Sons, New York.
Bulent, A., & Alptekin E. (2004). The genetic algorithm method for estimation in nonlinear regression. G.U Journal of Science 17(2), 43-51.
de Almeida, B. S. G., & Leite, V. C. (2019). Particle swarm optimization: A powerful technique for solving engineering problems. Swarm intelligence-recent advances, new perspectives and applications, 1-21.
Chandrashaker R. B., Venkat Prasad, Reddy P., & Rajeshwari M., Kavya Y. Sai (2017). Correlation of GA and PSO for Analysis of Efficient optimization. International Journal of Advance Research and Development, 2(4).
Friedl, G., & Kuczmann, M. (2014). Population and gradient based optimization techniques, a theoretical overview. Acta Technica Jaurinensis, 7(4), 378-387.
Holland J. (1975) Adaptation in Natural and Artificial Systems, The University of Michigan Press, Ann Arbor, Mich, USA, 1975.
Hsin-Hsiung Huang. (2010). Nonlinear Regression Analysis. International Encyclopedia of Education. Oxford: Elsevier.
Ghosh, M., Guha, R., Alam, I., Lohariwal, P., Jalan, D. & Sarkar, R. (2020). Binary Genetic Swarm Optimization: A Combination of GA and PSO for Feature Selection. Journal of Intelligent Systems, 29(1), 1598-1610.
Gianfranco C. & Andrea M. (2020). Metaheuristic Optimization of Power and Energy Systems: Underlying Principles and Main Issues of the ‘Rush to Heuristics’. Engergies, 13(19):5097.
Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley. Reading, MA.
Kapanoglu M., Koc, I.O., and Erdogmus, S., (2007). Genetic algorithms in parameter estimation for nonlinear regression models: an experimental approach, Journal of Statistical Computation and Simulation, 77(10): 851-867.
Kennedy, J., & Eberhart, R. (1995, November). Particle swarm optimization. In Proceedings of ICNN'95-international conference on neural networks (Vol. 4, pp. 1942-1948). IEEE.
Liu R. (2014). A Particle Swarm Optimization Based Simultaneous Learning Framework for Clustering and Classification. Pattern Recognition, 47, 2143 - 2152.
Madsen K., Nielsen H.B., Tingleff O. (2004). Methods for non-linear least squares problems. 2nd Edition Informatics and Mathematical Modelling Technical University of Denmark.
Malekan M and Khosravi A. (2018). Investigation of convective heat transfer of ferro fluid using CFD simulation and adaptive neuro-fuzzy inference system optimized with particle swarm optimization. Powder Technology, 333, 364-376.
Malik, H., Iqbal, A., Joshi, P., Agrawal, S., & Bakhsh, F. I. (Eds.). (2021). Metaheuristic and evolutionary computation: algorithms and applications (pp. 46-61). Springer Nature Singapore.
Khanduja, N., & Bhushan, B. (2021). Recent advances and application of metaheuristic algorithms: A survey (2014–2020). Metaheuristic and Evolutionary Computation: Algorithms and Applications, 207-228.
Pakize E., & Simge E. (2016). Nonlinear regression using Particle swarm optimization and Genetic algorithm. International Journal of Computer Applications. 153(6), 28 - 36.
Rajakumar, R., Dhavachelvan, P., & Vengattaraman, T. (2016, October). A survey on nature inspired meta-heuristic algorithms with its domain specifications. In 2016 international conference on communication and electronics systems (ICCES) (pp. 1-6). IEEE.
Desale, S.A., Rasool, A., Andhale, S., & Rane, P.V. (2015). Heuristic and Meta-Heuristic Algorithms and Their Relevance to the Real World: A Survey. International journal of computer engineering in research trends, 351. 2349-7084.
Satish K. Gupta. An overview of Genetic algorithms: A structural Anlaysis. (2021). International Journal of Innovative Science and Research Technology. 6(5), 1305-1309.
Sengupta, S., Basak, S., & Peters, R. A. (2018). Particle Swarm Optimization: A survey of historical and recent developments with hybridization perspectives. Machine Learning and Knowledge Extraction, 1(1), 157-191.
Archontoulis, S. V., & Miguez, F. E. (2015). Nonlinear regression models and applications in agricultural research. Agronomy Journal, 107(2), 786-798.
Özsoy, V. S., & Örkçü, H. H. (2016). Estimating the parameters of nonlinear regression models through particle swarm optimization. Gazi University Journal of Science, 29(1), 187-199.
Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE transactions on evolutionary computation, 1(1), 67-82.
Copyright (c) 2023 FUDMA JOURNAL OF SCIENCES
This work is licensed under a Creative Commons Attribution 4.0 International License.
FUDMA Journal of Sciences
