PERFORMANCE COMPARISON OF DIFFERENT ACTIVATION FUNCTIONS IN NEURAL NETWORKS FOR BIOMASS ENERGY CONTENT PREDICTION
Abstract
The application of artificial neural networks to solve complex linear and nonlinear problems such as predicting biomass higher heating value (HHV) requires meticulous choice of the number of layers and neurons per layer, the training algorithms, and the activation functions among other hyper-parameters. Although some studies have examined these hyper-parameters, the effects of different activation functions on biomass HHV prediction have not attracted credible research attention. This study, therefore, employs three distinct activation functions (logsig, tansig, and purelin) in artificial neural networks for biomass HHV prediction based on proximate analysis. A 3-10-1 network architecture was used and the variation of the hidden layer and output layer activation functions yielded nine models (M1-M9) whose performances were assessed and compared using statistical indices. The results showed that the best performance was observed by model M2 which utlized the logsig function in the hidden layer and the tansig function in the output layer. This model had the highest determination coefficient of 0.8814 and the lowest mean square error and mean absolute error of 0.0017 and 0.0281 respectively. Understanding how these hyper-parameters influence biomass HHV prediction would guide the energy community to identify an optimal pathway to bioenergy production.
References
Adeleke, A. A., Adedigba, A., Adeshina, S. A., Ikubanni, P. P., Lawal, M. S., Olosho, A. I., Yakubu, H. S., Ogedengbe, T. S., Nzerem, P., & Okolie, J. A. (2024). Comparative studies of machine learning models for predicting higher heating values of biomass. Digital Chemical Engineering, 12(100159), 10. https://doi.org/10.1016/j.dche.2024.100159 DOI: https://doi.org/10.1016/j.dche.2024.100159
Afolabi, I. C., Epelle, E. I., Gunes, B., Gle, F., & Okolie, J. A. (2022). Data-Driven Machine Learning Approach for Predicting the Higher Heating Value of Different Biomass Classes. Clean Technologies, 4(4), 12271241. https://doi.org/10.3390/cleantechnol4040075 DOI: https://doi.org/10.3390/cleantechnol4040075
Balarabe, M. A., & Isah, M. N. (2019). Feed-Forward and Cascade Back Propagation Artificial Neural Network Models for. FUDMA Journal of Sciences, 3(1), 428433.
Brandi, I., Pezo, L., Bilandija, N., Peter, A., & uri, J. (2022). Artificial Neural Network as a Tool for Estimation of the Higher Heating Value of Miscanthus Based on Ultimate Analysis. 10(20), 112. DOI: https://doi.org/10.3390/math10203732
Dashti, A., Noushabadi, A. S., Raji, M., Razmi, A., Ceylan, S., & Mohammadi, A. H. (2019). Estimation of biomass higher heating value (HHV) based on the proximate analysis: Smart modeling and correlation. Fuel, 257(115931), 111. https://doi.org/10.1016/j.fuel.2019.115931 DOI: https://doi.org/10.1016/j.fuel.2019.115931
Dastres, R., Soori, M., Neural, A., Systems, N., & Journal, I. (2021). Artificial Neural Network Systems To cite this version: HAL IAd: hal-03349542. International Journal of Imaging and Robotics (IJIR), 21(2), 1325.
Dodo, U. A., Ashigwuike, E. C., & Abba, S. I. (2022). Machine learning models for biomass energy content prediction: A correlation-based optimal feature selection approach. Bioresource Technology Reports, 19(101167), 11. https://doi.org/10.1016/j.biteb.2022.101167 DOI: https://doi.org/10.1016/j.biteb.2022.101167
Dodo, U. A., Ashigwuike, E. C., Emechebe, J. N., & Abba, I. S. (2022). Prediction of energy content of biomass based on hybrid machine learning ensemble algorithm. Energy Nexus, 8, 15. https://doi.org/10.1016/j.nexus.2022.100157 DOI: https://doi.org/10.1016/j.nexus.2022.100157
Dodo, U. A., Dodo, M. A., Belgore, A. T., Husein, M. A., Ashigwuike, E. C., Mohammed, A. S., & Abba, S. I. (2024). Comparative study of different training algorithms in backpropagation neural networks for generalized biomass higher heating value prediction. Green Energy and Resources, 2(1), 13. https://doi.org/10.1016/j.gerr.2024.100060 DOI: https://doi.org/10.1016/j.gerr.2024.100060
Dodo, U. A., Dodo, M. A., Shehu, A. F., & Badamasi, Y. A. (2023). Performance Analysis of Intelligent Computational Algorithms for Biomass Higher Heating Value Prediction. Nigerian Journal of Technological Development, 20(4), 4452. https://doi.org/10.4314/njtd.v18i4.1856 DOI: https://doi.org/10.4314/njtd.v20i4.1856
Estiati, I., Freire, F. B., Freire, J. T., Aguado, R., & Olazar, M. (2016). Fitting performance of artificial neural networks and empirical correlations to estimate higher heating values of biomass. Fuel, 180, 377383. https://doi.org/10.1016/j.fuel.2016.04.051 DOI: https://doi.org/10.1016/j.fuel.2016.04.051
Gle, F., Pekaslan, D., Williams, O., & Lester, E. (2022). Predictability of higher heating value of biomass feedstocks via proximate and ultimate analyses A comprehensive study of artificial neural network applications. Fuel, 320(123944), 116. https://doi.org/10.1016/j.fuel.2022.123944 DOI: https://doi.org/10.1016/j.fuel.2022.123944
Gunamantha, M. (2016). Prediction of Higher Heating Value Bioorganic Fraction of Municipal Solid Waste from Proximate Analysis Data. International Journal of Engineering Research & Technology, 5(2), 442447. http://www.ijert.org
Jaki, O., Jaki, Z., Guha, K., Silva, A. G., & Laskar, N. M. (2023). Comparing artificial neural network algorithms for prediction of higher heating value for different types of biomass. Soft Computing, 27(9), 59335950. https://doi.org/10.1007/s00500-022-07641-4 DOI: https://doi.org/10.1007/s00500-022-07641-4
Kujawska, J., Kulisz, M., Oleszczuk, P., & Cel, W. (2023). Improved Prediction of the Higher Heating Value of Biomass Using an Artificial Neural Network Model Based on the Selection of Input Parameters. Energies, 16(10), 116. https://doi.org/10.3390/en16104162 DOI: https://doi.org/10.3390/en16104162
Laabid, Z., Moumen, A., Mansouri, K., & Siadat, A. (2023). Numerical study of the speeds response of the various intelligent models using the tansig, logsig and purelin activation functions in different layers of artificial neural network. IAES International Journal of Artificial Intelligence, 12(1), 155161. https://doi.org/10.11591/ijai.v12.i1.pp155-161 DOI: https://doi.org/10.11591/ijai.v12.i1.pp155-161
Lederer, J. (2021). Activation Functions in Artificial Neural Networks: A Systematic Overview. In Ruhr-University Bochum, Germany (pp. 142). http://arxiv.org/abs/2101.09957
Lpez, O. A. M., Lpez, A., & Crossa, J. M. (2022). Fundamentals of Artificial Neural Networks and Deep Learning. In Multivariate Statistical Machine Learning Methods for Genomic Prediction. https://doi.org/10.1007/978-3-030-89010-0_10 DOI: https://doi.org/10.1007/978-3-030-89010-0_10
Matveeva, A., & Bychkov, A. (2022). How to Train an Artificial Neural Network to Predict Higher Heating Values of Biofuel. Energies, 15(19), 113. https://doi.org/10.3390/en15197083 DOI: https://doi.org/10.3390/en15197083
Nandi, A., Jana, N. D., & Das, S. (2020). Improving the Performance of Neural Networks with an Ensemble of Activation Functions. Proceedings of the International Joint Conference on Neural Networks, December 2024. https://doi.org/10.1109/IJCNN48605.2020.9207277 DOI: https://doi.org/10.1109/IJCNN48605.2020.9207277
Nhuchhen, D. R., & Salam, P. A. (2012). Estimation of higher heating value of biomass from proximate analysis: A new approach. Fuel, 99, 5563. https://doi.org/10.1016/j.fuel.2012.04.015 DOI: https://doi.org/10.1016/j.fuel.2012.04.015
Phichai, K., Pragrobpondee, P., Khumpart, T., & Hirunpraditkoon, S. (2013). Prediction Heating Values of Lignocellulosics from Biomass Characteristics. International Journal of Chemical, Materials Science and Engineering, 7(7), 14.
Pokhrel, S. (2024). Intelligent Lighting Control Systems for Energy Savings in Hospital Buildings using Artificial Neural Networks. FUDMA Journal of Sciences, 15(1), 3748.
Qamar, R., & Baqar, A. Z. (2023). Artificial Neural Networks: An Overview. 2023, 124133. DOI: https://doi.org/10.58496/MJCSC/2023/015
Qian, X., Lee, S., Soto, A., & Chen, G. (2018). Regression Model to Predict the Higher Heating Value of Poultry Waste from Proximate Analysis. Resources, 7(39), 114. https://doi.org/10.3390/resources7030039 DOI: https://doi.org/10.3390/resources7030039
Rasamoelina, A. D., Adjailia, F., & Sincak, P. (2020). A Review of Activation Function for Artificial Neural Network. SAMI 2020 - IEEE 18th World Symposium on Applied Machine Intelligence and Informatics, Proceedings, July 2024, 281286. https://doi.org/10.1109/SAMI48414.2020.9108717 DOI: https://doi.org/10.1109/SAMI48414.2020.9108717
Reyes-Tllez, E. D., Parrales, A., Ramrez-Ramos, G. E., Hernndez, J. A., Urquiza, G., Heredia, M. I., & Sierra, F. Z. (2020). Analysis of transfer functions and normalizations in an ann model that predicts the transport of energy in a parabolic trough solar collector. Desalination and Water Treatment, 200, 2341. https://doi.org/10.5004/dwt.2020.26063 DOI: https://doi.org/10.5004/dwt.2020.26063
Shehu, A. F., & Belgore, A. T. (2023). Machine Learning Approach to Wind Speed Prediction using Soft Computing Tools. Journal of Science Technology and Education, 11(2), 349355. www.atbuftejoste.net
Suryadevara, S., Kumar, A., & Yanamala, Y. (2021). A Comprehensive Overview of Artificial Neural Networks: Evolution , Architectures , and Applications. 12(01), 5176.
Szandaa, T. (2021). Review and comparison of commonly used activation functions for deep neural networks. In Studies in Computational Intelligence (Vol. 903, pp. 203224). https://doi.org/10.1007/978-981-15-5495-7_11 DOI: https://doi.org/10.1007/978-981-15-5495-7_11
Uzun, H., Yldz, Z., Goldfarb, J. L., & Ceylan, S. (2017). Improved prediction of higher heating value of biomass using an artificial neural network model based on proximate analysis. Bioresource Technology, 234, 122130. https://doi.org/10.1016/j.biortech.2017.03.015 DOI: https://doi.org/10.1016/j.biortech.2017.03.015
Veza, I., Irianto, Panchal, H., Paristiawan, P. A., Idris, M., Fattah, I. M. R., Putra, N. R., & Silambarasan, R. (2022). Improved prediction accuracy of biomass heating value using proximate analysis with various ANN training algorithms. Results in Engineering, 16(100688), 16. https://doi.org/10.1016/j.rineng.2022.100688 DOI: https://doi.org/10.1016/j.rineng.2022.100688
Xing, J., Luo, K., Wang, H., Gao, Z., & Fan, J. (2019). A comprehensive study on estimating higher heating value of biomass from proximate and ultimate analysis with machine learning approaches. Energy, 188(116077), 135. https://doi.org/10.1016/j.energy.2019.116077 DOI: https://doi.org/10.1016/j.energy.2019.116077
Yang, X., Li, H., Wang, Y., & Qu, L. (2023). Predicting Higher Heating Value of Sewage Sludges via Artificial Neural Network Based on Proximate and Ultimate Analyses. Water (Switzerland), 15(4), 16. https://doi.org/10.3390/w15040674 DOI: https://doi.org/10.3390/w15040674
Copyright (c) 2025 FUDMA JOURNAL OF SCIENCES
This work is licensed under a Creative Commons Attribution 4.0 International License.
FUDMA Journal of Sciences
