# APPLICATION OF POWER NUMERICAL METHOD FOR THE STATIONARY DISTRIBUTION OF MARKOV CHAIN

• Sunday Agboola Nigerian Army University Biu
• Dr. Adebiyi Adeleke University Ede, Osun State, Nigeria
Keywords: Eigenvalue, infinitesimal generator, normalization, power method, stationary distribution

### Abstract

The evolution of this model is represented by transitions from one state to the next. Also, the physical or mathematical behavior of this system can also be illustrated by identifying all of the possible states and explaining how it transitions between them. The iterative solution approaches for the stationary distribution of Markov chains, which begin with an initial estimate of the solution vector  and it becomes closer and closer to the true solution with each iteration are investigated. Our goal is to compute solutions of stationary distribution of Markov chain by utilizing the power iterative method which leaves the transition matrices unchanged and saves time by considering the discretization effect, and the convergency. Matrices operations such as multiplication with one or more vectors, lower, diagonal and upper concepts of matrix, with the help of several existing Markov chain laws, theorems, formulas, and the normalization principle are applied. For the illustrative examples, the stationary distribution vectors  and table of convergence are obtained.

### References

Agboola, S. O. (2022a). The Decomposition and Aggregation Algorithmic Numerical Iterative Solution Methods for the Stationary Distribution of Markov Chain, Journal of Scientific and Engineering 2022, 9(1): 116 - 123. CODEN (USA): JSERBR. www.jsaer.com. DOI: https://doi.org/10.4314/dujopas.v9i1a.13

Agboola, S. O. (2022b). Analysis and Applications of Random Walk and Gambler’s Ruin on Irreducible Periodic Markov Chain, Jewel Journal of Scientific Research, Faculty of Science, Federal University of Kashere. (JJSR) 7(1): 1 – 8, 2022. www.fukashere.edu.ng/jjsr .

Agboola, S. O. (2021). Direct Equation Solving methods Algorithms Compositions of Lower -Upper Triangular Matrix and Grassmann–Taksar–Heyman for the stationary Distribution of Markov chains, International Journal of Applied Science and Mathematics (IJASM). 8(6): 87 – 96. www.ijasm.org.

Agboola, S. O. and Ayinde, S. A. (2022). On the Application of Successive Overrelaxation Algorithmic and Block Numerical Iterative Solutions for the Stationary Distribution in Markov Chain, Nigerian Journal of Pure and Applied Sciences, Faculty of Physical Sciences and Faculty of Life Sciences, University of Ilorin, Nigeria (NJPAS). 35 (1): 4263 – 4272. http://njpas.com.ng. DOI: https://doi.org/10.48198/NJPAS/22.A02

Agboola, S. O. and Ayinde, S. A. (2021). The Performance Measure Analysis on the States Classification in Markov Chain, Dutse Journal of Pure and Applied Sciences (DUJOPAS), Faculty of Science Journal, Federal University Dutse, Jigawa State.7(04b): 19 – 29.https://fud.edu.ng/dujopas. DOI: https://doi.org/10.4314/dujopas.v7i4b.4

Agboola, S. O. and Ayoade, A. A. (2022). Performance Measure Analysis of The Reachability Matrix And Absorption Probabilities For Close And Open Classification Group Of States In Markov Chain, Nigerian Journal of Scientific Research, (NJSR), Ahmadu Bello University Journals, Faculty of Science, ABU, Zaria. 20(4): 1 – 6. https://journals.abu.edu.ng. Njsr.abu.edu.ng.

Agboola, S. O. and Ayoade, A. A. (2021). On the Analysis of Matrix Geometric and Analytical Block Numerical Iterative Methods for Stationary Distribution in the Structured Markov Chains, International Journal of Contemporary Applied Researches (IJCAR), 8(11): 51 – 65, Turkey.http://www.ijcar.net.

Agboola, S. O. and Badmus, N. I. (2021a). The Application of Runge-Kutta and Backward Differentiation Methods for Solving Transient Distribution in Markov Chain, Nigerian Journal of Mathematics and Applications. Department of Mathematics, University of Ilorin, Nigeria. 31: 191 – 201.

Agboola, S. O. and Badmus, N. I. (2021b). Application of Renewal Reward Processes in Homogeneous DOI: https://doi.org/10.33003/fjs-2021-0504-785

Discrete Markov Chain, Faculties of Life and Physical Sciences Journal, Federal University DutsinMa. FUDMA Journal of Sciences (FJS). 5(4): 210 – 215. https://fjs.fudutsinma.edu.ng

Agboola, S. O. and Nehad A. S. (2022). On the Application of Matrix Scaling And Powering Methods of Small State Spaces For Solving Transient Distribution In Markov Chain, FUDMA Journal of Sciences (FJS), Faculties of Earth Science and Physical Science Journal, Federal University DutsinMa.6(1): 135 – 140.https://fjs.fudutsinma.edu.ng DOI: https://doi.org/10.33003/fjs-2022-0601-849

Azizah, A., Welastica, R., Nur, F., Ruchjana, B. and Abdullah, A.(2019). An application of Markov chain for predicting rainfall data at West Java using data mining approach, Earth and Environmental Science, 303(1): 203 – 216. DOI: https://doi.org/10.1088/1755-1315/303/1/012026

Clemence. T. (2019). Markov chain modelling of HIV, Tuberculosis, and Hepatitis B transmission in Ghana, Hindawi, Interdisciplinary Perspective on Infectious Disease, 27(1): 204 – 214. DOI: https://doi.org/10.1155/2019/9362492

Dayar., T. (1998). Permuting Markov chains to nearly completely decomposable form. Technical Report BU-CEIS-9808, Department of Computer Engineering and Information Science, Bilkent University, Ankara, Turkey. pp. 18 – 31.

Pesch, T., Schroder, S., Allelein, H. and Hake, J. (2015). A new Markov chain related statistical approach for modelling synthetic wind power time series, New Journal of Physics, DentschePhysikalishe, 35(2): 64 – 85.

Philippe, B and Sidje, B. (1993) Transient Solution of Markov Processes by Krylov Subspaces, Technical Report, IRISA—Campus de Beaulieu, Rennes, France. pp. 11 - 24.

Ramaswami, V. (1988). A Stable Recursion for the Steady State Vector in Markov chains of M/G/1 type. Communication in Statist. Stochastic Models, 4(1): 183–188. DOI: https://doi.org/10.1080/15326348808807077

Ramaswami, V and Neuts, M. F. (1980). Some explicit formulas and computational methods for infinite server queues with phase type arrivals. Journal of Applied Probability,17(1): 498–514. DOI: https://doi.org/10.2307/3213039

Romanovsky, V.I. (1970). Discrete Markov Chains, Wolters-Noord off, Groningen, Netherlands. pp. 23 – 44.

Stewart, W. J. (1994). Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton, N.J. pp. 14 – 38.

Stewart, W. J. (2009). Probability, Markov Chain, Queues and Simulation, Princeton University Press, United Kingdom. pp. 1- 42. DOI: https://doi.org/10.1515/9781400832811

Uzun, B. and Kiral, E. (2017). Application of Markov chain-fuzzy states to gold price, Science Direct. ELSEVIER,120(1): 365 – 371. DOI: https://doi.org/10.1016/j.procs.2017.11.251

Vermeer, S. And Trilling, D. (2020): Toward a better understanding of a new user journeys: A Markov chain approach. Journalism Journal,21(1):879 – 894. DOI: https://doi.org/10.1080/1461670X.2020.1722958

Zakaria, N. N., Mahmod, O., Rajalmgan, S., Hamita, D., Lazim, A. and Evizal, A. (2019). Markov chain model development for forecasting air pollution index of Miri, Sarawak, Sustainability11(1): 5190 -5202. DOI: https://doi.org/10.3390/su11195190

Published
2023-04-30
How to Cite
AgboolaS., & Adebiyi O. O. (2023). APPLICATION OF POWER NUMERICAL METHOD FOR THE STATIONARY DISTRIBUTION OF MARKOV CHAIN. FUDMA JOURNAL OF SCIENCES, 7(2), 19 - 24. https://doi.org/10.33003/fjs-2023-0702-1625
Section
Research Articles