APPLICATION OF POWER NUMERICAL METHOD FOR THE STATIONARY DISTRIBUTION OF MARKOV CHAIN
Abstract
The evolution of this model is represented by transitions from one state to the next. Also, the physical or mathematical behavior of this system can also be illustrated by identifying all of the possible states and explaining how it transitions between them. The iterative solution approaches for the stationary distribution of Markov chains, which begin with an initial estimate of the solution vector and it becomes closer and closer to the true solution with each iteration are investigated. Our goal is to compute solutions of stationary distribution of Markov chain by utilizing the power iterative method which leaves the transition matrices unchanged and saves time by considering the discretization effect, and the convergency. Matrices operations such as multiplication with one or more vectors, lower, diagonal and upper concepts of matrix, with the help of several existing Markov chain laws, theorems, formulas, and the normalization principle are applied. For the illustrative examples, the stationary distribution vectors and table of convergence are obtained.
References
Agboola, S. O. (2022a). The Decomposition and Aggregation Algorithmic Numerical Iterative Solution Methods for the Stationary Distribution of Markov Chain, Journal of Scientific and Engineering 2022, 9(1): 116 - 123. CODEN (USA): JSERBR. www.jsaer.com.
Agboola, S. O. (2022b). Analysis and Applications of Random Walk and Gambler’s Ruin on Irreducible Periodic Markov Chain, Jewel Journal of Scientific Research, Faculty of Science, Federal University of Kashere. (JJSR) 7(1): 1 – 8, 2022. www.fukashere.edu.ng/jjsr .
Agboola, S. O. (2021). Direct Equation Solving methods Algorithms Compositions of Lower -Upper Triangular Matrix and Grassmann–Taksar–Heyman for the stationary Distribution of Markov chains, International Journal of Applied Science and Mathematics (IJASM). 8(6): 87 – 96. www.ijasm.org.
Agboola, S. O. and Ayinde, S. A. (2022). On the Application of Successive Overrelaxation Algorithmic and Block Numerical Iterative Solutions for the Stationary Distribution in Markov Chain, Nigerian Journal of Pure and Applied Sciences, Faculty of Physical Sciences and Faculty of Life Sciences, University of Ilorin, Nigeria (NJPAS). 35 (1): 4263 – 4272. http://njpas.com.ng.
Agboola, S. O. and Ayinde, S. A. (2021). The Performance Measure Analysis on the States Classification in Markov Chain, Dutse Journal of Pure and Applied Sciences (DUJOPAS), Faculty of Science Journal, Federal University Dutse, Jigawa State.7(04b): 19 – 29.https://fud.edu.ng/dujopas.
Agboola, S. O. and Ayoade, A. A. (2022). Performance Measure Analysis of The Reachability Matrix And Absorption Probabilities For Close And Open Classification Group Of States In Markov Chain, Nigerian Journal of Scientific Research, (NJSR), Ahmadu Bello University Journals, Faculty of Science, ABU, Zaria. 20(4): 1 – 6. https://journals.abu.edu.ng. Njsr.abu.edu.ng.
Agboola, S. O. and Ayoade, A. A. (2021). On the Analysis of Matrix Geometric and Analytical Block Numerical Iterative Methods for Stationary Distribution in the Structured Markov Chains, International Journal of Contemporary Applied Researches (IJCAR), 8(11): 51 – 65, Turkey.http://www.ijcar.net.
Agboola, S. O. and Badmus, N. I. (2021a). The Application of Runge-Kutta and Backward Differentiation Methods for Solving Transient Distribution in Markov Chain, Nigerian Journal of Mathematics and Applications. Department of Mathematics, University of Ilorin, Nigeria. 31: 191 – 201.
Agboola, S. O. and Badmus, N. I. (2021b). Application of Renewal Reward Processes in Homogeneous
Discrete Markov Chain, Faculties of Life and Physical Sciences Journal, Federal University DutsinMa. FUDMA Journal of Sciences (FJS). 5(4): 210 – 215. https://fjs.fudutsinma.edu.ng
Agboola, S. O. and Nehad A. S. (2022). On the Application of Matrix Scaling And Powering Methods of Small State Spaces For Solving Transient Distribution In Markov Chain, FUDMA Journal of Sciences (FJS), Faculties of Earth Science and Physical Science Journal, Federal University DutsinMa.6(1): 135 – 140.https://fjs.fudutsinma.edu.ng
Azizah, A., Welastica, R., Nur, F., Ruchjana, B. and Abdullah, A.(2019). An application of Markov chain for predicting rainfall data at West Java using data mining approach, Earth and Environmental Science, 303(1): 203 – 216.
Clemence. T. (2019). Markov chain modelling of HIV, Tuberculosis, and Hepatitis B transmission in Ghana, Hindawi, Interdisciplinary Perspective on Infectious Disease, 27(1): 204 – 214.
Dayar., T. (1998). Permuting Markov chains to nearly completely decomposable form. Technical Report BU-CEIS-9808, Department of Computer Engineering and Information Science, Bilkent University, Ankara, Turkey. pp. 18 – 31.
Pesch, T., Schroder, S., Allelein, H. and Hake, J. (2015). A new Markov chain related statistical approach for modelling synthetic wind power time series, New Journal of Physics, DentschePhysikalishe, 35(2): 64 – 85.
Philippe, B and Sidje, B. (1993) Transient Solution of Markov Processes by Krylov Subspaces, Technical Report, IRISA—Campus de Beaulieu, Rennes, France. pp. 11 - 24.
Ramaswami, V. (1988). A Stable Recursion for the Steady State Vector in Markov chains of M/G/1 type. Communication in Statist. Stochastic Models, 4(1): 183–188.
Ramaswami, V and Neuts, M. F. (1980). Some explicit formulas and computational methods for infinite server queues with phase type arrivals. Journal of Applied Probability,17(1): 498–514.
Romanovsky, V.I. (1970). Discrete Markov Chains, Wolters-Noord off, Groningen, Netherlands. pp. 23 – 44.
Stewart, W. J. (1994). Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton, N.J. pp. 14 – 38.
Stewart, W. J. (2009). Probability, Markov Chain, Queues and Simulation, Princeton University Press, United Kingdom. pp. 1- 42.
Uzun, B. and Kiral, E. (2017). Application of Markov chain-fuzzy states to gold price, Science Direct. ELSEVIER,120(1): 365 – 371.
Vermeer, S. And Trilling, D. (2020): Toward a better understanding of a new user journeys: A Markov chain approach. Journalism Journal,21(1):879 – 894.
Zakaria, N. N., Mahmod, O., Rajalmgan, S., Hamita, D., Lazim, A. and Evizal, A. (2019). Markov chain model development for forecasting air pollution index of Miri, Sarawak, Sustainability11(1): 5190 -5202.
Copyright (c) 2023 FUDMA JOURNAL OF SCIENCES
This work is licensed under a Creative Commons Attribution 4.0 International License.
FUDMA Journal of Sciences
