In this article, we develop and analyze a continuous-time Markov chain (CTMC) model to study the resurgence of dengue. We also explore the large population asymptotic behavior of probabilistic model of dengue using th...In this article, we develop and analyze a continuous-time Markov chain (CTMC) model to study the resurgence of dengue. We also explore the large population asymptotic behavior of probabilistic model of dengue using the law of large numbers (LLN). Initially, we calculate and estimate the probabilities of dengue extinction and major outbreak occurrence using multi-type Galton-Watson branching processes. Subsequently, we apply the LLN to examine the convergence of the stochastic model towards the deterministic model. Finally, theoretical numerical simulations are conducted exploration to validate our findings. Under identical conditions, our numerical results demonstrate that dengue could vanish in the stochastic model while persisting in the deterministic model. The highlighting of the law of large numbers through numerical simulations indicates from what population size a deterministic model should be considered preferable.展开更多
A new stochastic epidemic model, that is, a general continuous time birth and death chain model, is formulated based on a deterministic model including vaccination. We use continuous time Markov chain to construct the...A new stochastic epidemic model, that is, a general continuous time birth and death chain model, is formulated based on a deterministic model including vaccination. We use continuous time Markov chain to construct the birth and death process. Through the Kolmogorov forward equation and the theory of moment generating function, the corresponding population expectations are studied. The theoretical result of the stochastic model and deterministic version is also given. Finally, numerical simulations are carried out to substantiate the theoretical results of random walk.展开更多
Motivated by the lack of a suitable constructive framework for analyzing popular stochastic models of Systems Biology, we devise conditions for existence and uniqueness of solutions to certain jump stochastic differen...Motivated by the lack of a suitable constructive framework for analyzing popular stochastic models of Systems Biology, we devise conditions for existence and uniqueness of solutions to certain jump stochastic differential equations (SDEs). Working from simple examples we find reasonable and explicit assumptions on the driving coefficients for the SDE representation to make sense. By “reasonable” we mean that stronger assumptions generally do not hold for systems of practical interest. In particular, we argue against the traditional use of global Lipschitz conditions and certain common growth restrictions. By “explicit”, finally, we like to highlight the fact that the various constants occurring among our assumptions all can be determined once the model is fixed. We show how basic long time estimates and some limit results for perturbations can be derived in this setting such that these can be contrasted with the corresponding estimates from deterministic dynamics. The main complication is that the natural path-wise representation is generated by a counting measure with an intensity that depends nonlinearly on the state.展开更多
We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The co...We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.展开更多
This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance mi...This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.展开更多
Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of off...Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.展开更多
In this paper we study the average sample-path cost （ASPC） problem for continuous-time Markov decision processes in Polish spaces. To the best of our knowledge, this paper is a first attempt to study the ASPC criter...In this paper we study the average sample-path cost （ASPC） problem for continuous-time Markov decision processes in Polish spaces. To the best of our knowledge, this paper is a first attempt to study the ASPC criterion on continuous-time MDPs with Polish state and action spaces. The corresponding transition rates are allowed to be unbounded, and the cost rates may have neither upper nor lower bounds. Under some mild hypotheses, we prove the existence of （ε〉 0）-ASPC optimal stationary policies based on two different approaches： one is the ＂optimality equation＂ approach and the other is the ＂two optimality inequalities＂ approach.展开更多
This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space...This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.展开更多
文摘In this article, we develop and analyze a continuous-time Markov chain (CTMC) model to study the resurgence of dengue. We also explore the large population asymptotic behavior of probabilistic model of dengue using the law of large numbers (LLN). Initially, we calculate and estimate the probabilities of dengue extinction and major outbreak occurrence using multi-type Galton-Watson branching processes. Subsequently, we apply the LLN to examine the convergence of the stochastic model towards the deterministic model. Finally, theoretical numerical simulations are conducted exploration to validate our findings. Under identical conditions, our numerical results demonstrate that dengue could vanish in the stochastic model while persisting in the deterministic model. The highlighting of the law of large numbers through numerical simulations indicates from what population size a deterministic model should be considered preferable.
文摘A new stochastic epidemic model, that is, a general continuous time birth and death chain model, is formulated based on a deterministic model including vaccination. We use continuous time Markov chain to construct the birth and death process. Through the Kolmogorov forward equation and the theory of moment generating function, the corresponding population expectations are studied. The theoretical result of the stochastic model and deterministic version is also given. Finally, numerical simulations are carried out to substantiate the theoretical results of random walk.
文摘Motivated by the lack of a suitable constructive framework for analyzing popular stochastic models of Systems Biology, we devise conditions for existence and uniqueness of solutions to certain jump stochastic differential equations (SDEs). Working from simple examples we find reasonable and explicit assumptions on the driving coefficients for the SDE representation to make sense. By “reasonable” we mean that stronger assumptions generally do not hold for systems of practical interest. In particular, we argue against the traditional use of global Lipschitz conditions and certain common growth restrictions. By “explicit”, finally, we like to highlight the fact that the various constants occurring among our assumptions all can be determined once the model is fixed. We show how basic long time estimates and some limit results for perturbations can be derived in this setting such that these can be contrasted with the corresponding estimates from deterministic dynamics. The main complication is that the natural path-wise representation is generated by a counting measure with an intensity that depends nonlinearly on the state.
基金Acknowledgements The authors would like to thank Professor Yong-Hua Mao for useful discussion. This work was supported in part by the National Natural Science Foundation of China （Grant Nos. 11571372, 11501576, 11771452） and the Excellent Young Scientific Research Fund of Hunan Provincial Education Department （Grant No. 15B252）.
文摘We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.
基金supported by the National Natural Science Foundation of China(10801056)the Natural Science Foundation of Ningbo(2010A610094)
文摘This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.
文摘Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.
基金Supported by the National Natural Science Foundation of China (No.10801056)the Natural Science Foundation of Ningbo (No. 2010A610094)K.C. Wong Magna Fund in Ningbo University
文摘In this paper we study the average sample-path cost （ASPC） problem for continuous-time Markov decision processes in Polish spaces. To the best of our knowledge, this paper is a first attempt to study the ASPC criterion on continuous-time MDPs with Polish state and action spaces. The corresponding transition rates are allowed to be unbounded, and the cost rates may have neither upper nor lower bounds. Under some mild hypotheses, we prove the existence of （ε〉 0）-ASPC optimal stationary policies based on two different approaches： one is the ＂optimality equation＂ approach and the other is the ＂two optimality inequalities＂ approach.
文摘This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.
