Optimal control of the treatment frequency in a stochastic model of Tuberculosis

This paper presents a stochastic model of the Tuberculosis(TB) infection with treatment in a population composed of four individuals compartments: susceptible individuals, latent infected individuals, active infected individuals and recovered individuals after the therapy. A preliminary survey of the model is performed on the stability before approaching the crucial left of the topic. The aim in this paper is to control the treatment frequency in a stochastic model of the TB infection while minimizing the cost of the measures. Then, we formulate an optimal control problem that consists in minimizing the relative cost of the dynamics of TB-model in order to reduce the prevalence and the mortality due to this infection. The optimal problem is solved by applying the Projection Stochastic Gradient Method in order to find the optimal numerical solution. Finally, we provide some numerical simulations of the controlled model. Keywords-Stochastic Model of TB; local and Global Stability; Optimal Control; Functional Cost; Projection Stochastic Gradient.


I. INTRODUCTION
The tuberculosis (TB) continues to make a lot of victims in our societies despite of the exist-ing treatment: the Bacillus Calmette-Guerin.The vaccine anti tubercular is used for preventive treatment for children.Nevertheless, other medicines exist as Rifampicin, Isoniazid, Pyrazinamide... for the curative treatment of the patients [23].The expenses are enormous when the treatment is long.The tuberculosis is one of the causes of elevated mortality in humane communities irrespective of the enormous financial resources made by worldwide governments for the treatment of this disease in the purpose of its eradication.So there is the necessity to integrate to a set of the available control an optimal measure that consists on respecting the dose of the treatment to short length in order to reduce this infection.
In this paper, we consider a stochastic model of the Tuberculosis (TB) infection in presence of treatment in a population composed of four compartments of individuals: susceptible individuals, latent infected individual, active infected individuals and recovered individuals after the therapy.The mathematical model of TB infection include in addition to the deterministic term, a stochastic term that translates the random noise.The random nature of this model is due to the fact that the contraction of the Mycobacterium Tuberculosis, the vector agent of the TB infection and his transmission within the population are done in an random manner according to the variable efficiency of control of the immune system of the individuals.The infection of TB contracts itself mainly by the inhalation of the bacteria distributed by the cough or the sneeze of a sick individual.The vector agent of this infection accommodates itself to the level of the lungs of an individual exposition susceptible of contamination and the immune system of this one controls and maintains the infection in the latent state; otherwise there is the risk that this infection develops itself toward the active state.While supposing that only the infected individuals of active TB transmit the infection, they must observe some hygienic rules, they must adopt a positive behavior with respect to the susceptible individuals (who must also take precaution), to follow the treatment up to finish as early as possible (in less than one year), constitute measures of adequate control.A preliminary survey of the model is performed before introducing a function of control representing the necessary dose of medicines in order to control the frequency of the treatment and to reduce considerably and quickly the prevalence of the disease.The main objective is the control of the treatment frequency in the stochastic model of the TB infection.So we formulate an optimal control problem that consists in minimizing the relative cost of the dynamics of the model in order to reduce the prevalence and mortality due to this infection.To solve this optimal control problem, we are going to apply the Stochastic Gradient Method with Projection in order to find the optimal numeric solution.Finally, thanks to the numerical simulation tool, we simulate this model without or with control as well as the optimal solution and the associated cost function in order to characterize an optimal decision.In epidemiology and others domain as biology, demography, economy..., many stochastic models deriving from their deterministic formulation.The reference of the literature for a variety of wellknown stochastic models deriving from their de-terministic counterparts include the books [1], [5], [6], [7], and [22].Our contribution is first in Sub Section II.A, the formulation of a stochastic model of TB with treatment from a deterministic model of TB-only (Sharomi [18]) which is formulated along the lines of the model in Feng and al. [26].Secondly in Sub Section II.B, we change this stochastic model by perturbations or by an affine change of variables affine to lead the survey of the stability of the random equilibrium because the used transformation keeps the law of probability of an random variable [12].Finally in Sub Section III.A, we control the treatment frequency in this stochastic model in order to reduce mortality due to the infection.The continuation of the paper is like follows: we recall the results that concern the projection method in Sub Section III.B.The gradient projection method is applied to the model in Sub Section III.C, and the numerical simulations are plotted in Sub Section III.D.

CONTROL
We start this section by the description of the variables and parameters of the model (see Table I) then follows it by the presentation of the model.The diagram of the stochastic model of the TB infection is given by Fig 1.

A. Diagram and Mathematical Stochastic Model of TB
The mathematical stochastic model of TB infection in the presence of treatment is written under the compact form by the following equation (1) (its formulation uses [1], [5], [6], [7], and [22]) where , is a 10-dimensional Brownian motion process and is defined on a space of (Ω, F, {F t } t≥0 , P ); where The TB force of infection λ I is defined by: with The force of exogenous infection λ r is defined by:

B. Analysis of the solution of model and Stability
In this part, we are going to show the existence and the uniqueness of the global solution positive of the model (1).We also address the existence and stability characterization of the Disease Free-Equilibrium (DFE) and of the endemic equilibrium point.

1) Existence and Uniqueness of solution:
Consider a region Ω ⊂ R 4 + defined by Then, we has the following result: Theorem 1.Let (S 0 , L 0 , T 0 , R 0 ) ∈ Ω an initial condition.Then there is a unique solution of the stochastic model (1) denoted X t = (S t , L t , T t , R t ) T such that Proof: See Appendix A.
2) Stochastic Stability of the random DFE: Let us recall the following that will a very helpful in the sequel Lemma 1.Let p ≥ 2, x, y ∈ R + and ε > 0 sufficiently small Proof: The inequalities above can be demonstrated with the help of the inequalities of Young: f or p, q > 0 and Proposition 1.The stochastic model (1) admits a random equilibrium point without TB (Disease-Free random Equilibrium) X 0 = Λ µ , 0, 0, 0 that is exponentially p-stable if p ≥ 2 and globally asymptotically stable.
Proof: By translation, we can always bring back a random equilibrium point X e to X e = 0 like in [25].
The existence of X 0 , disease-free random equilibrium point is proved by the following change variable for the stochastic model ( 1) As a consequence, the stochastic model (1) reads as wherein where with The existence of a disease-free random equilibrium of the model (8) gives the existence of disease-free random equilibrium of (1).In fact, Denote by X(0) ≡ 0 ∈ R 4 .The equalities f (t, 0) = 0 and G(t, 0) = 0 are verified for t ≥ 0. So X(0) a disease-free random equilibrium of the model (8).Therefore, we have i.e., X 0 = Λ µ , 0, 0, 0 is a disease-free random equilibrium of the model (1).Now, consider a Lyapunov function: Let us note by A a differential operator associated to the stochastic model ( 1), operating on a function The application of the lemma 1 and the theorem given by Afanas'ev in [24], allows us to obtain finally AV ≤ 0 (necessary to demonstrate). Therefore, For, p = 2, we say that X 0 is exponentially 2stable or stable in mean square [24].In the sense of Lyapunov, X 0 is globally asymptotically stable.
It marks the end of the proof.
3) Stability of the endemic random equilibrium: Preliminary: Suppose that the infection of TB evolves of manner linear i.e. without random noise G(X t , t) ≡ 0. Then the model (1) become dX t = f (X t , t)dt which admits a basic reproduction number R τ 0 given by: )dt admits a unique endemic equilibrium point biologically meaningful, X * that is locally asymptotically stable [18].The existence of a random endemic equilibrium [X * = (S * , L * , T * , R * )] is guaranteed by the condition R τ 0 > 1 almost surely (see [8]).At this random endemic equilibrium Biomath 6 (2017), 1705077, http://dx.doi.org/10.11145/j.biomath.2017.05.077 and where If R τ 0 = 1 ie.a 0 = 0, the equation ( 14) admits a hopeless solution corresponding to X 0 the unique equilibrium without TB and it admits another solution to the real negative part corresponding to the endemic equilibrium which is biologically not pertinent.
If R τ 0 < 1 ie.a 0 > 0, then a 2 a 0 < 0 and if the discriminant of ( 14) is positive i.e. a 2 1 −4a 2 a 0 > 0. It follows itself that the equation ( 14) admits two solutions to part real negatives that corresponding to two equilibriums no pertinent.
If R τ 0 > 1 ie.a 0 < 0 then according to the Descartes rule of sign, the equation ( 14) admits one positive solution Now, suppose that the random noise of the dynamic system of TB has a nature to perturb the states variables S t , L t , T t , and R t of the stochastic term G(t, X t ) around of S * , L * , T * , and R * respectively (see also [25]).Then the model (1) becomes that can be centered to X * by the change variables The linearized system of (15) around X * = (S * , L * , T * , R * ) T as in [4] takes the form where with .
The random endemic equilibrium [X * = (S * , L * , T * , R * ) T ] of the model (1) exists whenever R τ 0 > 1 and condition (i.) is fulfilled.It is exponentially 2-stable and globally asymptotically stable in sense of Lyapunov if the supplementary condition (ii.) is satisfied.
We study in the following section, the optimal control of the treatment frequency in a stochastic model of TB.The condition R τ 0 < 1 is needed for the effective stability of TB in a population because the biological pertinence of the endemic equilibrium exists whenever R τ 0 > 1 almost surely.The control permits then to adjust this endemic situation unstable.

III. OPTIMAL CONTROL OF THE TREATMENT FREQUENCY IN THE TB MODEL A. Optimal control problem
Let (Ω, F, {F t } t≥0 , P ) a complete filtered probability space {F t } t≥0 produced by a standard 10dimensional Brownian Motion {W t } t≥0 .Let T > 0 a fixed real number named the horizon of the finite time.Let's note by L 2 (Ω, F T , R) the space of random variables.F T -measurable to real values and integrable square and by L 2 F (0; T , R) a space of process F t -adapted to real values and integrable square such that E[ Let K ∈ U ad a compact convex sub set of L 2 (0, T ).Consider an optimal control problem that consists in minimizing the cost J (., .), the objective function defined for the time t ∈ [0, T ], the state X ∈ R 4 and function of control u ∈ U ad by: ) relative to the state X t ∈ R 4 of the TB model governed in general by: Biomath 6 (2017), 1705077, http://dx.doi.org/10.11145/j.biomath.2017.05.077 and in particular by: where u = u t : τ −→ u(t)τ , for all rate τ of (2).This part deals with the study of the particular case where the control doesn't appear in the stochastic term.The control is said optimal when this dose reached its value optimal positive i.e. u = u op > 0. If this optimal value is not reached, i.e. u ∈ [−1; 0[∪]0; u op [, then the control is said less efficient; it is said without effect when u = 0 and finally the control is said efficient when the optimal value is passed) i.e. u ∈]u op , 1].The aim is therefore to control the frequencies of the TB treatment in order to reduce number of new cases.The problem of the optimal control is translated to: Find an admissible control optimal u = u * such that Set F (u) = J (X, u), then the optimal control problem (23) becomes a optimization problem wherein F (u) is a functional convex.

B. Gradient Projection Method
We want to solve ( 24) by the projection stochastic gradient method.For this purpose, let us recall the results that concern the projection method on a convex closed K an the stochastic algorithm: Proposition 2. Let, H a Hilbert space, provided with a norm .induced by the scalar product (•|•) and let K ⊂ H a nonempty convex closed set.Then for all u ∈ H, 1) an unique ũ ∈ K exists such that where ũ = P K (u) is the orthogonal projection of u on K. 2) ũ is charcterized by 1) The existence of ũ ∈ K holds true because K is closed.Let's suppose that the dimension of H is finite.Let us consider K ∩B(u; u− v ) the intersection of K with a ball B. On this compact, the function v −→ u − v is continuous.Of all minimizing sequence we can extract a convergent sequence, its limit is ũ.The uniqueness comes from the convexity of K and Pythagoras' theorem.2) For the characterization of ũ; suppose ũ = P K (u) then we has for all Dividing by ε then we obtain Applying the inequality of Cauchy-Schwarz, we have Proposition 3. The algorithm of the stochastic gradient consists in making evolve the variable u of the optimization problem (24) according to the formula of the following recurrence convergent sequence (u n ) n≥0 of limit u: where q n > 0 with lim n−→+∞ q n = q and ∇ denotes the gradient.
C. Projection gradient method applied to the stochastic model of TB with control Proposition 4. Consider H = U is a Hilbert space and U ad ⊂ U a closed convex subset.Let P K the projection operator on K defined in U by P K (ω) = P K ω ∈ K; ∀ ω ∈ U, then problem (24) admits an unique solution u or an optimal control such that Proof: H = U is a Hilbert space and U ad ⊂ U a closed convex subset.The necessary and sufficient condition of the optimality problem ( 24) is given by Let P K the projection operator on K defined in U by P K (ω) = P K ω ∈ K; ∀ ω ∈ U, such that we have It is equivalent to It follows that the solution u of ( 24) is given by Indeed, the optimality condition gives then for q > 0 we have , the last implication gives For the optimal control problem of the treatment frequency of TB, we are going to define the following iteration scheme for n = 0, 1, ...
where F n is the functional approached to the n th iteration of F n .
The convergence of this scheme, and the calculation of F n .are given in [17], [13].For u(•) an optimal control and X(•), the optimal stat corresponding to X(•) and for v( where Biomath 6 (2017), 1705077, http://dx.doi.org/10.11145/j.biomath.2017.05.077 We define a adjoint functional p, F t -adapted and defined by The right hand side of the equation ( 27) permits to get finally F n (u)(v) from (28), that reads as The Projection Gradient Method applied to the stochastic model of TB with control, consist therefore in considering the system (30) of two equations ( 22) and (28) in order to solve it numerically, (30) The numerical resolution of (30) uses the iteration scheme (31) below for n = 0, 1... and then the Euler scheme for the two equations of (30) (see [17]) ,   For the following figures, we take ϕ(x, u) = (x 2 + u 2 )exp( −t x 2 + u 2 ), h(x) = x 2 , n = 0.05, e n < 10 −7 , p 0 = 0.01 and the rest Λ, β T , τ , X 0 are variable.
Fig. 2 give a schematic plot of the model ( 1) not depending of u.The aim is to show, for a initial condition given, the asymptotic behavior of the solution around a random endemic equilibrium when the hard epidemic a long time R τ 0 > 1. While, Fig. 3(resp.Fig. 4) shows a numerical illustration of optimal control u, see (a) and (b) (resp. of cost F (u), see (c) and (d)).The orthogonal projection of the minimum point of F (u) on the closed subset [−1; 1], gives a numerical value of optimal control u * ; e.g. the minimum point • of F (u) represented in (c), is valued as F (u * ) = 2.7066 giving u * = 0 if u 0 = 1.Thanks to Matlab, we can value the cost F (u * ) and the optimal control u * for a control initially chosen u 0 as Fig. 6.In Fig. 5, the trajectory without control, see (t. 1 ), (resp.with control, see (t. 2 )) of the active infected individuals T is creasing between 0 and 2 years (resp.decreasing between 0 and 1 year and is annulling constantly thereafter).(t. 3 ) and (t. 4 ) show that a control of treatment intervened 0.5 years equal to 6 months after the infection, permits to reduce to nothing numbers of the active infected individuals of TB.Let's note that initial value of sequence q n is chosen as q 0 = 0.1 for Fig. 6 and (d); q 0 = 0.6 for Fig. 4 (c).

IV. CONCLUSION
The stochastic model (1) of TB without control admits for an initial state X(0), a positive and unique solution X t ∈ Ω of probability one.It exist for this model an unique disease equilibrium free (DEF) exponentially 2-stable and globally asymptotically stable (in Lyapunov sense).Under a given condition, the model (1) admits a random endemic equilibrium exponentially p-stable (p ≥ 2) and globally stable.The introduction of a treatment control function in model (1) gives an optimal control problem governed by model (22).The Projection Gradient method permits to determine numerically the optimal control as well as the cost function corresponding to this problem.For example, with a treatment rate equal to τ = 8% and with an initial value equal to u 0 = 0.2 of the function control, we obtain u * = 0.3606, the admissible optimal control and F (u * ) = 2.7073, the cost.Also with τ = 8% and u 0 = 1, we obtain u * = 0. We therefore deduce that the optimal control is without effect when u 0 , the initial dose of the medicines taken by a patient ranges from 80% to 100 %.On the other hand the optimal control is efficient admissible when this initial dose is lower to 50 %.Thanks to the presence of the optimal control in the stochastic model ( 1) of TB, we can reduce considerably and quickly (less than one year) the number of the active infected individuals.As in Fig. 5(t.2) and Fig. 7 (t.3)-(t.4), the trajectory with control of the active infected individuals T is decreasing between 0 and 1 year and becomes null constantly thereafter.This work is therefore a contribution that enters well in the same line of struggle against mortality due to the infections that several governments as well as humanitarian associations advocated so much.

APPENDIX A: PROOF OF THEOREM 1
Let N t = S t +L t +T t +R t , the random variable giving the total number of the population at the time t.We have where ∧ tn θ = t n ∧ θ = min(t n ; θ).Taking the mathematical expectations for all terms of inequations (33), we obtain According to the definition of function V , we have V (X tn∧ θ ) ≥ 0. Hence Biomath 6 (2017), 1705077, http://dx.doi.org/10.11145/j.biomath.2017.05.077

Fig. 1 .
Fig. 1.Diagram of the stochastic model of TB with treatment Biomath 6 (2017), 1705077, http://dx.doi.org/10.11145/j.biomath.2017.05.077 where X n , u n and p n are the present steps of the functions constructed.D. Numerical SimulationsAlgorithm[17]: Stage 1 To choose the arbitrary initial control For n = 0, 1, • • •, let u = u n , to make the buckle iteration of Stage 1 to Stage 5; Stage 2 To use the implicit Euler scheme for the discretization in time of the SDE (22) Stage 3 To use the implicit Euler scheme for the discretization in time of the adjoint equation; (28) Stage 4 To use the iteration scheme (31) of the gradient method to update the controls;
34)Let a set A n = {t n ≤ θ}.Denote by I An (resp.I An ) the indicator function of A n (resp. of the complementary A n ).Thus E[V (X tn∧θ )] = E[V (X tn∧θ )I An ]+E[V (X tn∧θ )I An ]

TABLE I RANDOM
VARIABLES AND PARAMETERS DESCRIPTION Stage 5 Calculate e n = u n − u n+1 .If e n is small enough, then exit.Otherwise; let u = u n+1 repeat the buckle iteration from Stage 2 to Stage 5.

.
dW (t)> .whereξ<G,dW(t)>=(µSt+µLt + (µ + δ T )T t + with = W i i = 1, ..., 10. because W i follow the same law of probability, namely the normal law.We need to show that ifX t = (S t , L t , T t , R t ) T ∈ R 4+ for all t ∈ [0; t [ where t is the explosion time, then we have for P -almost surely (P −as)In fact, if X t ∈ R 4+ for all t ∈ [0; t [, then N t is given such that for P −as.:dN t = (Λ − µN t − δ T T t − ξ <G,dW (t)> )dt ≤ (Λ − µN t )dtAccording to the lemma of Gronwall, we obtain: And as by hypothesis (S 0 , L 0 , T 0 , R 0 ) ∈ Ω i.e.The terms f (t, X t ) and G(t, X t ) of the stochastic model (1) being locally Lipschitz, there is an unique local solution X t = (S t , L t , T t , R t ) T for all t ∈ [0; t [ fixed.Therefore, the unique local solution X t = (S t , L t , T t , R t ) T ∈ R 4 + .In the sequel we show that X t is global solution P −almost surely i.e. t = ∞.Let n 0 > 0, an integer sufficiently large such that (S 0 , L 0 , T 0 , R 0 ) ∈ Set E t = {S t , L t , T t , R t } and for all integer n ≥ n 0 , we define the stop-timest n = inf {H n } with H n = t ∈ [0,t ] : min E t ∈ 0; 1 n or max E t ∈ [n; +∞[ .Let us show that t ∞ = ∞ so that we has t = ∞.For it, let us suppose by absurd that t ∞ < ∞, there is θ > 0 such that for all p ∈]0; 1[ we have P {t ∞ ≤ θ} > p. Consequently, there is an integer n 1 ≥ n 0 such that for all set A n = {t n ≤ θ}, we have P {A n } > p n ≥ n 1 .