RELATED FIELDS
Volume6, Number3, September2016 pp.467–488
AN OPTIMAL MEANREVERSION TRADING RULE UNDER A MARKOV CHAIN MODEL
Jingzhi Tie and Qing Zhang
Department of Mathematics University of Georgia Athens, GA 30602, USA
(Communicated by Jiongmin Yong)
Abstract. This paper is concerned with a meanreversion trading rule. In contrast to most market models treated in the literature, the underlying market is solely determined by a twostate Markov chain. The major advantage of such Markov chain model is its striking simplicity and yet its capability of capturing various market movements. The purpose of this paper is to study an optimal trading rule under such a model. The objective of the problem under consideration is to find a sequence stopping (buying and selling) times so as to maximize an expected return. Under some suitable conditions, explicit solutions to the associated HJ equations (variational inequalities) are obtained.
The optimal stopping times are given in terms of a set of threshold levels. A verification theorem is provided to justify their optimality. Finally, a numerical example is provided to illustrate the results.
1. Introduction. Major market models treated in mathematical finance in recent years can be classified into two broad categories: CoxRossRubinstein’s binomial tree model (BTM) and Brownian motion based models. See related books by Hull [15], Elliott and Kopp [8], Fouque et al. [10], Karatzas and Shreve [17], and Musiela and Rutkowski [20] among others.
The BTM is widely used in option pricing due to its simplicity and yet clear advantage when pricing American type options. However, a main drawback of the BTM is its nonMarkovian nature. The lack of Markovian property makes it diffi cult to work with mathematically, not to mention closedform solutions. To preserve the BTM’s simplicity and enhance its mathematical tractability, a twostate Markov chain model was considered in Zhang [29]. The Markov chain model in [29] appears to be natural for not frequently traded securities such as illiquid stocks. Its capabil ity of capturing this type of markets makes it preferable in related applications. In addition, it can be seen from Example 1 to follow that the Markov chain model is closely related to the traditional meanreversion diffusion when the jump rates of the Markov chain are large. Related Markov chain based models can be found in Van der Hoek and Elliott [23] and Norberg [21]. Van der Hoek and Elliott introduced a stock price model based on stock dividend rates and a Markov chain noise. Norberg used a Markov chain to represent interest rate and considered a market model driven
2010Mathematics Subject Classification. Primary: 93E20, 91G80; Secondary: 49L20.
Key words and phrases. Meanreversion Markovian asset, optimal stopping, variational inequalities.
467
by a Markov chain. In particular, the market model in [21] resembles a GBM in which the ‘drift’ is approximated by the duration between jumps and the ‘diffusion’
is given in terms of jump times. An additional advantage of a Markov chain driven model is that its price is almost everywhere differentiable. Such differentiability is desirable in the optimal control type market analysis proposed by Barmish and Primbs [1]. In connection with dynamic programming problems, the corresponding HamiltonJacobi (HJ) equations are of the first order, which are easier to analyze than those under traditional Brownian motion based models.
Meanreversion models are often used in financial markets to capture price move ments that have the tendency to move towards an ‘equilibrium.’ There are many studies in connection with meanreversion stock returns; see e.g., Cowles and Jones [5], Fama and French [9], and Gallagher and Taylor [11] among others. In addi tion to equity markets, meanreversion models are used for stochastic interest rates (Vasicek [24] and Hull [15]); stochastic volatility (Hafner and Herwartz [14]); and energy markets (Blanco and Soronow [2]. See also related results in option pricing with a meanreversion asset by Bos, Ware and Pavlov [3].
Mathematical trading rules have been studied for many years. For example, Zhang [28] considered a selling rule determined by two threshold levels, a target price and a stoploss limit. Such optimal threshold levels are obtained by solving a set of twopoint boundary value problems. Guo and Zhang [13] studied the optimal selling rule under a model with regime switching. Using a smoothfit technique, they obtained the optimal threshold levels by solving a set of algebraic equations.
These papers are concerned with the selling side of trading in which the underlying price models are of GBM type. Recently, Dai et al. [7] developed a trend following rule based on a conditional probability indicator. They showed that the optimal trading rule can be determined by two threshold curves which can be obtained by solving the associated HJ equations. Similar idea was developed following a confidence interval approach by Iwarere and Barmish [16]. In addition, Merhi and Zervos [18] studied an investment capacity expansion/reduction problem following a dynamic programming approach under a geometric Brownian motion market model.
In connection with meanreversion trading, Zhang and Zhang [27] obtained a buy low and sellhigh policy by charactering the ‘low’ and ‘high’ levels in terms of two threshold levels. Further development along this line can be found in pairs trading in Song and Zhang [22]. Under a meanreversion model, a pair of stocks can be traded simultaneously, when their price divergence exceeds a predetermined level, to bet on their eventual convergence. More details in connection with pairs trading and extensive empirical market tests can be found in Gatev et al. [12] and references therein.
To take advantage of many new features of Markov chain models and explore related meanreversion markets, it is the purpose of this paper to study the corre sponding trading strategies. The objective is to buy and sell the meanreversion security to maximize a reward function. A fixed (commission or slippage) cost will be imposed to each transaction. We study the problem following a dynamic programming approach and establish the associated HJ equations for the value functions. We show that the corresponding optimal stopping (buying and selling) times can be determined by four threshold levels depending on market states. These key levels can be obtained by solving a set of algebraic like equations. In solving the associated HJ equations, hypergeometric series arise naturally in the analysis.
Various boundary conditions and convexity conditions of hypergeometric functions
are involved extensively. We provide a set of sufficient conditions that guarantee the optimality of the trading rules. Finally, to illustrate the results, we present a market backtesting example with the daily closing prices of Dow Jones Industrial Average from 1961 to 1980.
The main contributions of this paper include: (1) Introduction of a new mean reversion model driven by a twostate Markov chain. Such model is very simple in structure and yet powerful enough to capture many important market features.
(2) Development of meanreversion trading strategies that are simple to implement.
The corresponding optimal stopping problem is solved using a smoothfit technique and hypergeometric analysis is involved in a substantial way.
This paper is organized as follows. In§2, we formulate the meanreversion trading problem under consideration. In §3, we study preliminary properties of the value functions. In§4, we consider the associate HJ equations and their solutions. Then in§5, we discuss additional inequalities including the variational inequalities arising from the HJ equations and related convexity conditions. In§6, we give a verification theorem under suitable conditions. A numerical example is considered in§7. Several technical lemmas are postponed and provided in Appendix.
2. Problem formulation. Let{α_{t}: t≥0}denote a twostate Markov chain with state space M={1,2} and the generatorQ=
−λ1 λ1
λ2 −λ2
. Here λ1 and λ2
are positive.
LetSt denote the stock price (or a function of the price) at timetgiven by the equation
dS_{t}
dt =a(b−St) +f(αt), t≥0, (1) where a > 0 is the rate of reversion and b the equilibrium. Here f(1) = f1 > 0 represents the uptick return rate andf(2) =f2<0 the downtick return rate.
Example 2.1. (Convergence to a MeanReversion Diffusion). In this exam ple, we present a simple Markov chain model and demonstrate how it approaches the corresponding meanreversion diffusion. Given ε >0, we considerf1=σ/√
ε, f2 = −σ/√
ε, λ1 = λ2 = 1/ε. Using the asymptotic normality given in Yin and Zhang [26, Theorem 5.9], we can show that the solution to (1) St=S_{t}^{ε} converges weakly to the solution ofdSt=a(b−St)dt+σdWt, whereWtis a standard Brownian motion.
In particular, taking a= 1,b= 2,σ= 0.5, andε= 0.1, we generate the Monte Carlo sample paths of (S_{t}^{ε}, α^{ε}_{t}). One sample is given in (a) of Figure 1. Similarly, we varyεand plot the corresponding sample paths in (b) and (c) of Figure1. It is clear from these pictures that asεgets smaller and smaller, the fluctuation ofα_{t}is more and more rapidly and the correspondingS_{t}^{ε} approaches to a meanreversion diffusion.
Remark 2.1. The main advantage of our model is its simplicity and flexibility in capturing various market conditions. For example, in a typical sideways (trendless) market, the price range can be precisely defined. In our model, such ‘price band’
arises naturally. A candidate for the band can be given by the interval [b+f2/a, b+ f1/a].
We consider the state processXt=St−b. ThenXtsatisfies dXt
dt =−aX_{t}+f(α_{t}), t≥0, (2)
0 0.5 1 1.5 2 2.5 3
0 5 10 15 20
0 0.5 1 1.5 2 2.5 3
0 5 10 15 20
0 0.5 1 1.5 2 2.5 3
0 5 10 15 20
Figure 1. Monte Carlo Sample Paths of (S_{t}^{ε}, α^{ε}_{t}) withε=0.1, 0.01, 0.001, resp.
with initial vector (X0, α0) = (x, α). Let I0 = [f2/a, f1/a]. Then Xt ∈ I0 for all t≥0 ifX0∈ I0.
LetFt ={Xr : r ≤t} denote the filtration generated by Xt. Note that αt is observable andFt={αr: r≤t}. Let
0≤τ_{1}^{b}≤τ_{1}^{s}≤τ_{2}^{b}≤τ_{2}^{s}≤ · · · (3)
denote a sequence of stopping times with respect to {Ft}. A buying decision is made atτ_{n}^{b} and a selling decision atτ_{n}^{s}, n= 1,2, . . ..
We consider the case that the net position at any time can be either long with one share of the stock or flat with no stock position. Leti= 0,1 denote the initial net position. If the initial net position is long (i = 1), then one should sell the stock before acquiring any future shares. The corresponding sequence of stopping times is denoted by Λ_{1} = (τ_{1}^{s}, τ_{2}^{b}, τ_{2}^{s}, τ_{3}^{b}, . . .). Likewise, if the initial net position is flat (i= 0), then one should start to buy a share of the stock. The corresponding sequence of stopping times is denoted by Λ_{0}= (τ_{1}^{b}, τ_{1}^{s}, τ_{2}^{b}, τ_{2}^{s}, . . .).
LetK >0 denote the fixed transaction cost (e.g., slippage and/or commission).
Given the initial state vector (X0, α0) = (x, α), the initial net positioni= 0,1, and the decision sequences (Λ0and Λ1), we consider the corresponding reward functions
Ji(x, α,Λi) =
E
( _{∞} X
n=1
h
e^{−ρτ}^{n}^{s}(Xτ_{n}^{s} −K)−e^{−ρτ}^{n}^{b}(X_{τ}b n+K)i
)
, ifi= 0, E
(
e^{−ρτ}^{1}^{s}(Xτ_{1}^{s}−K) +
∞
X
n=2
h
e^{−ρτ}^{n}^{s}(Xτ_{n}^{s} −K)−e^{−ρτ}^{n}^{b}(X_{τ}b n+K)i
)
, ifi= 1, (4) whereρ >0 is a given discount factor.
In this paper, given random variablesx_{n}, the termE{
∞
X
n=1
x_{n}} is interpreted as
lim sup
N→∞
E
N
X
n=1
x_{n}.
For i = 0,1, let V_{i}(x, α) denote the value functions with the initial vector (X_{0}, α_{0}) = (x, α) and initial net positionsi= 0,1. That is,
V_{i}(x, α) = sup
Λ_{i}
J_{i}(x, α,Λ_{i}). (5)
It follows from (2) thatX_{t}=xe^{−at}+e^{−at}Rt
0e^{ar}f(α_{r})dr. Therefore,J_{i}(x, α,Λ_{i}) is linear inxandVi(x, α) is convex inxfor fixedi= 0,1 andα= 1,2.
In practice, the transaction costK >0 is typically small when compared to all other parameters. Here we assume it is smaller than both (f_{1}/a) and (−f2/a). In addition, bothλ_{1}andλ_{2} are large numbers relative toa. So we also assumeλ_{1}> a andλ_{2}> a.
We summarize conditions to be imposed in this paper:
(A1)f_{1}>0 andf_{2}<0;
(A2)K <min{f_{1}/a,f_{2}/a}.
(A3)λ1> aandλ2> a.
Note that in this paper the stock price X_{t} is differentiable and the value of α_{t} can be given in terms of the derivative ofXtwith respect tot.
3. Properties of the value functions.
Lemma 3.1. The following inequalities hold.
(a) 0≤V_{0}(x, α)≤
ρ+ 2a ρa
max{f_{1},f_{2}}, for all(x, α)∈ I_{0}× M.
(b) −K ≤ V0(x, α)−V1(x, α) +x ≤ K, for all (x, α) ∈ I0× M. Therefore, V1(x, α)is also bounded onI0× M.
Proof. It is clear thatV0(x, α)≥0. To show it is bounded from above, we first note that
d(e^{−ρt}X_{t})
dt =e^{−ρt}(−(ρ+a)X_{t}+f(α_{t})).
Given Λ0, integrate both sides over [τ_{n}^{b}, τ_{n}^{s}] to obtain e^{−ρτ}^{n}^{s}Xτ_{n}^{s}−e^{−ρτ}^{n}^{b}X_{τ}b
n= Z τ_{n}^{s}
τ_{n}^{b}
e^{−at}(−(ρ+a)Xt+f(αt))dt.
It follows that, for (x, α)∈ I0× M, J0(x, α,Λ0) =E
∞
X
n=1
h
e^{−ρτ}^{n}^{s}(Xτ_{n}^{s}−K)−e^{−ρτ}^{n}^{b}(X_{τ}b n+K)i
≤E
∞
X
n=1
h
e^{−ρτ}^{n}^{s}Xτ_{n}^{s}−e^{−ρτ}^{n}^{b}X_{τ}b n
i
≤E
∞
X
n=1
Z τ_{n}^{s} τ_{n}^{b}
e^{−at}(−(ρ+a)Xt+f(αt))dt
≤E Z ∞
0
e^{−at}((ρ+a)Xt+f(αt))dt
≤
ρ+ 2a ρa
max{f_{1},f_{2}}.
This implies (a).
To show thatV_{0}(x, α)−V_{1}(x, α) +x≤K, or V_{1}(x, α)≥V_{0}(x, α) +x−K, we note that, for a given Λ_{1} withτ_{1}^{s}= 0,
V_{1}(x, α)≥J_{1}(x, α,Λ_{1}) +x−K.
Since the rest stopping times in Λ1are arbitrary, it follows thatV1(x, α)≥V0(x, α)+
x−K. Similarly, we can show the second inequality in (b).
4. HJ equations. LetAdenote the generator of (Xt, αt), i.e., for any differentiable functionsh(x, i),i= 1,2,
( Ah(x,1) = (−ax+f1)h^{0}(x,1) +λ1(h(x,2)−h(x,1)), Ah(x,2) = (−ax+f2)f2h^{0}(x,2) +λ2(h(x,1)−h(x,2)),
whereh^{0}denotes the derivative ofhwith respect tox. The associated HJ equations should have the form:
min{ρv_{0}(x, α)− Av_{0}(x, α), v_{0}(x, α)−v_{1}(x, α) +x+K}= 0,
min{ρv_{1}(x, α)− Av_{1}(x, α), v_{1}(x, α)−v_{0}(x, α)−x+K}= 0, (6) forα= 1,2.
In this paper, our goal is to find the value functions by solving these HJ equations with all value function properties satisfied. Then we proceed to show that such functionsv_{i}(x, α) are equal to the value functions.
4.1. Solving (ρ− A)vi(x, α) = 0. To solve these HJ equations, for each fixed i= 0,1, we consider equations when ρvi(x, α)− Avi(x, α) = 0,α= 1,2. Using the generatorA, we can write
(ρ+λ_{1})v_{i}(x,1) = (−ax+f_{1})v^{0}_{i}(x,1) +λ_{1}v_{i}(x,2),
(ρ+λ_{2})v_{i}(x,2) = (−ax+f_{2})v^{0}_{i}(x,2) +λ_{2}v_{i}(x,1). (7) For each fixed i = 0,1 and α = 1,2, let y_{α}(x) = v_{i}(x, α). Then, using the first equation, we can writey_{2} in terms ofy_{1}:
y_{2}(x) = 1 λ1
[(ρ+λ_{1})y_{1}(x) + (ax−f_{1})y^{0}_{1}(x)].
Its derivative with respect toxis given by y^{0}_{2}(x) = 1
λ1
[(ρ+λ1+a)y_{1}^{0}(x) + (ax−f1)y_{1}^{00}(x)].
Substitute these into the second equation in (7) to obtain (ax−f_{1})(ax−f_{2})y^{00}_{1}(x) + [(ρ+λ_{2})(ax−f_{1})
+(ρ+λ_{1}+a)(ax−f_{2})]y_{1}^{0}(x) +ρ(ρ+λ_{1}+λ_{2})y_{1}(x) = 0.
To simplify this equation, let
ξ=ax−f2
f_{1}−f_{2} andu(ξ) =y1(x). Then, it follows that
ax−f_{2}= (f_{1}−f_{2})ξ and ax−f_{1}= (f_{1}−f_{2})(ξ−1).
Moreover, applying the chain rule to obtain y_{1}^{0}(x) =
a f_{1}−f_{2}
u^{0}(ξ) and y_{1}^{00}(x) = a
f_{1}−f_{2} ^{2}
u^{00}(ξ).
This reduces the equation fory1(x) into an equation foru(ξ) as
a^{2}ξ(ξ−1)u^{00}(ξ) +a[(ρ+λ2)(t−1) + (ρ+λ1+a)t]u^{0}(ξ) +ρ(ρ+λ1+λ2)]u(ξ) = 0.
Next, we introduce the additional parameters γ1=ρ
a, γ2= ρ+λ_{1}+λ_{2}
a , γ3=ρ+λ_{2} a .
Then, we can write the above differential equation in terms of (γ_{1}, γ_{2}, γ_{3}) as ξ(1−ξ)u^{00}(ξ) + [γ_{3}−(γ_{1}+γ_{2}+ 1)t]u^{0}(ξ)−γ_{1}γ_{2}u(ξ) = 0. (8) This is exactly the classical hypergeometric equation. Related properties of hyper geometric functions can be found in [19] and [25].
For any real numbersηi,i= 1,2,3, let F(η1, η2;η3, ξ) =
∞
X
n=0
(η1)n(η2)n
n!(η3)n
ξ^{n}, where
(η)0= 1 and (η)n=η(η+ 1)· · ·(η+n−1) = Γ(η+n) Γ(η) ,
for anyη =η_{i}. Then, for 0< ξ <1, two independent solutions of (8) can be given by
F ρ
a,ρ+λ1+λ2
a ;ρ+λ2
a , ξ
andF ρ
a,ρ+λ1+λ2
a ;ρ+λ2+a a ,1−ξ
.
Let
F1(x) =F ρ
a,ρ+λ1+λ2
a ;ρ+λ2
a ,ax−f2
f_{1}−f_{2}
, Fe1(x) =F
ρ
a,ρ+λ1+λ2
a ;ρ+λ2+a
a ,1−ax−f2
f1−f2
.
(9)
We consider these solutions over a neighborhood inI0 containing the lower end f2/a. Then any solution over this interval can be written as a linear combination of F1 and Fe1, i.e., for some constants C1 and Ce1, vi(x,1) =C1F1(x) +Ce1Fe1(x), i= 0,1. In particular, we need this solution to be bounded nearf2/a. This requires Ce1= 0 becauseFe1(x)→ ∞asx→f2/a(see Lemma 8.1in Appendix).
Therefore, on the neighborhood ofI0containingf_{2}/awe can assume thatv_{1}(x,1)
=C_{1}F_{1}(x). Using this solution, we can writev_{1}(x,2) in terms ofF_{1}(x):
v_{1}(x,2) = C_{1} λ1
[(ρ+λ_{1})F_{1}(x) + (ax−f_{1})F_{1}^{0}(x)] :=C_{1}G_{1}(x). (10) Alternatively, we can use the second equation in (7) and writey1in terms ofy2:
y1(x) = 1
λ_{2}[(ρ+λ2)y2(x) + (ax−f2)y^{0}_{2}(x)]
and its derivative in terms ofy^{0}_{2} andy^{00}_{2} y^{0}_{1}(x) = 1
λ2
[(ρ+λ2+a)y_{2}^{0}(x) + (ax−f2)y_{2}^{00}(x)].
Substituting these into the first equation in (7) to obtain (ax−f_{1})(ax−f_{2})y^{00}_{2}(x) + [(ρ+λ_{1})(ax−f_{2})
+(ρ+λ_{2}+a)(ax−f_{1})]y_{1}^{0}(x) +ρ(ρ+λ_{1}+λ_{2})y_{2}(x) = 0.
Similarly, letw(ξ) =y2(x) with
ξ=f1−ax f1−f2
,
Then, we can write the above differential equation in terms ofw(ξ):
ξ(1−ξ)w^{00}(ξ) + [γ_{3}^{0} −(γ_{1}+γ_{2}+ 1)t]w^{0}(ξ)−γ_{1}γ_{2}w(ξ) = 0, withγ_{3}^{0} = (ρ+λ_{1})/a. Its two independent solutions can be given by
F ρ
a,ρ+λ1+λ2
a ;ρ+λ1
a , ξ
andF ρ
a,ρ+λ1+λ2
a ;ρ+λ1+a a ,1−ξ
. Let
F2(x) =F ρ
a,ρ+λ1+λ2
a ;ρ+λ1
a ,f1−ax f_{1}−f_{2}
, Fe2(x) =F
ρ
a,ρ+λ1+λ2
a ;ρ+λ2+a
a ,1−f1−ax f1−f2
.
(11)
Similarly, we consider these solutions over a neighborhood in I0 containing the upper end f1/a. Again, any solution can be written as a linear combination of F2
andFe2, i.e., for some constantsC2 andCe2,vi(x,2) =C2F2(x) +Ce2Fe2(x),i= 0,1.
We now need this solution to be bounded nearf1/a, which impliesCe2= 0 because Fe_{2}(x)→ ∞asx→f_{1}/a.

f_{1} a f_{2}
a
b2 b1
v0(1) =v1(1)−x−K
v0(1) =v1(1)−x−K ρv0(1) =Av_{0}(1)
v_{0}(2) =v_{1}(2)−x−K ρv_{0}(2) =Av_{0}(2) ρv_{0}(2) =Av_{0}(2)

f_{1} a f_{2}
a
s_{1} s_{2}
ρv1(1) =Av_{1}(1) ρv1(1) =Av_{1}(1)
ρv_{1}(2) =Av_{1}(2) v_{1}(2) =v_{0}(2)+x−K v_{1}(2) =v_{0}(2)+x−K v1(1) =v0(1)+x−K
Figure 2. Buying Thresholds: b1 andb2; Selling Thresholds: s1 ands2. Therefore, on the neighborhood ofI0containingf1/awe can assume thatv0(x,2)
=C2F2(x). Using this solution, we have v_{0}(x,1) = C_{2}
λ2
[(ρ+λ_{2})F_{2}(x) + (ax−f_{2})F_{2}^{0}(x)] :=C_{2}G_{2}(x). (12) Lemma 4.1. The solutions of (ρ− A)vi(x, α) = 0 can be given as
v0(x,1) =C2G1(x), v0(x,2) =C2F2(x), v1(x,1) =C1F1(x), v1(x,2) =C1G1(x), whereF_{1},F_{2},G_{1}, andG_{2} are defined in (9), (11), (10), and (12), respectively.
4.2. Order of threshold levels. Ifαt= 1, letb1 denote the buying threshold (to buy whenXt< b1) ands1the selling threshold (to sell whenXt> s1), respectively.
Likewise, if αt = 2, let b2 and s2 denote the corresponding buying and selling thresholds, respectively.
In the purely selling case, it is shown in [29] thats_{1}> s_{2}. Intuitively,b_{1} should be greater thanb_{2} as well.
In addition, in this paper, we consider the case whenb_{1}< s_{2}.
The opposite inequalityb_{1}≥s_{2}means to buy whenα_{t}= 1 andX_{t}< b_{1}and sell as soon as αt jumps to 2 andXt> s2. This case rarely arises in practice because the jump rates λ1 and λ2 are typically large so thatαt switches between 1 and 2 frequently.
In view of these, we consider the threshold levels having the following order f2
a < b2< b1< s2< s1<f1
a.
The corresponding equalities in the HJ equations in (6) are marked in Figure2.
Using convexity of the value functions and variational inequalities in (6), it is shown in Lemma8.2(Appendix) that
b2= f2−ρK
ρ+a ands1=f1+ρK ρ+a .
It is easy to check, under Assumption (A2), thatb2 and s1 given above indeed satisfy the inequalitiesb_{2}> f_{2}/aands_{1}< f_{1}/a.
4.3. Solving (ρ− A)v1(x,1) = 0 with v_{1}(x,2) = v_{0}(x,2) +x−K and (ρ− A)v0(x,2) = 0with v_{0}(x,1) =v_{1}(x,1)−x−K. Forx∈ I0, let
g_{1}(x) =
as1−f1
ax−f_{1} ^{ρ+λ}_{a}^{1}
andg_{2}(x) =
ab2−f2
ax−f_{2} ^{ρ+λ}_{a}^{2}
.
Next, we solve the HJ equations on [s2, s1]. Note that on this interval,v0(x,1) = C2G2(x),v0(x,2) =C2F2(x), andv1(x,2) =v0(x,2) +x−K=C2F2(x) +x−K.
To solveρv1(x,1) =Av1(x,1) on this interval, we write it as (ax−f_{1})v^{0}_{1}(x,1) + (ρ+λ_{1})v_{1}(x,1) =λ_{1}v_{1}(x,2).
Differentiate (1/g_{1}(x))v_{1}(x,1) and then integrate froms_{1} toxto obtain v_{1}(x,1) =v_{1}(s_{1},1)g_{1}(x)−λ_{1}(ax−f_{1})^{−}^{ρ+λ}^{a}^{1}
Z x s_{1}
(f_{1}−as)^{ρ+λ}^{a}^{1}^{−1}v_{1}(s,2)ds.
Recall thatv1(x,2) =C2F2(x) +x−K. We write
v1(x,1) =g1(x)A+k1(x)C2+k2(x), whereA=v1(s1,1) and
k1(x) =−λ1(f1−ax)^{−}^{ρ+λ}^{a}^{1} Z x
s1
(f1−as)^{ρ+λ}^{a}^{1}^{−1}F2(s)ds, k2(x) =−λ1(f1−ax)^{−}^{ρ+λ}^{a}^{1}
Z x s1
(f1−as)^{ρ+λ}^{a}^{1}^{−1}(s−K)ds.
Similarly, we solve the HJ equations on [b2, b1]. Note that on this interval, v1(x,1) = C1F1(x), v1(x,2) = C1G1(x), and v0(x,1) = v1(x,1)−x−K = C1F1(x)−x−K. To solve ρv0(x,2) = Av0(x,2) on this interval, we first write it as
(ax−f2)v^{0}_{0}(x,2) + (ρ+λ2)v0(x,2) =λ2v0(x,1).
Differentiate (1/g2(x))v0(x,2) and then integrate over [b2, x] to obtain v0(x,2) =v0(b2,2)g2(x) +λ2(ax−f2)^{−}^{ρ+λ}^{a}^{2}
Z x b2
(as−f2)^{ρ+λ}^{a}^{2}^{−1}v0(s,1)ds.
Recall thatv0(x,1) =C1F1(x)−x−K. We write
v_{0}(x,2) =g_{2}(x)B+h_{1}(x)C_{1}+h_{2}(x), whereB=v_{0}(b_{2},2) and
h_{1}(x) =λ_{2}(ax−f_{2})^{−}^{ρ+λ}^{a}^{2} Z x
b_{2}
(as−f_{2})^{ρ+λ}^{a}^{2}^{−1}F_{1}(s)ds, h2(x) =−λ2(ax−f2)^{−}^{ρ+λ}^{a}^{2}
Z x b_{2}
(as−f2)^{ρ+λ}^{a}^{2}^{−1}(s+K)ds.
Using the power series form ofF1(x) =
∞
X
n=0
pn
ax−f2
f1−f2
n
, we can write
h1(x) =λ2
∞
X
n=0
pn
ρ+λ2+an
ax−f2
f1−f2
n
−g2(x)
ab2−f2
f1−f2
n . Similarly, we have
h_{2}(x) =−λ_{2} a
1
ρ+λ2+a((ax−f_{2})−(ab_{2}−f_{2})g_{2}(x)) +f_{2}+aK ρ+λ2
(1−g_{2}(x))
.
Likewise, writeF2(x) =
∞
X
n=0
qn
f_{1}−ax f1−f2
n
to obtain
k_{1}(x) =−λ1
∞
X
n=0
q_{n} ρ+λ1+an
g_{1}(x)
f_{1}−as_{1} f1−f2
n
−
f_{1}−ax f1−f2
n
and
k2(x) =λ1
a
− 1
ρ+λ_{1}+a((f1−ax)−(f1−as1)g1(x)) +f1−aK
ρ+λ_{1} (1−g1(x))
. 4.4. Smoothfit conditions. Note that the convexity ofvi(x, α) inxonI0implies that they are continuous in the interior ofI0. The continuity of these functions at the threshold levels leads to the following conditions:
x=b_{2} : C_{1}G_{1}(b_{2})−b_{2}−K=g_{2}(b_{2})B+h_{1}(b_{2})C_{1}+h_{2}(b_{2});
x=b_{1} :
C_{1}F_{1}(b_{1})−b_{1}−K=C_{2}G_{2}(b_{1}),
g_{2}(b_{1})B+h_{1}(b_{1})C_{1}+h_{2}(b_{1}) =C_{2}F_{2}(b_{1}), x=s_{2} :
C1F1(s2) =g1(s2)A+k1(s2)C2+k2(s2), C1G1(s2) =C2F2(s2) +s2−K,
x=s1 : C2G2(s1) +s1−K=g1(s1)A+k1(s1)C2+k2(s1).
(13)
Recall thatb2= (f2−ρK)/(ρ+a) ands1= (f1+ρK)/(ρ+a). Using the first and last equations in (13), we can write
B=
G1(b2)−h1(b2) g_{2}(b_{2})
C1−b2+K+h2(b2)
g_{2}(b_{2}) =:B1C1+B2
A=
G2(s1)−k1(s1) g_{1}(s_{1})
C2+s1−K−k2(s1)
g_{1}(s_{1}) =:A1C2+A2.
Then, we can write the second and the third equations in terms ofC1andC2: F1(b1) −G2(b1)
g2(b1)B1+h1(b1) −F2(b1)
C1
C2
=
b1+K
−g2(b1)B2−h2(b1)
. Similarly, we can write the fourth and the fifth equations as follows:
G_{1}(s_{2}) −F2(s_{2}) F_{1}(s_{2}) −[g1(s_{2})A_{1}+k_{1}(s_{2})]
C_{1} C_{2}
=
s_{2}−K g_{1}(s_{2})A_{2}+k_{2}(s_{2})
. If the above 2×2 matrices are invertible, then we can eliminate C1 and C2 and obtain
F1(b1) −G2(b1) g2(b1)B1+h1(b1) −F2(b1)
−1
b1+K
−g2(b1)B2−h2(b1)
=
G1(s2) −F2(s2) F1(s2) −[g1(s2)A1+k1(s2)]
^{−1}
s2−K g1(s2)A2+k2(s2)
.
(14)
The solutions to the HJ equations (6) should have the form:
v0(x,1) =
C1F1(x)−x−K iff2/a≤x≤b1, C2G2(x) ifb1< x≤f1/a, v0(x,2) =
C1G1(x)−x−K iff2/a≤x≤b2, g2(x)B+h1(x)C1+h2(x) ifb2< x≤b1, C2F2(x) ifb1< x≤f1/a, v1(x,1) =
C_{1}F_{1}(x) iff_{2}/a≤x≤s_{2}, g_{1}(x)A+k_{1}(x)C_{2}+k_{2}(x) ifs_{2}< x≤s_{1}, C_{2}G_{2}(x) +x−K ifs_{1}< x≤f_{1}/a, v_{1}(x,2) =
C_{1}G_{1}(x) iff_{2}/a≤x≤s_{2}, C_{2}F_{2}(x) +x−K ifs_{2}< x≤f_{1}/a,
(15)
In Figure 3, these functions are so labelled in their corresponding intervals in Figure2.

f_{1} a f_{2}
a
b2 b1
v0(1) =C1F1−x−K
v0(1) =C1F1−x−K v0(1) =C2G2
v_{0}(2) =C_{1}G_{1}−x−K v_{0}(2)=g_{2}B+h_{1}C_{1}+h_{2} v_{0}(2) =C_{2}F_{2}

f_{1} a f_{2}
a
s_{2} s_{1}
v1(1) =C1F1 v1(1)=g1A+k1C2+k2
v_{1}(2) =C_{1}G_{1} v_{1}(2) =C_{2}F_{2}+x−K v_{1}(2) =C_{2}F_{2}+x−K v1(1) =C2G2+x−K
Figure 3. HJ equation solutions.
5. Variational inequalities and convexity. In this section, we consider the variational inequalities in (6) and related convexity inequalities.
5.1. Variational inequalities. Note that with the functions given in (15), all variational inequalities in (6) have to be satisfied, i.e., for (x, α)∈ I0× M,
ρv0(x, α)− Av0(x, α)≥0, v0(x, α)−v1(x, α) +x+K≥0, ρv_{1}(x, α)− Av1(x, α)≥0, v_{1}(x, α)−v_{0}(x, α)−x+K≥0.
(16)
In what follows, we consider these inequalities on each intervals: [f_{2}/a, b_{2}], [b_{2}, b_{1}], [b1, s2], [s2, s1], and [s1, f1/a].
First, we show that these inequalities on [f2/a, b2] are automatically satisfied.
Note that on [f2/a, b2], we have
v_{0}(x,1) =v_{1}(x,1)−x−K, v_{0}(x,2) =v_{1}(x,2)−x−K, ρv_{1}(x,1) =Av_{1}(x,1), ρv_{1}(x,2) =Av_{1}(x,2).
(17)
We need to show
ρv0(x,1)≥ Av0(x,1), ρv0(x,2)≥ Av0(x,2), v1(x,1)≥v0(x,1) +x−K, v_{1}(x,2)≥v_{0}(x,2) +x−K.
(18)
It is easy to see that the last two inequalities follow directly from the first two equalities in (17). To see the first inequalityρv_{0}(x,1)≥ Av_{0}(x,1), note that it can be written in terms ofv1(x,1)andv1(x,2):
ρ(v1(x,1)−x−K)≥(f1−ax)(v^{0}_{1}(x,1)−1) +λ1(v1(x,2)−v1(x,1)), which is equivalent to
−ρ(x+K)≥ −(f1−ax), forx < b2,
because ρv1(x,1) = Av1(x,1). The above inequality is clearly satisfied because b2= (f2−ρK)/(ρ+a) andf2< f1.
Similarly, on [s1, f1/a], we can uses1= (f1+ρK)/(ρ+a) and show that all VIs in (16) hold.
Next, we consider the VIs on [b2, b1]. We have
v0(x,1) =v1(x,1)−x−K, ρv0(x,2) =Av0(x,2), ρv1(x,1) =Av1(x,1), ρv1(x,2) =Av1(x,2).
(19)
We need the following inequalities to hold:
ρv0(x,1)≥ Av0(x,1), v0(x,2)≥v1(x,2)−x−K, v1(x,1)≥v0(x,1) +x−K, v_{1}(x,2)≥v_{0}(x,2) +x−K.
(20)
The third inequality is automatically satisfied. The second and forth ones can be written as:
−K≤v0(x,2)−v1(x,2) +x≤K.
Finally, the first inequality in (20) can be given as v0(x,2)−v1(x,2) +x≤ 1
λ1
[f1−ρK−(ρ+a)x]−K.
We can work out all needed inequalities this way on all other intervals and sum marize these inequalities as follows:
On [b2, b1] : −K≤W2(x)≤min
K, 1 λ1
[f1−ρK−(ρ+a)x]−K
, On [b_{1}, s_{2}] : −K≤W_{1}(x)≤K and −K≤W_{2}(x)≤K,
On [s_{2}, s_{1}] : max
−K, 1 λ2
[f_{2}+ρK−(ρ+a)x] +K
≤W_{1}(x)≤K, (21)
whereW1(x) =v0(x,1)−v1(x,1) +xandW2(x) =v0(x,2)−v1(x,2) +x.
5.2. Convexity. Recall that the value functions are convex onI0. We expect the solutions to the HJ equations in (6) to be also convex.
Recall also that
G_{i}(x) = 1 λi
((ρ+λ_{i})F_{i}(x) + (ax−f_{i})F_{1}^{0}(x)), fori= 1,2.
Lemma 5.1. F1,F2,G1,G2 are convex on[f2/a, f1/a].
Proof. Clearly, both F1 and F2 are convex because their second derivatives are positive.
To show the convexities ofG1 andG2, we first consider g(ξ) = (γ_{1}+γ_{2}−γ_{3})u(ξ)−(1−ξ)du
dξ(ξ),
with u(ξ) = F(γ_{1}, γ_{2};γ_{3}, ξ). Multiply its both sides by (−(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}^{−1}) to obtain
−(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}^{−1}g(ξ) =−(γ1+γ2−γ3)(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}^{−1}u(ξ)+(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}du dξ(ξ).
The right hand side of the above equation is equal to d
dξ
(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}F(γ_{1}, γ_{2};γ_{3}, ξ) .
We next apply the formula d^{n}
dξ^{n}
(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}F(γ1, γ2;γ3, ξ)
=(γ_{3}−γ_{1})_{n}(γ_{3}−γ_{2})_{n} (γ3)n
(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}^{−n}F(γ1, γ2;γ3+n, ξ), withn= 1 and get
−(1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}^{−1}g(ξ) =(γ3−γ1)(γ3−γ2)
γ_{3} (1−ξ)^{γ}^{1}^{+γ}^{2}^{−γ}^{3}^{−1}F(γ1, γ2;γ3+ 1, ξ).
Hence we conclude that
g(ξ) =−(γ3−γ1)(γ3−γ2) γ3
F(γ1, γ2;γ3+ 1, ξ).
Foru(ξ) =F_{ρ}
a,^{ρ+λ}^{1}_{a}^{+λ}^{2};^{ρ+λ}_{a} ^{2}, ξ
, the correspondingg(ξ) is given by g(ξ) = λ2λ1
ρ+λ_{2}F ρ
a,ρ+λ1+λ2
a ;ρ+λ2
a + 1, ξ
. Then the convexity ofF_{ρ}
a,^{ρ+λ}^{1}_{a}^{+λ}^{2};^{ρ+λ}_{a}^{2} + 1, ξ
implies thatg(ξ) is convex. It is easy to check with change of variablet= (ax−f2)/(f1−f2) thatG1(x) = (λ1/a)g(ξ).
SoG1is convex. Similarly, we can show the convexity ofG2. Lemma 5.2. h1(x)andk1(x)are convex.
Proof. Here, we only show the convexity ofh1(x). The convexity fork1(x) can be given in a similar way. Recall that
h1(x) =λ2(ax−f2)^{−}^{ρ+λ}^{a}^{2} Z x
b2
(as−f2)^{ρ+λ}^{a}^{2}^{−1}F1(s)ds, where
F1(x) =F
γ1, γ2;γ3,ax−f2
f_{1}−f_{2}
withγ1= ρ
a, γ2=ρ+λ1+λ2
a andγ3= ρ+λ2
a . Let
w1(ξ) =h1(x) =h1
(f_{1}−f_{2})t+f_{2} a
, witht= ax−f_{2} f1−f2
. It follows that
w_{1}(ξ) = λ_{2} aξ^{−γ}^{3}
Z ξ eb_{2}
r^{γ}^{3}^{−1}F(γ_{1}, γ_{2};γ_{3}, r)dr, whereeb2= (ab2−f2)/(f1−f2). Note that
Z
F(α, β;γ, ξ)ξ^{γ−1}dt= ξ^{γ}
γ F(α, β;γ+ 1, ξ) +k.
Apply the above formula, we obtain w1(ξ) = λ2
aγ3
h
F(γ1, γ2;γ3+ 1, ξ)−F(γ1, γ2;γ3+ 1,eb2)i . Recall the derivative formula for hypergeometric function:
d^{m}
dξ^{m}F(α, β;γ, ξ) =(α)_{m}(β)_{m} (γ)m
F(α+m, β+m;γ+m, ξ).
It follows that
w^{00}_{1}(ξ) =λ2γ1(γ1+ 1)γ2(γ2+ 1)
aγ_{3}(γ_{3}+ 1)(γ_{3}+ 2) F(γ1+ 2, γ2+ 2;γ3+ 3, ξ).
In view of this and the convexity of the hypergeometric functions with positive parameters, the convexity ofw_{1}(ξ) follows on [0,1]. So is the convexity ofh_{1}(x).
Lemma 5.3. g2B+h1C1+h2 andg1A+k1C2+k2 are convex provided that C_{1}≥0, C_{2}≥0, A+A_{0}≥0, B+B_{0}≥0, (22) where
A0= λ1
a
f1−as1
ρ+λ_{1}+a−f1−aK ρ+λ_{1}
andB0= λ2
a
ab2−f2
ρ+λ_{2}+a+f2+aK ρ+λ_{2}
. Proof. It is easy to see that bothg1(x) andg2(x) are convex. In addition, note that g1A+k2 can be written as g1(A+A0)+ a linear function. SoA+A0>0 implies that g1A+k2 is convex. Then in view of the convexity ofk1(x) in Lemma5.2, it follows that g1A+k1C2+k2 is convex. Similarly, we can show the convexity of
g_{2}B+h_{1}C_{1}+h_{2}.
To guarantee the convexity of the functionsv_{i}(x, α), it is sufficient to require the convexity ofv0(x,1) atx=b1and v1(x,2) atx=s2.
The convexity ofv0(x,1) atx=b1 is equivalent to (v_{0}(b_{1},1))^{0}_{−}≤(v_{0}(b_{1},1))^{0}_{+}. Therefore, we need
C1F_{1}^{0}(b1)−1≤C2G^{0}_{2}(b1). (23) Similarly, The convexity ofv1(x,2) atx=s2requires
C_{1}G^{0}_{1}(s_{2})≤C_{2}F_{2}^{0}(s_{2}) + 1. (24) Theorem 5.1. Assume that there exist b1 and s2 such that (a) the equalities in (14) hold with invertible matrices; (b) the additional VI inequalities in (21) hold;
(c)the inequalities in(22)hold; (d)the convexity inequalities in(23)and(24)hold.
Then, vi(x, α) defined in(15)satisfy the HJ equations (6).
6. A verification theorem.
Theorem 6.1. Assume the conditions of Theorem 5.1. Then, (a)v(x, i) =V(x, i),i= 1,2.
(b)Moreover, let
Db = (b1, f1/a)× {1} ∪(b2, f1/a)× {2}, Ds= (f2/a, s1)× {1} ∪(f2/a, s2)× {2}
denote the continuation regions. The set of stopping times in Λ_{0} and Λ_{1} can be defined as the first exit times from these continuation regions in sequence. Let Λ^{∗}_{0} = (τ_{1}^{b∗}, τ_{1}^{s∗}, τ_{2}^{b∗}, τ_{2}^{s∗}, . . .), where τ_{1}^{b∗} = inf{t : (X_{t}, α_{t}) 6∈ D_{b}}, τ_{1}^{s∗} = inf{t ≥ τ_{1}^{b∗} : (X_{t}, α_{t}) 6∈ D_{s}},.., τ_{n+1}^{b∗} = inf{t ≥ τ_{n}^{s∗} : (X_{t}, α_{t}) 6∈ D_{b}}, τ_{n+1}^{s∗} = inf{t ≥ τ_{n+1}^{b∗} : (X_{t}, α_{t})6∈D_{s}},..., and let Λ^{∗}_{1} = (τ_{1}^{s∗}, τ_{2}^{b∗}, τ_{2}^{s∗}, . . .), where τ_{1}^{s∗}= inf{t≥0 : (Xt, αt) 6∈ Ds},.., τ_{n+1}^{b∗} = inf{t ≥ τ_{n}^{s∗} : (Xt, αt) 6∈ Db}, τ_{n+1}^{s∗} = inf{t ≥ τ_{n+1}^{b∗} : (X_{t}, α_{t})6∈D_{s}},... Ifv_{0}(x)≥0 andτ_{n}^{s∗}→ ∞, a.s., then, Λ^{∗}_{0} andΛ^{∗}_{1} are optimal.