The objective of the problem under consideration is to find a sequence stopping (buying and selling) times so as to maximize an expected return

RELATED FIELDS

Volume6, Number3, September2016 pp.467–488

AN OPTIMAL MEAN-REVERSION TRADING RULE UNDER A MARKOV CHAIN MODEL

Jingzhi Tie and Qing Zhang

Department of Mathematics University of Georgia Athens, GA 30602, USA

(Communicated by Jiongmin Yong)

Abstract. This paper is concerned with a mean-reversion trading rule. In contrast to most market models treated in the literature, the underlying market is solely determined by a two-state Markov chain. The major advantage of such Markov chain model is its striking simplicity and yet its capability of capturing various market movements. The purpose of this paper is to study an optimal trading rule under such a model. The objective of the problem under consideration is to find a sequence stopping (buying and selling) times so as to maximize an expected return. Under some suitable conditions, explicit solutions to the associated HJ equations (variational inequalities) are obtained.

The optimal stopping times are given in terms of a set of threshold levels. A verification theorem is provided to justify their optimality. Finally, a numerical example is provided to illustrate the results.

1. Introduction. Major market models treated in mathematical finance in recent years can be classified into two broad categories: Cox-Ross-Rubinstein’s binomial tree model (BTM) and Brownian motion based models. See related books by Hull [15], Elliott and Kopp [8], Fouque et al. [10], Karatzas and Shreve [17], and Musiela and Rutkowski [20] among others.

The BTM is widely used in option pricing due to its simplicity and yet clear advantage when pricing American type options. However, a main drawback of the BTM is its non-Markovian nature. The lack of Markovian property makes it diffi- cult to work with mathematically, not to mention closed-form solutions. To preserve the BTM’s simplicity and enhance its mathematical tractability, a two-state Markov chain model was considered in Zhang [29]. The Markov chain model in [29] appears to be natural for not frequently traded securities such as illiquid stocks. Its capabil- ity of capturing this type of markets makes it preferable in related applications. In addition, it can be seen from Example 1 to follow that the Markov chain model is closely related to the traditional mean-reversion diffusion when the jump rates of the Markov chain are large. Related Markov chain based models can be found in Van der Hoek and Elliott [23] and Norberg [21]. Van der Hoek and Elliott introduced a stock price model based on stock dividend rates and a Markov chain noise. Norberg used a Markov chain to represent interest rate and considered a market model driven

2010Mathematics Subject Classification. Primary: 93E20, 91G80; Secondary: 49L20.

Key words and phrases. Mean-reversion Markovian asset, optimal stopping, variational inequalities.

467

by a Markov chain. In particular, the market model in [21] resembles a GBM in which the ‘drift’ is approximated by the duration between jumps and the ‘diffusion’

is given in terms of jump times. An additional advantage of a Markov chain driven model is that its price is almost everywhere differentiable. Such differentiability is desirable in the optimal control type market analysis proposed by Barmish and Primbs [1]. In connection with dynamic programming problems, the corresponding Hamilton-Jacobi (HJ) equations are of the first order, which are easier to analyze than those under traditional Brownian motion based models.

Mean-reversion models are often used in financial markets to capture price move- ments that have the tendency to move towards an ‘equilibrium.’ There are many studies in connection with mean-reversion stock returns; see e.g., Cowles and Jones [5], Fama and French [9], and Gallagher and Taylor [11] among others. In addi- tion to equity markets, mean-reversion models are used for stochastic interest rates (Vasicek [24] and Hull [15]); stochastic volatility (Hafner and Herwartz [14]); and energy markets (Blanco and Soronow [2]. See also related results in option pricing with a mean-reversion asset by Bos, Ware and Pavlov [3].

Mathematical trading rules have been studied for many years. For example, Zhang [28] considered a selling rule determined by two threshold levels, a target price and a stop-loss limit. Such optimal threshold levels are obtained by solving a set of two-point boundary value problems. Guo and Zhang [13] studied the optimal selling rule under a model with regime switching. Using a smooth-fit technique, they obtained the optimal threshold levels by solving a set of algebraic equations.

These papers are concerned with the selling side of trading in which the underlying price models are of GBM type. Recently, Dai et al. [7] developed a trend following rule based on a conditional probability indicator. They showed that the optimal trading rule can be determined by two threshold curves which can be obtained by solving the associated HJ equations. Similar idea was developed following a confidence interval approach by Iwarere and Barmish [16]. In addition, Merhi and Zervos [18] studied an investment capacity expansion/reduction problem following a dynamic programming approach under a geometric Brownian motion market model.

In connection with mean-reversion trading, Zhang and Zhang [27] obtained a buy- low and sell-high policy by charactering the ‘low’ and ‘high’ levels in terms of two threshold levels. Further development along this line can be found in pairs trading in Song and Zhang [22]. Under a mean-reversion model, a pair of stocks can be traded simultaneously, when their price divergence exceeds a predetermined level, to bet on their eventual convergence. More details in connection with pairs trading and extensive empirical market tests can be found in Gatev et al. [12] and references therein.

To take advantage of many new features of Markov chain models and explore related mean-reversion markets, it is the purpose of this paper to study the corre- sponding trading strategies. The objective is to buy and sell the mean-reversion security to maximize a reward function. A fixed (commission or slippage) cost will be imposed to each transaction. We study the problem following a dynamic programming approach and establish the associated HJ equations for the value functions. We show that the corresponding optimal stopping (buying and selling) times can be determined by four threshold levels depending on market states. These key levels can be obtained by solving a set of algebraic like equations. In solving the associated HJ equations, hypergeometric series arise naturally in the analysis.

Various boundary conditions and convexity conditions of hypergeometric functions

are involved extensively. We provide a set of sufficient conditions that guarantee the optimality of the trading rules. Finally, to illustrate the results, we present a market backtesting example with the daily closing prices of Dow Jones Industrial Average from 1961 to 1980.

The main contributions of this paper include: (1) Introduction of a new mean- reversion model driven by a two-state Markov chain. Such model is very simple in structure and yet powerful enough to capture many important market features.

(2) Development of mean-reversion trading strategies that are simple to implement.

The corresponding optimal stopping problem is solved using a smooth-fit technique and hypergeometric analysis is involved in a substantial way.

This paper is organized as follows. In§2, we formulate the mean-reversion trading problem under consideration. In §3, we study preliminary properties of the value functions. In§4, we consider the associate HJ equations and their solutions. Then in§5, we discuss additional inequalities including the variational inequalities arising from the HJ equations and related convexity conditions. In§6, we give a verification theorem under suitable conditions. A numerical example is considered in§7. Several technical lemmas are postponed and provided in Appendix.

2. Problem formulation. Let{α_t: t≥0}denote a two-state Markov chain with state space M={1,2} and the generatorQ=

−λ1 λ1

λ2 −λ2

. Here λ1 and λ2

are positive.

LetSt denote the stock price (or a function of the price) at timetgiven by the equation

dS_t

dt =a(b−St) +f(αt), t≥0, (1) where a > 0 is the rate of reversion and b the equilibrium. Here f(1) = f1 > 0 represents the uptick return rate andf(2) =f2<0 the downtick return rate.

Example 2.1. (Convergence to a Mean-Reversion Diffusion). In this exam- ple, we present a simple Markov chain model and demonstrate how it approaches the corresponding mean-reversion diffusion. Given ε >0, we considerf1=σ/√

ε, f2 = −σ/√

ε, λ1 = λ2 = 1/ε. Using the asymptotic normality given in Yin and Zhang [26, Theorem 5.9], we can show that the solution to (1) St=S_t^ε converges weakly to the solution ofdSt=a(b−St)dt+σdWt, whereWtis a standard Brownian motion.

In particular, taking a= 1,b= 2,σ= 0.5, andε= 0.1, we generate the Monte Carlo sample paths of (S_t^ε, α^ε_t). One sample is given in (a) of Figure 1. Similarly, we varyεand plot the corresponding sample paths in (b) and (c) of Figure1. It is clear from these pictures that asεgets smaller and smaller, the fluctuation ofα_tis more and more rapidly and the correspondingS_t^ε approaches to a mean-reversion diffusion.

Remark 2.1. The main advantage of our model is its simplicity and flexibility in capturing various market conditions. For example, in a typical sideways (trendless) market, the price range can be precisely defined. In our model, such ‘price band’

arises naturally. A candidate for the band can be given by the interval [b+f2/a, b+ f1/a].

We consider the state processXt=St−b. ThenXtsatisfies dXt

dt =−aX_t+f(α_t), t≥0, (2)

0 0.5 1 1.5 2 2.5 3

0 5 10 15 20

0 0.5 1 1.5 2 2.5 3

0 5 10 15 20

0 0.5 1 1.5 2 2.5 3

0 5 10 15 20

Figure 1. Monte Carlo Sample Paths of (S_t^ε, α^ε_t) withε=0.1, 0.01, 0.001, resp.

with initial vector (X0, α0) = (x, α). Let I0 = [f2/a, f1/a]. Then Xt ∈ I0 for all t≥0 ifX0∈ I0.

LetFt ={Xr : r ≤t} denote the filtration generated by Xt. Note that αt is observable andFt={αr: r≤t}. Let

0≤τ₁^b≤τ₁^s≤τ₂^b≤τ₂^s≤ · · · (3)

denote a sequence of stopping times with respect to {Ft}. A buying decision is made atτ_n^b and a selling decision atτ_n^s, n= 1,2, . . ..

We consider the case that the net position at any time can be either long with one share of the stock or flat with no stock position. Leti= 0,1 denote the initial net position. If the initial net position is long (i = 1), then one should sell the stock before acquiring any future shares. The corresponding sequence of stopping times is denoted by Λ₁ = (τ₁^s, τ₂^b, τ₂^s, τ₃^b, . . .). Likewise, if the initial net position is flat (i= 0), then one should start to buy a share of the stock. The corresponding sequence of stopping times is denoted by Λ₀= (τ₁^b, τ₁^s, τ₂^b, τ₂^s, . . .).

LetK >0 denote the fixed transaction cost (e.g., slippage and/or commission).

Given the initial state vector (X0, α0) = (x, α), the initial net positioni= 0,1, and the decision sequences (Λ0and Λ1), we consider the corresponding reward functions

Ji(x, α,Λi) =





















 E

( _∞ X

n=1

h

e^−ρτns(Xτ_n^s −K)−e^−ρτnb(X_τb n+K)i

)

, ifi= 0, E

(

e^−ρτ1s(Xτ₁^s−K) +

∞

X

n=2

h

e^−ρτns(Xτ_n^s −K)−e^−ρτnb(X_τb n+K)i

)

, ifi= 1, (4) whereρ >0 is a given discount factor.

In this paper, given random variablesx_n, the termE{

∞

X

n=1

x_n} is interpreted as

lim sup

N→∞

E

N

X

n=1

x_n.

For i = 0,1, let V_i(x, α) denote the value functions with the initial vector (X₀, α₀) = (x, α) and initial net positionsi= 0,1. That is,

V_i(x, α) = sup

Λ_i

J_i(x, α,Λ_i). (5)

It follows from (2) thatX_t=xe^−at+e^−atRt

0e^arf(α_r)dr. Therefore,J_i(x, α,Λ_i) is linear inxandVi(x, α) is convex inxfor fixedi= 0,1 andα= 1,2.

In practice, the transaction costK >0 is typically small when compared to all other parameters. Here we assume it is smaller than both (f₁/a) and (−f2/a). In addition, bothλ₁andλ₂ are large numbers relative toa. So we also assumeλ₁> a andλ₂> a.

We summarize conditions to be imposed in this paper:

(A1)f₁>0 andf₂<0;

(A2)K <min{f₁/a,|f₂|/a}.

(A3)λ1> aandλ2> a.

Note that in this paper the stock price X_t is differentiable and the value of α_t can be given in terms of the derivative ofXtwith respect tot.

3. Properties of the value functions.

Lemma 3.1. The following inequalities hold.

(a) 0≤V₀(x, α)≤

ρ+ 2a ρa

max{f₁,|f₂|}, for all(x, α)∈ I₀× M.

(b) −K ≤ V0(x, α)−V1(x, α) +x ≤ K, for all (x, α) ∈ I0× M. Therefore, V1(x, α)is also bounded onI0× M.

Proof. It is clear thatV0(x, α)≥0. To show it is bounded from above, we first note that

d(e^−ρtX_t)

dt =e^−ρt(−(ρ+a)X_t+f(α_t)).

Given Λ0, integrate both sides over [τ_n^b, τ_n^s] to obtain e^−ρτnsXτ_n^s−e^−ρτnbX_τb

n= Z τ_n^s

τ_n^b

e^−at(−(ρ+a)Xt+f(αt))dt.

It follows that, for (x, α)∈ I0× M, J0(x, α,Λ0) =E

∞

X

n=1

h

e^−ρτns(Xτ_n^s−K)−e^−ρτnb(X_τb n+K)i

≤E

∞

X

n=1

h

e^−ρτnsXτ_n^s−e^−ρτnbX_τb n

i

≤E

∞

X

n=1

Z τ_n^s τ_n^b

e^−at(−(ρ+a)Xt+f(αt))dt

≤E Z ∞

0

e^−at((ρ+a)|Xt|+|f(αt)|)dt

≤

ρ+ 2a ρa

max{f₁,|f₂|}.

This implies (a).

To show thatV₀(x, α)−V₁(x, α) +x≤K, or V₁(x, α)≥V₀(x, α) +x−K, we note that, for a given Λ₁ withτ₁^s= 0,

V₁(x, α)≥J₁(x, α,Λ₁) +x−K.

Since the rest stopping times in Λ1are arbitrary, it follows thatV1(x, α)≥V0(x, α)+

x−K. Similarly, we can show the second inequality in (b).

4. HJ equations. LetAdenote the generator of (Xt, αt), i.e., for any differentiable functionsh(x, i),i= 1,2,

( Ah(x,1) = (−ax+f1)h⁰(x,1) +λ1(h(x,2)−h(x,1)), Ah(x,2) = (−ax+f2)f2h⁰(x,2) +λ2(h(x,1)−h(x,2)),

whereh⁰denotes the derivative ofhwith respect tox. The associated HJ equations should have the form:

min{ρv₀(x, α)− Av₀(x, α), v₀(x, α)−v₁(x, α) +x+K}= 0,

min{ρv₁(x, α)− Av₁(x, α), v₁(x, α)−v₀(x, α)−x+K}= 0, (6) forα= 1,2.

In this paper, our goal is to find the value functions by solving these HJ equations with all value function properties satisfied. Then we proceed to show that such functionsv_i(x, α) are equal to the value functions.

4.1. Solving (ρ− A)vi(x, α) = 0. To solve these HJ equations, for each fixed i= 0,1, we consider equations when ρvi(x, α)− Avi(x, α) = 0,α= 1,2. Using the generatorA, we can write

(ρ+λ₁)v_i(x,1) = (−ax+f₁)v⁰_i(x,1) +λ₁v_i(x,2),

(ρ+λ₂)v_i(x,2) = (−ax+f₂)v⁰_i(x,2) +λ₂v_i(x,1). (7) For each fixed i = 0,1 and α = 1,2, let y_α(x) = v_i(x, α). Then, using the first equation, we can writey₂ in terms ofy₁:

y₂(x) = 1 λ1

[(ρ+λ₁)y₁(x) + (ax−f₁)y⁰₁(x)].

Its derivative with respect toxis given by y⁰₂(x) = 1

λ1

[(ρ+λ1+a)y₁⁰(x) + (ax−f1)y₁⁰⁰(x)].

Substitute these into the second equation in (7) to obtain (ax−f₁)(ax−f₂)y⁰⁰₁(x) + [(ρ+λ₂)(ax−f₁)

+(ρ+λ₁+a)(ax−f₂)]y₁⁰(x) +ρ(ρ+λ₁+λ₂)y₁(x) = 0.

To simplify this equation, let

ξ=ax−f2

f₁−f₂ andu(ξ) =y1(x). Then, it follows that

ax−f₂= (f₁−f₂)ξ and ax−f₁= (f₁−f₂)(ξ−1).

Moreover, applying the chain rule to obtain y₁⁰(x) =

a f₁−f₂

u⁰(ξ) and y₁⁰⁰(x) = a

f₁−f₂ ²

u⁰⁰(ξ).

This reduces the equation fory1(x) into an equation foru(ξ) as

a²ξ(ξ−1)u⁰⁰(ξ) +a[(ρ+λ2)(t−1) + (ρ+λ1+a)t]u⁰(ξ) +ρ(ρ+λ1+λ2)]u(ξ) = 0.

Next, we introduce the additional parameters γ1=ρ

a, γ2= ρ+λ₁+λ₂

a , γ3=ρ+λ₂ a .

Then, we can write the above differential equation in terms of (γ₁, γ₂, γ₃) as ξ(1−ξ)u⁰⁰(ξ) + [γ₃−(γ₁+γ₂+ 1)t]u⁰(ξ)−γ₁γ₂u(ξ) = 0. (8) This is exactly the classical hypergeometric equation. Related properties of hyper- geometric functions can be found in [19] and [25].

For any real numbersηi,i= 1,2,3, let F(η1, η2;η3, ξ) =

∞

X

n=0

(η1)n(η2)n

n!(η3)n

ξⁿ, where

(η)0= 1 and (η)n=η(η+ 1)· · ·(η+n−1) = Γ(η+n) Γ(η) ,

for anyη =η_i. Then, for 0< ξ <1, two independent solutions of (8) can be given by

F ρ

a,ρ+λ1+λ2

a ;ρ+λ2

a , ξ

andF ρ

a,ρ+λ1+λ2

a ;ρ+λ2+a a ,1−ξ

.

Let

F1(x) =F ρ

a,ρ+λ1+λ2

a ;ρ+λ2

a ,ax−f2

f₁−f₂

, Fe1(x) =F

ρ

a,ρ+λ1+λ2

a ;ρ+λ2+a

a ,1−ax−f2

f1−f2

.

(9)

We consider these solutions over a neighborhood inI0 containing the lower end f2/a. Then any solution over this interval can be written as a linear combination of F1 and Fe1, i.e., for some constants C1 and Ce1, vi(x,1) =C1F1(x) +Ce1Fe1(x), i= 0,1. In particular, we need this solution to be bounded nearf2/a. This requires Ce1= 0 becauseFe1(x)→ ∞asx→f2/a(see Lemma 8.1in Appendix).

Therefore, on the neighborhood ofI0containingf₂/awe can assume thatv₁(x,1)

=C₁F₁(x). Using this solution, we can writev₁(x,2) in terms ofF₁(x):

v₁(x,2) = C₁ λ1

[(ρ+λ₁)F₁(x) + (ax−f₁)F₁⁰(x)] :=C₁G₁(x). (10) Alternatively, we can use the second equation in (7) and writey1in terms ofy2:

y1(x) = 1

λ₂[(ρ+λ2)y2(x) + (ax−f2)y⁰₂(x)]

and its derivative in terms ofy⁰₂ andy⁰⁰₂ y⁰₁(x) = 1

λ2

[(ρ+λ2+a)y₂⁰(x) + (ax−f2)y₂⁰⁰(x)].

Substituting these into the first equation in (7) to obtain (ax−f₁)(ax−f₂)y⁰⁰₂(x) + [(ρ+λ₁)(ax−f₂)

+(ρ+λ₂+a)(ax−f₁)]y₁⁰(x) +ρ(ρ+λ₁+λ₂)y₂(x) = 0.

Similarly, letw(ξ) =y2(x) with

ξ=f1−ax f1−f2

,

Then, we can write the above differential equation in terms ofw(ξ):

ξ(1−ξ)w⁰⁰(ξ) + [γ₃⁰ −(γ₁+γ₂+ 1)t]w⁰(ξ)−γ₁γ₂w(ξ) = 0, withγ₃⁰ = (ρ+λ₁)/a. Its two independent solutions can be given by

F ρ

a,ρ+λ1+λ2

a ;ρ+λ1

a , ξ

andF ρ

a,ρ+λ1+λ2

a ;ρ+λ1+a a ,1−ξ

. Let

F2(x) =F ρ

a,ρ+λ1+λ2

a ;ρ+λ1

a ,f1−ax f₁−f₂

, Fe2(x) =F

ρ

a,ρ+λ1+λ2

a ;ρ+λ2+a

a ,1−f1−ax f1−f2

.

(11)

Similarly, we consider these solutions over a neighborhood in I0 containing the upper end f1/a. Again, any solution can be written as a linear combination of F2

andFe2, i.e., for some constantsC2 andCe2,vi(x,2) =C2F2(x) +Ce2Fe2(x),i= 0,1.

We now need this solution to be bounded nearf1/a, which impliesCe2= 0 because Fe₂(x)→ ∞asx→f₁/a.

-

f₁ a f₂

a

b2 b1

v0(1) =v1(1)−x−K

v0(1) =v1(1)−x−K ρv0(1) =Av₀(1)

v₀(2) =v₁(2)−x−K ρv₀(2) =Av₀(2) ρv₀(2) =Av₀(2)

-

f₁ a f₂

a

s₁ s₂

ρv1(1) =Av₁(1) ρv1(1) =Av₁(1)

ρv₁(2) =Av₁(2) v₁(2) =v₀(2)+x−K v₁(2) =v₀(2)+x−K v1(1) =v0(1)+x−K

Figure 2. Buying Thresholds: b1 andb2; Selling Thresholds: s1 ands2. Therefore, on the neighborhood ofI0containingf1/awe can assume thatv0(x,2)

=C2F2(x). Using this solution, we have v₀(x,1) = C₂

λ2

[(ρ+λ₂)F₂(x) + (ax−f₂)F₂⁰(x)] :=C₂G₂(x). (12) Lemma 4.1. The solutions of (ρ− A)vi(x, α) = 0 can be given as

v0(x,1) =C2G1(x), v0(x,2) =C2F2(x), v1(x,1) =C1F1(x), v1(x,2) =C1G1(x), whereF₁,F₂,G₁, andG₂ are defined in (9), (11), (10), and (12), respectively.

4.2. Order of threshold levels. Ifαt= 1, letb1 denote the buying threshold (to buy whenXt< b1) ands1the selling threshold (to sell whenXt> s1), respectively.

Likewise, if αt = 2, let b2 and s2 denote the corresponding buying and selling thresholds, respectively.

In the purely selling case, it is shown in [29] thats₁> s₂. Intuitively,b₁ should be greater thanb₂ as well.

In addition, in this paper, we consider the case whenb₁< s₂.

The opposite inequalityb₁≥s₂means to buy whenα_t= 1 andX_t< b₁and sell as soon as αt jumps to 2 andXt> s2. This case rarely arises in practice because the jump rates λ1 and λ2 are typically large so thatαt switches between 1 and 2 frequently.

In view of these, we consider the threshold levels having the following order f2

a < b2< b1< s2< s1<f1

a.

The corresponding equalities in the HJ equations in (6) are marked in Figure2.

Using convexity of the value functions and variational inequalities in (6), it is shown in Lemma8.2(Appendix) that

b2= f2−ρK

ρ+a ands1=f1+ρK ρ+a .

It is easy to check, under Assumption (A2), thatb2 and s1 given above indeed satisfy the inequalitiesb₂> f₂/aands₁< f₁/a.

4.3. Solving (ρ− A)v1(x,1) = 0 with v₁(x,2) = v₀(x,2) +x−K and (ρ− A)v0(x,2) = 0with v₀(x,1) =v₁(x,1)−x−K. Forx∈ I0, let

g₁(x) =

as1−f1

ax−f₁ ^ρ+λ_a¹

andg₂(x) =

ab2−f2

ax−f₂ ^ρ+λ_a²

.

Next, we solve the HJ equations on [s2, s1]. Note that on this interval,v0(x,1) = C2G2(x),v0(x,2) =C2F2(x), andv1(x,2) =v0(x,2) +x−K=C2F2(x) +x−K.

To solveρv1(x,1) =Av1(x,1) on this interval, we write it as (ax−f₁)v⁰₁(x,1) + (ρ+λ₁)v₁(x,1) =λ₁v₁(x,2).

Differentiate (1/g₁(x))v₁(x,1) and then integrate froms₁ toxto obtain v₁(x,1) =v₁(s₁,1)g₁(x)−λ₁(ax−f₁)^−ρ+λa1

Z x s₁

(f₁−as)^ρ+λa1−1v₁(s,2)ds.

Recall thatv1(x,2) =C2F2(x) +x−K. We write

v1(x,1) =g1(x)A+k1(x)C2+k2(x), whereA=v1(s1,1) and











k1(x) =−λ1(f1−ax)^−ρ+λa1 Z x

s1

(f1−as)^ρ+λa1−1F2(s)ds, k2(x) =−λ1(f1−ax)^−ρ+λa1

Z x s1

(f1−as)^ρ+λa1−1(s−K)ds.

Similarly, we solve the HJ equations on [b2, b1]. Note that on this interval, v1(x,1) = C1F1(x), v1(x,2) = C1G1(x), and v0(x,1) = v1(x,1)−x−K = C1F1(x)−x−K. To solve ρv0(x,2) = Av0(x,2) on this interval, we first write it as

(ax−f2)v⁰₀(x,2) + (ρ+λ2)v0(x,2) =λ2v0(x,1).

Differentiate (1/g2(x))v0(x,2) and then integrate over [b2, x] to obtain v0(x,2) =v0(b2,2)g2(x) +λ2(ax−f2)^−ρ+λa2

Z x b2

(as−f2)^ρ+λa2−1v0(s,1)ds.

Recall thatv0(x,1) =C1F1(x)−x−K. We write

v₀(x,2) =g₂(x)B+h₁(x)C₁+h₂(x), whereB=v₀(b₂,2) and











h₁(x) =λ₂(ax−f₂)^−ρ+λa2 Z x

b₂

(as−f₂)^ρ+λa2−1F₁(s)ds, h2(x) =−λ2(ax−f2)^−ρ+λa2

Z x b₂

(as−f2)^ρ+λa2−1(s+K)ds.

Using the power series form ofF1(x) =

∞

X

n=0

pn

ax−f2

f1−f2

n

, we can write

h1(x) =λ2

∞

X

n=0

pn

ρ+λ2+an

ax−f2

f1−f2

n

−g2(x)

ab2−f2

f1−f2

n . Similarly, we have

h₂(x) =−λ₂ a

1

ρ+λ2+a((ax−f₂)−(ab₂−f₂)g₂(x)) +f₂+aK ρ+λ2

(1−g₂(x))

.

Likewise, writeF2(x) =

∞

X

n=0

qn

f₁−ax f1−f2

n

to obtain

k₁(x) =−λ1

∞

X

n=0

q_n ρ+λ1+an

g₁(x)

f₁−as₁ f1−f2

n

−

f₁−ax f1−f2

n

and

k2(x) =λ1

a

− 1

ρ+λ₁+a((f1−ax)−(f1−as1)g1(x)) +f1−aK

ρ+λ₁ (1−g1(x))

. 4.4. Smooth-fit conditions. Note that the convexity ofvi(x, α) inxonI0implies that they are continuous in the interior ofI0. The continuity of these functions at the threshold levels leads to the following conditions:



















x=b₂ : C₁G₁(b₂)−b₂−K=g₂(b₂)B+h₁(b₂)C₁+h₂(b₂);

x=b₁ :

C₁F₁(b₁)−b₁−K=C₂G₂(b₁),

g₂(b₁)B+h₁(b₁)C₁+h₂(b₁) =C₂F₂(b₁), x=s₂ :

C1F1(s2) =g1(s2)A+k1(s2)C2+k2(s2), C1G1(s2) =C2F2(s2) +s2−K,

x=s1 : C2G2(s1) +s1−K=g1(s1)A+k1(s1)C2+k2(s1).

(13)

Recall thatb2= (f2−ρK)/(ρ+a) ands1= (f1+ρK)/(ρ+a). Using the first and last equations in (13), we can write









 B=

G1(b2)−h1(b2) g₂(b₂)

C1−b2+K+h2(b2)

g₂(b₂) =:B1C1+B2

A=

G2(s1)−k1(s1) g₁(s₁)

C2+s1−K−k2(s1)

g₁(s₁) =:A1C2+A2.

Then, we can write the second and the third equations in terms ofC1andC2: F1(b1) −G2(b1)

g2(b1)B1+h1(b1) −F2(b1)

C1

C2

=

b1+K

−g2(b1)B2−h2(b1)

. Similarly, we can write the fourth and the fifth equations as follows:

G₁(s₂) −F2(s₂) F₁(s₂) −[g1(s₂)A₁+k₁(s₂)]

C₁ C₂

=

s₂−K g₁(s₂)A₂+k₂(s₂)

. If the above 2×2 matrices are invertible, then we can eliminate C1 and C2 and obtain

F1(b1) −G2(b1) g2(b1)B1+h1(b1) −F2(b1)

−1

b1+K

−g2(b1)B2−h2(b1)

=

G1(s2) −F2(s2) F1(s2) −[g1(s2)A1+k1(s2)]

⁻¹

s2−K g1(s2)A2+k2(s2)

.

(14)

The solutions to the HJ equations (6) should have the form:



































v0(x,1) =

C1F1(x)−x−K iff2/a≤x≤b1, C2G2(x) ifb1< x≤f1/a, v0(x,2) =







C1G1(x)−x−K iff2/a≤x≤b2, g2(x)B+h1(x)C1+h2(x) ifb2< x≤b1, C2F2(x) ifb1< x≤f1/a, v1(x,1) =







C₁F₁(x) iff₂/a≤x≤s₂, g₁(x)A+k₁(x)C₂+k₂(x) ifs₂< x≤s₁, C₂G₂(x) +x−K ifs₁< x≤f₁/a, v₁(x,2) =

C₁G₁(x) iff₂/a≤x≤s₂, C₂F₂(x) +x−K ifs₂< x≤f₁/a,

(15)

In Figure 3, these functions are so labelled in their corresponding intervals in Figure2.

-

f₁ a f₂

a

b2 b1

v0(1) =C1F1−x−K

v0(1) =C1F1−x−K v0(1) =C2G2

v₀(2) =C₁G₁−x−K v₀(2)=g₂B+h₁C₁+h₂ v₀(2) =C₂F₂

-

f₁ a f₂

a

s₂ s₁

v1(1) =C1F1 v1(1)=g1A+k1C2+k2

v₁(2) =C₁G₁ v₁(2) =C₂F₂+x−K v₁(2) =C₂F₂+x−K v1(1) =C2G2+x−K

Figure 3. HJ equation solutions.

5. Variational inequalities and convexity. In this section, we consider the variational inequalities in (6) and related convexity inequalities.

5.1. Variational inequalities. Note that with the functions given in (15), all variational inequalities in (6) have to be satisfied, i.e., for (x, α)∈ I0× M,











ρv0(x, α)− Av0(x, α)≥0, v0(x, α)−v1(x, α) +x+K≥0, ρv₁(x, α)− Av1(x, α)≥0, v₁(x, α)−v₀(x, α)−x+K≥0.

(16)

In what follows, we consider these inequalities on each intervals: [f₂/a, b₂], [b₂, b₁], [b1, s2], [s2, s1], and [s1, f1/a].

First, we show that these inequalities on [f2/a, b2] are automatically satisfied.

Note that on [f2/a, b2], we have











v₀(x,1) =v₁(x,1)−x−K, v₀(x,2) =v₁(x,2)−x−K, ρv₁(x,1) =Av₁(x,1), ρv₁(x,2) =Av₁(x,2).

(17)

We need to show











ρv0(x,1)≥ Av0(x,1), ρv0(x,2)≥ Av0(x,2), v1(x,1)≥v0(x,1) +x−K, v₁(x,2)≥v₀(x,2) +x−K.

(18)

It is easy to see that the last two inequalities follow directly from the first two equalities in (17). To see the first inequalityρv₀(x,1)≥ Av₀(x,1), note that it can be written in terms ofv1(x,1)andv1(x,2):

ρ(v1(x,1)−x−K)≥(f1−ax)(v⁰₁(x,1)−1) +λ1(v1(x,2)−v1(x,1)), which is equivalent to

−ρ(x+K)≥ −(f1−ax), forx < b2,

because ρv1(x,1) = Av1(x,1). The above inequality is clearly satisfied because b2= (f2−ρK)/(ρ+a) andf2< f1.

Similarly, on [s1, f1/a], we can uses1= (f1+ρK)/(ρ+a) and show that all VIs in (16) hold.

Next, we consider the VIs on [b2, b1]. We have











v0(x,1) =v1(x,1)−x−K, ρv0(x,2) =Av0(x,2), ρv1(x,1) =Av1(x,1), ρv1(x,2) =Av1(x,2).

(19)

We need the following inequalities to hold:











ρv0(x,1)≥ Av0(x,1), v0(x,2)≥v1(x,2)−x−K, v1(x,1)≥v0(x,1) +x−K, v₁(x,2)≥v₀(x,2) +x−K.

(20)

The third inequality is automatically satisfied. The second and forth ones can be written as:

−K≤v0(x,2)−v1(x,2) +x≤K.

Finally, the first inequality in (20) can be given as v0(x,2)−v1(x,2) +x≤ 1

λ1

[f1−ρK−(ρ+a)x]−K.

We can work out all needed inequalities this way on all other intervals and sum- marize these inequalities as follows:















On [b2, b1] : −K≤W2(x)≤min

K, 1 λ1

[f1−ρK−(ρ+a)x]−K

, On [b₁, s₂] : −K≤W₁(x)≤K and −K≤W₂(x)≤K,

On [s₂, s₁] : max

−K, 1 λ2

[f₂+ρK−(ρ+a)x] +K

≤W₁(x)≤K, (21)

whereW1(x) =v0(x,1)−v1(x,1) +xandW2(x) =v0(x,2)−v1(x,2) +x.

5.2. Convexity. Recall that the value functions are convex onI0. We expect the solutions to the HJ equations in (6) to be also convex.

Recall also that

G_i(x) = 1 λi

((ρ+λ_i)F_i(x) + (ax−f_i)F₁⁰(x)), fori= 1,2.

Lemma 5.1. F1,F2,G1,G2 are convex on[f2/a, f1/a].

Proof. Clearly, both F1 and F2 are convex because their second derivatives are positive.

To show the convexities ofG1 andG2, we first consider g(ξ) = (γ₁+γ₂−γ₃)u(ξ)−(1−ξ)du

dξ(ξ),

with u(ξ) = F(γ₁, γ₂;γ₃, ξ). Multiply its both sides by (−(1−ξ)^{γ1+γ2−γ3−1}) to obtain

−(1−ξ)^{γ1+γ2−γ3−1}g(ξ) =−(γ1+γ2−γ3)(1−ξ)^{γ1+γ2−γ3−1}u(ξ)+(1−ξ)^{γ1+γ2−γ3}du dξ(ξ).

The right hand side of the above equation is equal to d

dξ

(1−ξ)^{γ1+γ2−γ3}F(γ₁, γ₂;γ₃, ξ) .

We next apply the formula dⁿ

dξⁿ

(1−ξ)^{γ1+γ2−γ3}F(γ1, γ2;γ3, ξ)

=(γ₃−γ₁)_n(γ₃−γ₂)_n (γ3)n

(1−ξ)^{γ1+γ2−γ3−n}F(γ1, γ2;γ3+n, ξ), withn= 1 and get

−(1−ξ)^{γ1+γ2−γ3−1}g(ξ) =(γ3−γ1)(γ3−γ2)

γ₃ (1−ξ)^{γ1+γ2−γ3−1}F(γ1, γ2;γ3+ 1, ξ).

Hence we conclude that

g(ξ) =−(γ3−γ1)(γ3−γ2) γ3

F(γ1, γ2;γ3+ 1, ξ).

Foru(ξ) =F_ρ

a,^ρ+λ1_a^+λ2;^ρ+λ_a ², ξ

, the correspondingg(ξ) is given by g(ξ) = λ2λ1

ρ+λ₂F ρ

a,ρ+λ1+λ2

a ;ρ+λ2

a + 1, ξ

. Then the convexity ofF_ρ

a,^ρ+λ1_a^+λ2;^ρ+λ_a² + 1, ξ

implies thatg(ξ) is convex. It is easy to check with change of variablet= (ax−f2)/(f1−f2) thatG1(x) = (λ1/a)g(ξ).

SoG1is convex. Similarly, we can show the convexity ofG2. Lemma 5.2. h1(x)andk1(x)are convex.

Proof. Here, we only show the convexity ofh1(x). The convexity fork1(x) can be given in a similar way. Recall that

h1(x) =λ2(ax−f2)^−ρ+λa2 Z x

b2

(as−f2)^ρ+λa2−1F1(s)ds, where

F1(x) =F

γ1, γ2;γ3,ax−f2

f₁−f₂

withγ1= ρ

a, γ2=ρ+λ1+λ2

a andγ3= ρ+λ2

a . Let

w1(ξ) =h1(x) =h1

(f₁−f₂)t+f₂ a

, witht= ax−f₂ f1−f2

. It follows that

w₁(ξ) = λ₂ aξ^−γ3

Z ξ eb₂

r^γ3−1F(γ₁, γ₂;γ₃, r)dr, whereeb2= (ab2−f2)/(f1−f2). Note that

Z

F(α, β;γ, ξ)ξ^γ−1dt= ξ^γ

γ F(α, β;γ+ 1, ξ) +k.

Apply the above formula, we obtain w1(ξ) = λ2

aγ3

h

F(γ1, γ2;γ3+ 1, ξ)−F(γ1, γ2;γ3+ 1,eb2)i . Recall the derivative formula for hypergeometric function:

d^m

dξ^mF(α, β;γ, ξ) =(α)_m(β)_m (γ)m

F(α+m, β+m;γ+m, ξ).

It follows that

w⁰⁰₁(ξ) =λ2γ1(γ1+ 1)γ2(γ2+ 1)

aγ₃(γ₃+ 1)(γ₃+ 2) F(γ1+ 2, γ2+ 2;γ3+ 3, ξ).

In view of this and the convexity of the hypergeometric functions with positive parameters, the convexity ofw₁(ξ) follows on [0,1]. So is the convexity ofh₁(x).

Lemma 5.3. g2B+h1C1+h2 andg1A+k1C2+k2 are convex provided that C₁≥0, C₂≥0, A+A₀≥0, B+B₀≥0, (22) where

A0= λ1

a

f1−as1

ρ+λ₁+a−f1−aK ρ+λ₁

andB0= λ2

a

ab2−f2

ρ+λ₂+a+f2+aK ρ+λ₂

. Proof. It is easy to see that bothg1(x) andg2(x) are convex. In addition, note that g1A+k2 can be written as g1(A+A0)+ a linear function. SoA+A0>0 implies that g1A+k2 is convex. Then in view of the convexity ofk1(x) in Lemma5.2, it follows that g1A+k1C2+k2 is convex. Similarly, we can show the convexity of

g₂B+h₁C₁+h₂.

To guarantee the convexity of the functionsv_i(x, α), it is sufficient to require the convexity ofv0(x,1) atx=b1and v1(x,2) atx=s2.

The convexity ofv0(x,1) atx=b1 is equivalent to (v₀(b₁,1))⁰₋≤(v₀(b₁,1))⁰₊. Therefore, we need

C1F₁⁰(b1)−1≤C2G⁰₂(b1). (23) Similarly, The convexity ofv1(x,2) atx=s2requires

C₁G⁰₁(s₂)≤C₂F₂⁰(s₂) + 1. (24) Theorem 5.1. Assume that there exist b1 and s2 such that (a) the equalities in (14) hold with invertible matrices; (b) the additional VI inequalities in (21) hold;

(c)the inequalities in(22)hold; (d)the convexity inequalities in(23)and(24)hold.

Then, vi(x, α) defined in(15)satisfy the HJ equations (6).

6. A verification theorem.

Theorem 6.1. Assume the conditions of Theorem 5.1. Then, (a)v(x, i) =V(x, i),i= 1,2.

(b)Moreover, let

Db = (b1, f1/a)× {1} ∪(b2, f1/a)× {2}, Ds= (f2/a, s1)× {1} ∪(f2/a, s2)× {2}

denote the continuation regions. The set of stopping times in Λ₀ and Λ₁ can be defined as the first exit times from these continuation regions in sequence. Let Λ^∗₀ = (τ₁^b∗, τ₁^s∗, τ₂^b∗, τ₂^s∗, . . .), where τ₁^b∗ = inf{t : (X_t, α_t) 6∈ D_b}, τ₁^s∗ = inf{t ≥ τ₁^b∗ : (X_t, α_t) 6∈ D_s},.., τ_n+1^b∗ = inf{t ≥ τ_n^s∗ : (X_t, α_t) 6∈ D_b}, τ_n+1^s∗ = inf{t ≥ τ_n+1^b∗ : (X_t, α_t)6∈D_s},..., and let Λ^∗₁ = (τ₁^s∗, τ₂^b∗, τ₂^s∗, . . .), where τ₁^s∗= inf{t≥0 : (Xt, αt) 6∈ Ds},.., τ_n+1^b∗ = inf{t ≥ τ_n^s∗ : (Xt, αt) 6∈ Db}, τ_n+1^s∗ = inf{t ≥ τ_n+1^b∗ : (X_t, α_t)6∈D_s},... Ifv₀(x)≥0 andτ_n^s∗→ ∞, a.s., then, Λ^∗₀ andΛ^∗₁ are optimal.