Full text of DOI:10.1007/PL00009877

Math. Control Signals Systems (2001) 14: 62±85
( 2001 Springer-Verlag London Limited
Mathematics of Control,
Signals, and Systems
Risk-Sensitive and Robust Escape Control for
Degenerate Diffusion Processes*
z
Ây
Michelle Boue and Paul Dupuis
Abstract. This paper considers the problem of controlling a possibly degenerate
small noise di¨usion so as to prevent it from leaving a prescribed set. The crite-
rion of interest is a risk-sensitive version of the mean escape time criterion. Using
a general representation formula, this criterion is expressed as the upper value of
a stochastic di¨erential game. It is shown that in the small noise limit this upper
value converges to the value of an associated deterministic di¨erential game. Our
approach di¨ers from standard PDE approaches in a number of ways. For ex-
ample, the upper game representation allows one to relate directly the prelimit
and the limit controls and, in fact, strategies that are nearly maximizing for the
robust problem can be used to de®ne nearly minimizing controls for the risk-sen-
sitive control problem for su½ciently small e > 0. The result provides a canonical
example of the use of variational representations in connecting risk-sensitive and
robust control.
Key words. Risk-sensitive control, Robust control, Escape time, Di¨usion pro-
cesses, Degenerate processes, Di¨erential games, Variational representations.
1. Introduction
Consider the controlled di¨usion process satisfying the stochastic di¨erential
equation
dX u;e ˆ bꢀX u;e; u† dt ‡ e1=2sꢀX u;e† dW
ꢀ1†
in Rd , where e > 0 is a parameter and W is r-dimensional Brownian motion. The
control u is assumed to take values in a compact set U H Rm. The objective is to
control the drift of the di¨usion so as to prevent it from leaving a prescribed set
G H Rd , which is assumed to be open with G (the closure of G) compact. The
optimization criterion we consider is the minimization of the quantity
ꢀ
ꢁ
ytu;e
e
Ex exp À
;
ꢀ2†
* Date received: November 21, 1998. Date revised: June 20, 1999. This research was supported in
part by the National Science Foundation (NSF-DMS-9403820) and by the O½ce of Naval Research
(ONR-N00014-96-1-0276).
y
Department of Mathematics, Statistics and Computer Science, St. Francis Xavier University,
PO Box 5000, Antigonish, Nova Scotia, Canada B2G 2W5. mboue@stfx.ca.
z
Lefschetz Center for Dynamical Systems, Division of Applied Mathematics, Brown University,
Providence, Rhode Island 02912, U.S.A. dupuis@cfm.brown.edu.
62

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
63
where tu;e ˆ infft V 0 : X u;eꢀt† B Gg, y is a positive design parameter, and Ex
denotes expectation conditioned on X u;eꢀ0† ˆ x. This criterion was introduced in
[DM] and has certain qualitative advantages over other criteria usually associated
with this problem (namely, the criterion of minimizing the probability of escape
over some interval ‰0; TŠ or that of maximizing the mean escape time). One can
view (2) as a risk-sensitive version of the mean escape time criterion. The e¨ect of
the transformation tu;e ! expfÀytu;e=eg is that when computing the expectation,
those paths that escape in a relatively short time are weighed more heavily, forc-
ing the optimal (or any nearly optimal) control to pay more attention to these
undesirable outcomes. The risk-sensitivity parameter y determines the extent to
which these paths are to be emphasized.
For a ®xed smooth control u, it is easy to verify via large deviation estimates
that (2) scales exponentially in e as e ! 0. Hence if one wishes to consider the
limit as e ! 0, it is natural to use the equivalent criterion
ꢀ
ꢁ
ytu;e
Àe log Ex exp À
:
ꢀ3†
e
The risk-sensitive problem of interest then consists of obtaining the value
ꢀ
ꢁ
ytu;e
e
V eꢀx† j sup À e log Ex exp À
:
u
By means of PDE viscosity solution techniques, and under the assumption
of a uniformly nondegenerate di¨usion (i.e., there exists m > 0 such that
infx A G sꢀx†sꢀx†0 V mI), it is shown in [DM] that as e ! 0 the value V eꢀx† con-
verges to the value function of a deterministic di¨erential game. This game repre-
sents a robust control problem where an opposing minimizing player is trying to
drive the process outside of the prescribed set but paying a quadratic cost for his
e¨orts. In this paper we extend the results in [DM] to the case of possibly degen-
erate di¨usions, as well as elaborate on the robust aspects of the limit problems
(a robust interpretation of the risk-sensitive problem is also possible [BJP], but we
do not include it here). Our approach uses probabilistic arguments and the repre-
sentation formulas for functionals of Brownian motion developed in [BD]. As we
now discuss, this approach has some interesting features which are not linked
in any way to the particular risk-sensitive problem considered, and in fact have
analogues for other cost structures as well.
As noted above, a key ingredient in the analysis is the representation of (3), for
each ®xed control, as the minimal cost for a stochastic optimal control problem.
An automatic consequence of the representation is a representation for V eꢀx† as
the value function for a stochastic di¨erential game. This game will also be small
noise, and the progressively measurable control u in the original control problem
will be transformed into a strategy for the maximizing player in this game. The
connection between the risk-sensitive value function and a deterministic di¨eren-
tial game is made immediately: the limit problem can be identi®ed as the obvious
candidate obtained by simply deleting the noise term. Probabilistic arguments are
used to prove convergence to the robust problem rigorously.

Â
M. Boue and P. Dupuis
64
It is interesting to observe that each progressively measurable control in the
prelimit risk-sensitive problem is transformed into a maximizing strategy for the
corresponding stochastic game. This is very natural, because it allows a direct
connection between the strategy for the game and the control for the risk-sensitive
problem. However, in a general setting, where the upper and lower values for the
game might di¨er, it would imply an advantage for the maximizing player (i.e.,
the true control), with the representation being in terms of the upper game. Al-
though natural, at ®rst glance the representation in terms of a game which favors
the maximizing player would seem to contradict the representation one would
expect to obtain via a PDE approach [FS], in which it is the lower game that
appears. Here the control that represents nature has the advantage and selects
the strategy (rather than an open loop control). As it turns out, there is no direct
contradiction because the upper and lower games coincide for the class of risk-
sensitive problems for which both representations have been established. How-
ever, this issue is not fully understood and needs to be explored further in settings
where the upper and lower values might di¨er.
Because of the e-dependence of the prelimit dynamics, it is not guaranteed that
a simple connection exists between optimal controls for the limit deterministic
di¨erential game and optimal controls for the prelimit stochastic control problem
for a given e > 0. An important feature of our approach is that we do in fact ob-
tain such a connection. In particular, it is shown that any nearly supremizing
strategy for the limit problem can be used to de®ne a nearly minimizing control
law for the risk-sensitive problem for su½ciently small e > 0. This control will
depend on e only through its appearance as a coe½cient of the driving noise pro-
cess. Note that in this context it is quite important that the strategy in the deter-
ministic game be chosen by the maximizing player, since that player represents
the control in the risk-sensitive problem.
For other examples in which variational representations are used in risk-
sensitive control problems we refer the reader to [BJP], [FH], and [MH]. Repre-
sentations and weak convergence methods in the discrete time setting can be
found in [DE].
The plan of the paper is as follows. In the next section basic assumptions and
notation are introduced. In Section 3 we give precise de®nitions regarding the dif-
ferential game to be considered and give a robust control interpretation of this
game. Section 4 is devoted to the asymptotic analysis, and includes the main
results. The proofs are provided in Section 5. Finally, two auxiliary lemmas are
proved in the Appendix.
2. Preliminaries
Let ꢀW; BꢀW†; P† be the canonical probability space for Brownian motion in Rr,
where W ˆ Cꢀ‰0; y† : Rr†, P is the r-dimensional Wiener measure, and B de-
notes the Borel s-algebra. The space Cꢀ‰0; y† : Rr† is endowed with any metric
equivalent to uniform convergence on compact sets. Under P, the coordinate
mapping process W ˆ fWꢀt; o† j oꢀt†; 0 U t < yg together with the ®ltration

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
65
fFW g j fsꢀWꢀs†; 0 U s U t†g is an r-dimensional Brownian motion. We con-
t
struct the augmented ®ltration fFg by de®ning Fj sꢀFW W N†, where N j
t
t
t
fN H Cꢀ‰0; y† : Rr† : there exists B A BꢀCꢀ‰0; y† : Rr†† with N H B and
PꢀB† ˆ 0g. For t ˆ y we set F j sꢀFW W N†. Relative to the continuous ®ltra-
y
tion fFg (which we always assume to be the underlying ®ltration) W is still a
t
Brownian motion.
Admissible Controls. We take as our class of admissible controls the set of U-
valued adapted right-continuous processes, which we denote by F. We consider
this class rather than the usual class of progressively measurable processes so as to
deal with measurability issues that arise in what follows. There is essentially no
loss of generality since under broad conditions there is always a nearly optimal
control for (3) belonging to this class. Moreover, even if this were not the case,
one would prefer to avoid controls with less regularity.
~
We identify two controls u and u in F as being the same on ‰0; sŠ if Pfuꢀt† ˆ
~
uꢀt† for all t A ‰0; sŠg ˆ 1. If u A F, then for every s A ‰0; y† there exists a Borel
measurable function us : ‰0; sŠ  Cꢀ‰0; sŠ : Rr†
7
uꢀt† ˆ usꢀt; Wꢀr†; 0 U r U s†
We make the following assumptions on the process and the control space.
Assumptions on the set G are given in Section 4.
for t A ‰0; sŠ:
ꢀ4†
A1. On an open set containing G  U, bꢀÁ ; Á† is bounded by B and Lipschitz con-
tinuous with constant K. On an open set containing G, sꢀÁ† is bounded and Lipschitz
with the same constants.
A2. Either sꢀÁ† is a square matrix and uniformly nondegenerate, or we can parti-
tion bꢀÁ ; Á† and sꢀÁ† in the form
ꢂ
ꢃ
ꢂ
ꢃ
b1ꢀx; u†
b2ꢀx†
s1ꢀx†
bꢀx; u† ˆ
;
sꢀx† ˆ
;
0
where s1ꢀÁ† is a square matrix and uniformly nondegenerate.
A3. The control space U H Rm is compact and independent of time.
3. De®nition of the Di¨erential Game
The game to be considered is de®ned in the Elliot±Kalton sense [EK], with
the extra requirement that strategies be measurable. We now give the precise
de®nitions.
Consider a di¨erential game with two players. At each time t A ‰0; y† player I
picks an element uꢀt† from the control space U, and player II picks a control vꢀt†
from Rr in such a way that the functions t ! uꢀt† and t ! vꢀt† belong to the sets
M j fu: ‰0; y† ! U : u is measurableg

Â
M. Boue and P. Dupuis
66
and
ꢀ
ꢁ
ꢄ
T
2
N j v: ‰0; y† ! Rr :
kvꢀt†k dt < y for all T < y ;
0
respectively. We identify any two controls that are equal almost everywhere and
consider M and N as metric spaces under any metric which is equivalent to con-
vergence in L2ꢀ‰0; TŠ : Rm† and L2ꢀ‰0; TŠ : Rr†, respectively, for all T < y.
Given x A Rd, the dynamics of the game are given by the di¨erential equation
_
j ˆ bꢀj; u† ‡ sꢀj†v;
Associated with these dynamics is the cost
jꢀ0† ˆ x:
ꢀ5†
ꢄ
tx
1
2
Cxꢀu; v† j
kvꢀt†k dt ‡ ytx;
2
0
where tx j infft V 0 : jꢀt† B Gg and y > 0. The goal of player I is to maximize
CxꢀÁ ; Á† and the goal of player II is to minimize it.
Remark. The same symbols u and v are used to represent both deterministic
controls, as in (5), and stochastic ones, as in (1). The intended use in each case
should be clear.
A mapping a: N ! M is said to be a strategy for the maximizing player pro-
vided it is measurable (with respect to the underlying Borel s-algebra) and if for
^
each s V 0 and v; v A N,
^
vꢀt† ˆ vꢀt†
for a:e: 0 U t U s
for a:e: 0 U t U s:
implies
^
a‰vŠꢀt† ˆ a‰vŠꢀt†
Similarly, a mapping b: M ! N is a strategy for the minimizing player provided it
^
is measurable and if for each s V 0 and u; u A M,
^
uꢀt† ˆ uꢀt†
for a:e: 0 U t U s
implies
^
b‰uŠꢀt† ˆ b‰uŠꢀt†
for a:e: 0 U t U s:
We denote by G the set of all maximizing strategies and by L the set of all mini-
mizing ones. The lower value of the game and the upper value of the game are
de®ned by
IÀꢀx† j inf sup Cxꢀu; b‰uŠ†
b A L
u A M
and
‡
I ꢀx† j sup inf Cxꢀa‰vŠ; v†;
v A N
a A G
‡
respectively. If IÀꢀx† ˆ I ꢀx†, then the game has a value.

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
67
Robust Interpretation. For simplicity, in this subsection we consider dynamics
that are driven only by a minimizing control, that is,
_
j ˆ bꢀj† ‡ sꢀj†v;
The case with a maximizing control can be treated similarly by ®xing an optimal
jꢀ0† ˆ x:
(or nearly optimal) strategy a. De®ne
ꢀ
ꢁ
ꢄ
tx
1
2
2
Vyꢀx† j inf
kvꢀt†k dt ‡ ytx
;
v A N
0
where tx is, as before, the ®rst exit time of j from G. The reason for including the
dependence on the design parameter y will be clear below. For any v A N such
that kvky < y,
ꢄ
tx
1
2
Vyꢀx† U
kvꢀt†k dt ‡ ytx
2
0
2
U 12 txkvky ‡ ytx;
and thus
Vyꢀx†
tx V
:
2
1
2 kvky ‡ y
Suppose that vꢀt† j Dbꢀt†, where Db is interpreted as a modeling error. Moreover,
assume that the value of tx is used as a quantitative measure of the performance
of the system and that a prespeci®ed level L is demanded (i.e., it is required that
tx V L). If there exists a value of the design parameter yà > 0 such that
Ã
yÃ
Vy ꢀx†
> L;
then an upper bound on the norm of the modeling error can be obtained so that
the desired performance is guaranteed for all modeling errors satisfying this
bound. More precisely, if
ꢂ
ꢃ
Ã
Vy ꢀx†
2
kDbky U 2
À yÃ
;
L
then
Ã
Vy ꢀx†
tx V
V L;
2
kDbky ‡ yÃ
1
2
as demanded. We note that the size of the tolerable error varies inversely propor-
tional to L, i.e., robustness increases as the bound on the performance measure
decreases. This result is in a sense analogous to the small gain theorem [ZDG].
4. Asymptotic Analysis
The goal of this section is to relate the upper and lower values of the di¨erential
game described in Section 3 with the limit as e ! 0 of the risk-sensitive escape

Â
M. Boue and P. Dupuis
68
control criterion. The main result is contained in Theorem 4.6. Its proof relies on
a variational representation for the quantity in (3), stated in Lemma 4.5. The
proofs of both results are contained in Section 5.
Remark on Notation. In the remainder of the paper we deal with di¨erent pro-
cesses and their escape times from di¨erent sets. We now introduce the notation
which is used throughout. The set Gd is de®ned as follows: if d V 0, then
Gd ˆ fx A Rd : inffkx À yk : y A Gg < dg
ꢀ6†
and if d < 0, then
Gd ˆ fx A Rd : inffkx À yk : y B Gg > Àdg:
ꢀ7†
We use tu;v;e;d to denote the stopping time given by
t
u;v;e;d j infft V 0 : X u;v;eꢀt† B Gdg;
where X u;v;e j fX u;v;eꢀt†; t A ‰0; y†g is the solution to
dX u;v;e ˆ bꢀX u;v;e; u† dt ‡ sꢀX u;v;e†v dt ‡ e1=2sꢀX u;v;e† dW:
When d ˆ 0 we simply write tu;v;e. Under this notation, the process X u;e and the
escape time tu;e introduced in Section 1 are denoted by X u;0;e and tu;0;e, respec-
tively. Finally, the exit times of the deterministic process j (with initial condition
x) appearing in the di¨erential game introduced in the previous section are
denoted by txu;v;0;d when exit is from Gd and by txu;v;0 if exit is from the original
set G.
De®nition 4.1. We de®ne A to be the set of all Rr-valued F-progressively mea-
t
surable processes v j fvꢀt†; t A ‰0; y†g which satisfy
ꢄ
T
2
E
kvꢀt†k dt < y
for all T < y:
0
Lemma 4.5 is a consequence of the following theorem, whose proof can be
found in [BD]. For the sake of completeness, we also state two other results from
[BD] that will later be needed in the proof of Lemma 4.5.
Theorem 4.2. Fix T < y. Let f : Cꢀ‰0; TŠ : Rr† ! R be a Borel measurable
bounded function. Then
ꢀ
ꢅ
ꢆꢁ
ꢄ
ꢄÁ
T
1
2
2
Àlog EeÀf ꢀW† ˆ inf E
kvꢀs†k ds ‡ f W ‡ vꢀs† ds
:
v A A
0
0
Let m and n be probability measures on a measurable space ꢀX; F†. The rela-
tive entropy of m with respect to n is de®ned by
ꢅ
ꢆ
ꢄ
dm
dn
Rꢀmkn† ˆ
log
dm
X
if m is absolutely continuous with respect to n, and Rꢀmkn† ˆ y otherwise.

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
69
Lemma 4.3. Let ꢀX; F† be a measurable space, with X a Polish space and F
the associated Borel s-algebra. Let P be a probability measure de®ned on it and
let f : X ! R be a bounded Borel-measurable function. Consider a sequence
fmT ; T A Ng of measures in PꢀX† (the set of probability measures on ꢀX; F†) sat-
isfying supT A N RꢀmT kP† < y. Assume mT converges weakly as T ! y to a prob-
ability measure m. Then
ꢄ
ꢄ
lim
f dmT ˆ
f dm:
T!y
X
X
Lemma 4.4. For any v A A and T < y, let mv be the measure induced on
„
Cꢀ‰0; TŠ : Rr† by W ‡ 0Á vꢀt† dt under P. Then
ꢀ
ꢁ
ꢄ
T
1
2
2
RꢀmvkP† U E
kvꢀt†k dt :
0
The next lemma states that Theorem 4.2 continues to hold over an unbounded
time interval when applied to the escape time functional of a controlled di¨usion.
Lemma 4.5. Assume A1±A3. Fix e > 0 and consider any control u A F. For
every s A ‰0; y† let us be a Borel measurable function such that uꢀt† ˆ us
ꢀt; Wꢀr†; 0 U r U s† for t A ‰0; sŠ w.p.1 (see (4)). Further, let X u;0;e be the di¨usion
process that is the unique strong solution to
dX u;0;e ˆ bꢀX u;0;e; u† dt ‡ e1=2sꢀX u;0;e† dW;
X u;0;eꢀ0† ˆ x;
ꢀ8†
and de®ne
t
u;0;e j infft V 0 : X u;0;eꢀt† B Gg:
Then the following representation holds:
(
)
ꢀ
ꢁ
ꢄ
~
tu; v; e
ytu;0;e
1
2
2
Àe log Ex exp À
ˆ inf Ex
kvꢀt†k dt ‡ ytu~;v;e
;
ꢀ9†
v A A
e
0
where tu~;v;e j infft V 0 : Xu~;v;eꢀt† B Gg and Xu~;v;e is the solution to
u~;v;e
dXu~;v;e ˆ bꢀXu~;v;e; u† dt ‡ sꢀX
†v dt ‡ e1=2sꢀXu~;v;e† dW;
~
Xu~;v;eꢀ0† ˆ x:
~
The control u above is such that for any s A ‰0; y†,
ꢅ
ꢆ
ꢄ
r
1
s
~
uꢀt† ˆ u t; Wꢀr† ‡
vꢀz† dz; 0 U r U s
for t A ‰0; sŠ
ꢀ10†
e1=2
0
w.p.1, and this control is admissible.
Denote by Cxdꢀu; v† the cost associated with the di¨erential game described in
Section 3 but with the set G replaced by Gd as de®ned in (6)±(7). De®ne the upper
‡
and lower values of this game as I ꢀx; d† and IÀꢀx; d†, respectively. The next
``continuity'' assumption of the set G is needed in the proof of Theorem 4.6.

Â
M. Boue and P. Dupuis
70
‡
A4. For the given initial condition x, I ꢀx; d† and IÀꢀx; d† are continuous at d ˆ 0.
‡
Remark. The functions I ꢀx; Á† and IÀꢀx; Á† are monotone nondecreasing func-
tions of d, and hence each has a countable number of discontinuities. Conse-
quently, assumption A4 is not very restrictive. Even in the cases where the
functions fail to be continuous, A4 is satis®ed for an arbitrarily small perturbation
of the set G. Moreover, it is often not hard to check this condition for a particular
case.
The following theorem contains the main result of the paper. As was noted
earlier, it relates the upper and lower values of the di¨erential game described in
Section 3 with the limit as e ! 0 of the risk-sensitive escape control criterion. It
also establishes a connection between nearly supremizing strategies for the di¨er-
ential game and nearly minimizing controls for the risk-sensitive problem.
‡
Theorem 4.6. Given x A G, assume A1±A4. Let I ꢀx† and IÀꢀx† be the upper and
lower values of the di¨erential game described in Section 3. For any u A F, let
X u;0;e be the unique strong solution to (8). The following conclusions hold:
(i) lim supe!0 supu A F À e log Ex expfÀytu;0;e=eg U IÀꢀx†:
(ii) Given c > 0 there exists a measurable function g: Cꢀ‰0; y† : Rr† ! M with
the following properties:
^
^
(a) if 0 U s < y and f ꢀt† ˆ f ꢀt† for 0 U t U s, then g‰ f Šꢀt† ˆ g‰ f Šꢀt† a.e.
for 0 U t U s;
(b) if we de®ne u ˆ g‰e1=2WŠ, then u A F and
ꢀ
ꢁ
ytu;0;e
‡
lim inf Àe log Ex exp À
V I ꢀx† À c:
e!0
e
‡
(iii) I ꢀx† ˆ IÀꢀx†, so the game has a value.
5. Proofs
~
~
Proof of Lemma 4.5. Given u A F and v A A, let u satisfy (10). The control u is
admissible, since for each s A ‰0; y† the measure of the set where the composition
„
usꢀÁ ; Wꢀr† ‡ ꢀ1=e1=2
†
r vꢀz† dz; 0 U r U s† fails to be right-continuous is zero.
0
This in turn follows from the right-continuity of usꢀÁ ; j† and the fact that for any
„
v A A the measure induced by W ‡ ꢀ1=e1=2
†
0Á vꢀz† dz is absolutely continuous
with respect to P.
De®ne the mapping t : Cꢀ‰0; y† : Rd†
7
jꢀt† B Gg, so that tu;0;e ˆ tꢀX u;0;e†. Since X u;0;e is a strong solution to (8), there
exists a Borel measurable function h mapping Cꢀ‰0; y† : Rr† into Cꢀ‰0; y† : Rd†
such that X u;0;e ˆ h‰WŠ w.p.1 [KS, Corollary 5.3.23]. Borel measurability of t
(due to lower semicontinuity) implies that the composition H j t ꢁ h is a Borel
measurable functional of W. Thus for T < y we can write H 5 T as a bounded
Borel measurable functional of Wꢀt†, t A ‰0; TŠ. Applying Theorem 4.2 to this

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
71
functional we obtain
(
)
ꢀ
ꢁ
ꢄ
~
tu; v; e5T
yꢀtu;0;e 5 T†
1
2
2
Àe log Ex exp À
ˆ inf Ex
kvꢀt†k dt ‡ yꢀtu~;v;e 5 T† ;
v A A
e
0
„
where we have used the fact that H‰W ‡ ꢀ1=e1=2
†
0Á vꢀs† dsŠ ˆ tu~;v;e w.p.1. Since
tu;0;e 5 T " tu;0;e a.s. as T ! y, we can apply the Monotone Convergence The-
orem to get
(
)
ꢀ
ꢁ
ꢄ
~
tu; v; e5T
ytu;0;e
1
2
2
Àe log Ex exp À
ˆ lim inf Ex
kvꢀt†k dt ‡ yꢀtu~;v;e 5 T† :
T!y v A A
e
0
All that remains to be veri®ed is that the limit in the last expression coincides with
the right-hand side of (9).
~
~
The upper bound is immediate since ꢀtu;v;e 5 T† U tu;v;e for every T < y.
For the lower bound it su½ces to show that, given d > 0, every subsequence of
2
„
~
tu; v; e5T
0
1
finfv A A Exf2
kvꢀt†k dt ‡ yꢀtu~;v;e 5 T†g; T < yg has a subsubsequence
satisfying
(
)
ꢄ
~
tu; v; e5T
1
2
2
lim inf inf Ex
kvꢀt†k dt ‡ yꢀtu~;v;e 5 T†
T!y v A A
0
(
)
ꢄ
~
tu; v; e
1
2
2
V inf Ex
kvꢀt†k dt ‡ ytu~;v;e À d:
ꢀ11†
v A A
0
For the remainder of the proof we work with a ®xed subsequence, and for conve-
nience we relabel this subsequence by T.
For T < y, let vT A A satisfy
(
)
ꢄ
~
tu; v; e5T
1
2
2
~
inf Ex
kvꢀt†k dt ‡ yꢀtu;v;e 5 T†
v A A
0
(
)
T
T
ꢄ
~
; v ; e
tu
5T
1
2
1
T ;vT
2
V Ex
kvT ꢀv†k dt ‡ yꢀtu~
;e 5 T†
À
;
ꢀ12†
T
0
„
r vT ꢀz† dz;
T
s
1=2
~
where for s A ‰0; y† and t A ‰0; sŠ we have u ꢀt† ˆ u ꢀt; Wꢀr†‡ꢀ1=e
†
0
T ;vT
0 U r U s† w.p.1. If we rede®ne vT ꢀt† j 0 for t V tu~
;e 5 T, then vT still sat-
is®es the preceding inequality. We can assume
ꢀ
ꢁ
ꢄ
y
1
2
T ;vT ;e
2
lim inf Ex
kvT ꢀt†k dt < y and lim inf Exftu~
g < y;
T!y
T!y
0
since otherwise the inequality in (11) holds trivially. Consequently, at least for a
subsubsequence we have
ꢀ
ꢁ
ꢄ
y
1
2
T ;vT ;e
2
lim sup Ex
T!y
kvT ꢀt†k dt < y and lim sup Exftu~
g < y:
ꢀ13†
T!y
0

Â
M. Boue and P. Dupuis
72
„
t
2
For L < y and each T < y, let ST;L j infft V 0 : kvT ꢀs†k ds > Lg and
0
de®ne
ꢀ
vT ꢀt†
for t U ST;L
,
otherwise.
vT;Lꢀt† j
0
„
T ;vT ;e
T; L;vT; L;e
y
0
2
On the set fo :
kvT ꢀt†k dt U Lg, the escape times tu~
and tu~
T ;vT
T ;vT
agree. Since vT is zero after tu~
;e 5 T, it must be that ST;L U tu~
;e 5 T on
„
y
0
2
fo :
kvT ꢀt†k dt > Lg. Thus
T ;vT
Exftu~
;e 5 Tg
T; L;v
T; L
~
V Exfꢀtu
;e 5 T†Ifo:
‡ ST;LIfo:
dt>Lgg
„
„
y
y
2
2
kvT ꢀt†k dtULg
kvT ꢀt†k
0
0
T; L;vT; L
T; L;vT; L
ˆ Exftu~
;e 5 Tg À Exfꢀtu~
;e 5 T À ST;L†Ifo:
dt>Lgg:
kvT ꢀt†k
„
y
2
0
~
Let F ˆ F T; L T; L , and let E~ denote expectation conditioned on this s-
tu~
T
; v
; e
F
T
„
~
is F -measurable, we can combine the last
y
algebra. Since Ifo:
display with (12) to get
2
T
kvT ꢀt†k dt>Lg
0
(
)
ꢄ
~
tu; v; e5T
1
2
2
~
inf Ex
kvꢀt†k dt ‡ yꢀtu;v;e 5 T†
v A A
0
(
)
T; L T; L
~
ꢄ
; v
; e
tu
5T
1
2
1
T; L;vT; L
2
V Ex
kvT;Lꢀt†k dt ‡ yꢀtu~
;e 5 T†
À
T
0
T; L;vT; L
À yE fE ꢀtu~
;e 5 T À ST;L†Ifo:
dt>Lgg:
ꢀ14†
ꢀ15†
„
y
2
~
F
x
kvT ꢀt†k
T
0
Hence, inequality (11) will follow once we show that
(
)
T; L T; L
~
ꢄ
; v
; e
tu
5T
1
2
1
T; L;vT; L
2
lim inf Ex
kvT;Lꢀt†k dt ‡ yꢀtu~
;e 5 T†
À
T!y
T
0
(
)
ꢄ
tu~; v; e
1
2
2
V inf Ex
kvꢀt†k dt ‡ ytu~;v;e
;
v A A
0
and that for L < y large enough, the third line in (14) can be made no smaller
than Àd as T ! y.
We ®rst prove the latter claim. Since vT;L is zero after ST;L, it is easily veri®ed
T; L;vT; L;e
that the conditional mean escape time E ꢀtu~
À ST;L† is almost surely
~
F
T
uniformly bounded. Therefore, this claim follows if for some L < y the proba-
„
y
0
2
bility of the set fo :
kvT ꢀt†k dt > Lg is su½ciently small. However, for large
enough T this follows from Tchebychev's inequality and the ®rst bound in (13).
We ®nally show (15). The family fvT;L; T < yg takes values in the set
ꢀ
ꢁ
ꢄ
y
2
f A L2ꢀ‰0; y† : Rr† :
k f ꢀt†k dt U L ;
0

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
73
which is metrizable as a compact Polish space under the weak topology [KF]. It
„
Á
follows that the family fꢀW; 0 vT;Lꢀt† dt†; T < yg is tight and therefore that we
can extract a further subsequence (which we again relabel as T ) such that
„
„
Á
Á
ꢀW; 0 vT;Lꢀt† dt† converges in distribution to ꢀW; 0 vꢀt† dt† in ꢀCꢀ‰0; y† : Rr††2.
„
This in turn implies convergence in distribution of W ‡ 0Á vT;Lꢀt† dt to W ‡
„
„
0Á vꢀt† dt. L etmT;L and m be the measures induced by W ‡ ꢀ1=e1=2
†
0Á vT;Lꢀt† dt
„
and by W ‡ 0Á vꢀt† dt respectively under P. Then by Lemma 4.4 we have that
ꢀ
ꢁ
ꢄ
T
1
2
RꢀmT;LkP† U Ex
kvT;Lꢀt†k dt ;
2e
0
which combined with the ®rst bound in (13) gives
lim sup RꢀmT;LkP† < y:
T!y
Lemma 4.3 then implies that for any bounded Borel measurable function f,
ꢄ
ꢄ
lim
f ꢀj† dmT;Lꢀj† ˆ
f ꢀj† dmꢀj†:
Cꢀ‰0;y†:Rr†
Cꢀ‰0;y†:Rr†
T!y
Since for any bounded and continuous function g : Cꢀ‰0; y† : Rr†
7! R the com-
position g ꢁ H is bounded and measurable, it follows that HꢀW ‡ ꢀ1=e1=2† Á
„
„
0Á vT;Lꢀt† dt† converges in distribution to HꢀW ‡ ꢀ1=e1=2
†
0Á vꢀt† dt†, and hence
T; L;vT; L;e
that tu~
lemma to the second line of (14) now yields
converges in distribution to tu~;v;e. An application of Fatou's
(
)
T; L T; L
ꢄ
~
; v
; e
tu
5T
1
2
1
T; L;vT; L
2
lim inf Ex
kvT;Lꢀt†k dt ‡ yꢀtu~
;e 5 T†
À
T!y
T
0
(
)
ꢄ
~ ~
tu; v; e
1
2
2
V Ex
kvꢀt†k dt ‡ ytu~;v;e
0
(
)
ꢄ
~
tu; v; e
1
2
2
V inf Ex
kvꢀt†k dt ‡ ytu~;v;e
;
v A A
0
as we wanted to show.
9
Proof of Theorem 4.6. (i) Let u be any admissible control and for each
s A ‰0; y† let us be a Borel measurable function such that for t A ‰0; sŠ, uꢀt† ˆ
s
~
u ꢀt; Wꢀr†; 0 U r U s† w.p.1 (refer to (4)). For any e > 0 and v A A, Lemma 4.5
yields
(
)
ꢀ
ꢁ
ꢄ
~ ~
tu; v; e
ytu;0;e
1
2
2
u~;v~;e
~
Àe log Ex exp À
U Ex
kvꢀt†k dt ‡ yt
;
ꢀ16†
ꢀ17†
e
0
where tu~;v~;e is the ®rst exit time from G of the process Xu~;v~;e de®ned by
dXu~;v~;e ˆ bꢀXu~;v~;e; u† dt ‡ sꢀX
†v dt ‡ e1=2sꢀXu~;v~;e† dW;
u~;v~;e
~
~

Â
M. Boue and P. Dupuis
74
~
and for every s A ‰0; y†, u satis®es
ꢅ
ꢆ
ꢄ
r
1
s
~
~
uꢀt† ˆ u t; Wꢀr† ‡
vꢀz† dz; 0 U r U s
for t A ‰0; sŠ w:p:1:
e1=2
0
In light of the inequality in (16), in order to prove (i) it su½ces to show that given
any c > 0 there exists e1 > 0 such that for 0 < e U e1 and any u A F there exists
~
v A A satisfying
(
)
ꢄ
~ ~
tu; v; e
1
2
2
u~;v~;e
~
Ex
kvꢀt†k dt ‡ yt
U IÀꢀx† ‡ 2c:
ꢀ18†
0
Henceforth c > 0 is ®xed.
Let d > 0 satisfy
c
IÀꢀx; d† U IÀꢀx† ‡
:
ꢀ19†
2
~
The construction of a process v with the aforementioned properties requires the
introduction of a family fby; y A Gg. All the elements of this family are strat-
egies for the minimizing player which are related to the di¨erential game with
domain Gd.
Let fyi; i ˆ 1; . . . ; kg be an h-net for the set G, where the value of h > 0 is
speci®ed below. Also, let fGi; i ˆ 1; . . . ; kg be a Borel measurable partition of G
such that for all y A Gi we have ky À yik < h. Assumptions A1 and A2 and the
fact that G is bounded imply the existence of h > 0 and B1 < y (both indepen-
dent of h) such that for each yi, i ˆ 1; . . . ; k, there exists a strategy by A L sat-
i
isfying the following conditions:
(1) for each u A M, by ‰uŠꢀÁ† is constant on intervals of the form ‰nh; nh ‡ h†,
i
n V 0;
(2) supu A Mkb ‰uŠꢀt†k U B1 a.s. in t;
yi
(3) when considering the di¨erential game on the set G2d with initial condition
yi then
sup C2dꢀu; b ‰uŠ† U B1:
ꢀ20†
yi
yi
u A M
For each y A Gnfxg let i be the index such that y A Gi, and de®ne by j b . This
yi
de®nition guarantees that if a process starts at y and strategy by is used, the pro-
cess is driven out of Gd in bounded time and with bounded cost. Indeed, continu-
ity of the solution to (5) with respect to initial conditions and (20) imply that if
h > 0 is small enough
sup sup Cydꢀu; by‰uŠ† U B1:
ꢀ21†
u A M
y A G
y0x
This condition determines the value of h.
With the initial condition x we associate a strategy bx whose cost is not only
bounded, but is also nearly optimal on Gd, i.e.,
c
sup Cxdꢀu; bx‰uŠ† U IÀꢀx; d† ‡
:
ꢀ22†
2
u A M

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
75
In view of Lemma A2, it may be assumed that the strategy bx is such that for
every u A M we have
kbx‰uŠꢀt†k U B2
ꢀa:s: in t†
ꢀ23†
for some B2 < y. Moreover, there exists h > 0 such that bx‰uŠꢀÁ† is constant on
intervals of the form ‰nh; nh ‡ h†, n V 0. We ®x h > 0 small enough so that both
this condition and conditions (1)±(3) above hold simultaneously.
~
We now proceed with the de®nition of the process v for which we claim (18) is
satis®ed for small enough e. The construction is done in a recursive fashion. In
what follows u A F is ®xed and T j ꢀꢀIÀꢀx; d† ‡ c=2† 4 B1†=y. Without loss of
generality we assume that T=h is integer.
For t A ‰0; h† de®ne
~
vꢀt† ˆ bx‰uŠꢀ0†
and then set
ꢅ
ꢆ
ꢄ
r
1
e1=2
h
~
~
uꢀt† ˆ u t; Wꢀr† ‡
vꢀz† dz; 0 U r U h :
0
~
~
Proceed similarly on each segment ‰nh; nh ‡ h† by de®ning vꢀt† ˆ bx‰uŠꢀnh† and
„
r
nh‡h
ꢀt; Wꢀr† ‡ ꢀ1=e1=2
†
n ˆ 1; . . . ; T=h À 1.
~
uꢀt† ˆ u
~
vꢀz† dz; 0 U r U nh ‡ h†,
0
~
~
Once v and u have been de®ned on ‰0; T†, these controls determine the dynamics
of the di¨usion Xu~;v~;e through (17) over the interval ‰0; TŠ. In turn, the behavior of
Xu~;v~;e over this interval determines a strategy bꢀ1† in terms of which v will be
~
~ ~
de®ned over the next interval ‰T; 2TŠ: if the di¨usion Xu;v;e exits G before time T,
then the strategy bꢀ1† is de®ned to be identically zero; otherwise, bꢀ1† is de®ned to
ꢀ1†
be equal to the strategy by, where y ˆ Xu~;v~;eꢀT†. Using the strategy b , v and u
~
~
can be de®ned alternately on intervals of the form ‰T ‡ nh; T ‡ ꢀn ‡ 1†h† by
ꢀ1†
~
~
vꢀt† ˆ b ‰uŠꢀT ‡ nh†;
ꢅ
ꢆ
ꢄ
r
1
e1=2
T‡ꢀn‡1†h
~
~
uꢀt† ˆ u
t; Wꢀr† ‡
vꢀz† dz; 0 U r U T ‡ ꢀn ‡ 1†h
0
for n ˆ 0; . . . ; T=h À 1. The dynamics of Xu~;v~;e over ‰T; 2TŠ are again determined
by (17), and a strategy bꢀ2† for the next segment is de®ned as identically zero if
exit from G has occurred and as by otherwise, where y ˆ Xu~;v~;eꢀ2T†. The de®ni-
tion continues recursively in the obvious fashion.
~
Our ®rst observation is that the process v just de®ned is an element of A.
Indeed, since the construction is given in terms of strategies, v is automatically
~
~
adapted. Moreover, the strategies considered are all right-continuous, so v is pro-
~
~
gressively measurable. Since v is also bounded, v A A.
The ®nal step is to show that for e small enough, condition (18) is satis®ed. We
start by comparing Xu~;v~;e with the solution to j ˆ bꢀj; u† ‡ sꢀj†bx, jꢀ0† ˆ x over
_
~
the interval ‰0; TŠ. Assumption A1 implies that for any g > 0 there exists e0 > 0
(independent of u) such that dꢀXu~;v~;e; j† U d on a set of probability at least 1 À g
~
for all e A ꢀ0; e0Š. Since T V IÀꢀx; d† ‡ c=2, (22) implies that j exits Gd before time
T w.p.1 and therefore
Pxftu~;v~;e < Tg V 1 À g:

Â
M. Boue and P. Dupuis
76
Moreover, (22) also implies
"
#
ꢄ
tu~; v; e
1
2
c
2
u~;v~;e
kvꢀt†k dt ‡ yt
Iftu~; v~; e<Tg U IÀꢀx; d† ‡
:
~
2
0
Now consider the di¨usion process Xu~;v~;e over an interval ‰mT; ꢀm ‡ 1†TŠ
with m V 1, and suppose that Xu~;v~;eꢀmT† ˆ y. For t A ‰0; TŠ we compare
ꢀm†
Xu~;v~;eꢀt ‡ mT† with jꢀt†, where jꢀÁ† solves j ˆ bꢀj; u† ‡ sꢀj†b , jꢀ0† ˆ y.
Assumption A1 implies that for any g > 0 there exists e00 > 0 (independent of u~
and of y A G) such that for e A ꢀ0; e00Š supft A ‰0;TŠgkXu~;v~;eꢀt ‡ mT† À jꢀt†k U d on a
_
~
set of probability at least 1 À g. By (21), j exits Gd before time T w.p.1. Therefore
Pxftu~;v~;e < mT ‡ T j tu~;v~;e V mTg V 1 À g;
and we also have from (21) that
"
#
ꢄ
tu~; v~; e
1
2
2
kv~ꢀt†k dt ‡ yꢀtu~;v~;e À mT† Iftu~; v~; e<Tꢀm‡1†g U B1:
mT
De®ne e1 j e0 5 e00. If 0 < e U e1, then for all m V 1,
Pxftu~;v~;e V mTg U gPxftu~;v~;e V mT À Tg:
Iterating we obtain
Pxftu~;v~;e V mTg U gm:
Combining the above estimates with (19) and (23) we obtain that for 0 < e U e1,
(
)
ꢄ
tu~; v~; e
1
2
2
u~;v~;e
~
Ex
kvꢀt†k dt ‡ yt
0
ꢅ
ꢆ
y
X
c
B22
U IÀꢀx; d† ‡
‡
‡ y TPxftu~;v~;e V Tg ‡ B1
Pxftu~;v~;e V mTg
2
2
mˆ1
ꢅ
ꢆ
B22
B1g
U IÀꢀx† ‡ c ‡
‡ y Tg ‡
;
2
1 À g
which is less than IÀꢀx† ‡ 2c if g is small enough. Since this is precisely (18), the
proof of (i) is complete.
(ii) We start by observing that there exists B < y such that for any admissible
control u A F,
ꢀ
ꢁ
ytu;0;e
lim sup À e log Ex exp À
U B:
ꢀ24†
e
e!0
Indeed, for any T < y we have
ꢀ
ꢁ
ꢀ
ꢁ
ꢁ
ytu;0;e
yT
Ex exp À
V exp À
Pxftu;0;e U Tg
e
e
ꢀ
yT
V exp À
inf Pxftu;0;e U Tg:
u A F
e

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
77
Large deviations estimates for the minimum escape probability [DK] imply that
for any T < y there is some B < y so that
ꢀ
ꢁ
ytu;0;e
lim sup Àe log Ex exp À
U yT ‡ lim sup Àe log inf Pxftu;0;e U Tg
u A F
e!0
e
e!0
U B:
‡
‡
‡
Fix c A ꢀ0; I ꢀx†=3† and let d > 0 satisfy I ꢀx; Àd† V I ꢀx† À c=2. Let a A G be
a ꢀc=2†-optimal strategy for the maximizing player in the domain GÀd, so that
c
inf CxÀdꢀa‰vŠ; v† V I ꢀx; Àd† À V I ꢀx† À c:
ꢀ25†
‡
‡
v A N
2
Thanks to Lemma A1 in the Appendix, we may assume without loss of generality
that the strategy a is such that for every v A N, a‰vŠꢀÁ† is a right-continuous piece-
wise constant function.
The strategy a is used to de®ne a process that will serve as a control for the
di¨usion. Given any process v j fvꢀt†; t A ‰0; y†g and D > 0, de®ne
8
0
for t A ‰0; D†,
<
vDꢀt† j
vꢀnD† À vꢀnD À D†
:
for t A ‰nD; nD ‡ D†; n V 1.
D
With this de®nition, set
D
uꢀt† j a‰ꢀe1=2W† Šꢀt†:
The process u is measurable because of the measurability properties of a. More-
over, u is right-continuous (hence progressively measurable) and hence it is an
admissible control.
Consider any subsequence of the family fX u;0;e; e > 0g of solutions to (8) using
the control just de®ned. According to Lemma 4.5, for each e > 0 there is ve A A
which satis®es
(
)
e
~
ꢀ
ꢁ
ꢄ
; e
tu; v
ytu;0;e
1
2
e
2
Àe log Ex exp À
V Ex
kveꢀt†k dt ‡ ytu~;v ;e À c;
ꢀ26†
e
0
„
e
Á
D
where u ˆ a‰ꢀe1=2W ‡ 0 veꢀs† ds† Š and tu~;v ;e is the ®rst exit time from G of the
~
solution to
e
e
e
u~;ve;e
dXu~;v ;e ˆ bꢀXu~;v ;e; u† dt ‡ sꢀX
†ve dt ‡ e1=2sꢀXu~;v ;e† dW;
~
e
Xu~;v ;eꢀ0† ˆ x:
e
We rede®ne veꢀt† ˆ 0 for t V tu~;v ;e. Observe that (26) and the bound in (24) imply
ꢀ
ꢁ
ꢄ
y
1
e
2
lim sup Ex
e!0
kveꢀt†k dt ‡ ytu~;v ;e U B ‡ c < y:
ꢀ27†
2
0
e
In order to prove (ii) we compare Xu~;v ;e with the process Xu~;D;e which solves
ꢂ
ꢃ
ꢄÁ
D
u~;D;e
dXu~;D;e ˆ bꢀXu~;D;e; u† dt ‡ sꢀX
† e1=2W ‡ veꢀs† ds dt;
~
0
Xu~;D;eꢀ0† ˆ x:

Â
M. Boue and P. Dupuis
78
Consistent with our notation, we use tu~;D;e;Àd to denote the ®rst exit time from
GÀd of the process Xu~;D;e. De®ne
e
Y eꢀt† ˆ Xu~;v ;eꢀt† À Xu~;D;eꢀt†
and consider the event
ꢀ
ꢁ
e
d
WTe ;D ˆ fo : tu;v ;e V Tg W o : sup kY eꢀt†k V
~
2
0UtUT
ꢀ
(
ꢁ
ꢄ
T
e
2
W
W
o :
kW Dꢀt†k dt V c
2
0
*
+
)
ꢂ
ꢃ
ꢄ
ꢄÁ
veꢀs† ds Dꢀt† dt V c :
T
o : e1=2
W Dꢀt†;
0
0
Suppose that we can show that there exist T < y, D > 0, eà > 0, and a sub-
subsequence of fX u;0;e; e > 0g such that for all 0 U e U eà we have
c
PxfWTe ;Dg U
:
ꢀ28†
‡
I ꢀx† À 3c
Then for 0 U e U eà we can write the following string of inequalities, each line of
which is explained below:
ꢀ
ꢁ
ytu;0;e
Àe log Ex exp À
e
(
)
e
ꢄ
~
; e
tu; v
1
e
2
V Ex
V Ex
kveꢀt†k dt ‡ ytu~;v ;e À c
2
0
8
9
=
2
3
5
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
e
2
ꢂ
ꢃ
ꢄ
ꢄ
~
ꢄÁ
tu; v
; e
D
<
1
2
e
veꢀs† ds ꢀt† dt ‡ ytu~;v ;e IꢀW
À c
4
c
e
T; D
ꢇ
†
:
ꢇ
;
0
0
8
<
9
2
3
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
2
ꢂ
ꢃ
~
ꢄÁ
tu; D; e; Àd
D
=
1
2
veꢀs† ds ꢀt† dt ‡ ytu~;D;e;Àd IꢀW
À c
4
5
c
e
T; D
V Ex
V Ex
ꢇ
†
ꢇ
:
;
0
0
*
+!
ꢂꢀ
ꢄ
ꢂ
ꢃ
ꢄ
~
ꢄÁ
tu; D; e; Àd
À
ekW Dꢀt†k ‡ 2e1=2 W Dꢀt†;
veꢀs† ds Dꢀt†
dt
1
2
2
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
2
ꢂ
ꢃ
ꢁ
ꢃ
~
ꢄÁ
tu; D; e; Àd
D
ꢇ
ꢇ
1
e1=2W ‡ veꢀs† ds ꢀt† dt ‡ ytu~;D;e;Àd IꢀW
À c
c
e
T; D
‡
ꢇ
†
ꢇ
2
0
0
‡
c
e
V Ex‰ꢀI ꢀx† À 3c†I
Š À c
ꢀWT; D
†
ꢅ
ꢆ
c
‡
‡
V ꢀI ꢀx† À 3c† 1 À
À c ˆ I ꢀx† À 5c:
‡
I ꢀx† À 3c

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
79
The ®rst and second lines in the above display correspond to (26). The next line is
a consequence of Jensen's inequality and the nonnegativity of the integrands. The
e
fourth line is derived from the fact that tu~;v ;e V tu~;D;e;Àd for o A ꢀWTe ;D†c. Next,
lines ®ve and six are obtained by adding and subtracting the necessary quantities
~
„
Á
D
to complete the cost CxÀdꢀu; ‰e1=2W ‡ 0 veꢀs† dsŠ †. This cost satis®es (25) a.s. and
thus using the de®nition of WTe ;D we obtain line seven. Finally, the last line is a
consequence of (28).
De®ne g: Cꢀ‰0; y† : Rr† ! M by g‰jŠ j a‰jDŠ. Clearly, g satis®es condition (a)
in (ii). Moreover, we have shown that if we de®ne u j g‰e1=2WŠ and if (28) holds,
then every subsequence of fX u;0;e; e > 0g has a subsubsequence satisfying condi-
tion (b) in (ii). A standard argument by contradiction implies that the conclusion
is valid for the original sequence. In light of this, all that remains to be veri®ed for
the proof of (ii) to be complete is (28).
‡
Let g j c=7ꢀI ꢀx† À 3c†. The bound expressed in (27) enables us to ®nd
T < y, e0 > 0, and L < y such that for 0 < e U e0,
e
Pxfo : tu~;v ;e V Tg U g
ꢀ29†
ꢀ30†
and
(
)
ꢂ
ꢃ
ꢄ
1=2
y
2
Px
kveꢀt†k dt
V L U g:
0
We ®x T and L for the remainder of the proof. We also ®x the value of R j
expfKꢀT ‡ T1=2L†g (K is the Lipschitz constant appearing in assumption A1).
Using (27) and assumption A1, it can be veri®ed that the family
fXu~;D;e; e > 0; D > 0g is tight. Let mꢀsꢀXu~;D;e†; Á† denote the modulus of continu-
ity of sꢀXu~;D;e†. Then there exist n < y, 0 < e1 U e0, and D1 > 0 such that for any
0 < e U e1 and 0 < D U D1,
ꢀ ꢅ
ꢆ
ꢁ
1
d
Px m sꢀXu~;D;e†;
V
< g:
ꢀ31†
32T1=2LR
n
For this n we ®x D > 0 so as to satisfy D1=2 U D11=2 5 ‰d=ꢀ128nTBLR†Š (this choice
is required in (35)).
We now proceed to show that for the given values of T and D there exists eà for
which (28) holds. We analyze each of the sets in the de®nition of WTe ;D separately.
Since (29) bounds the probability of the ®rst set in the de®nition, we start by con-
„
T
2
sidering the set fo : ꢀe=2†
kW Dꢀt†k dt V cg, We have
0
(
)
ꢀ
ꢁ
ꢄ
‰T=DŠ
T
X
e
e
2
2
Px
kW Dꢀt†k dt V c U Px
D
kW DꢀiD†k V c :
2
2
0
iˆ1
The random variables W DꢀiD† are independent and identically distributed

Â
M. Boue and P. Dupuis
80
Nꢀ0; ꢀ1=D†Ir†, so an application of Tchebychev's inequality leads to
(
)
‰T=DŠ
X
e
Te
2
P
D
kW DꢀiD†k V c
U
< g
ꢀ32†
2
2cD
iˆ1
for any 0 < e U e2 with e2 U e1 5 2cDg=T.
„
„
T
Á
D
As for the event fo : e1=2
hW Dꢀt†; ‰ 0 veꢀs† dsŠ ꢀt†i dt V cg, Schwarz' in-
0
equality, the fact that ab U ꢀa2 ‡ b2†=2 and Jensen's inequality yield
*
+
ꢂ
ꢃ
ꢄ
ꢄÁ
T
e1=2
W Dꢀt†;
veꢀs† ds Dꢀt† dt
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢂ
ꢃ
ꢄ
ꢄÁ
D
T
U e1=2
kW Dꢀt†k
veꢀs† ds ꢀt† dt
ꢇ
ꢇ
ꢇ
ꢇ
0
0
ꢇ
T ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
2
ꢂ
ꢃ
ꢄ
ꢄ
ꢄ
ꢄÁ
D
T
e1=2
e1=2
2
U
kW Dꢀt†k dt ‡
veꢀs† ds ꢀt† dt
ꢇ
ꢇ
ꢇ
ꢇ
2
2
0
0
0
ꢄ
T
T
e1=2
e1=2
2
2
U
kW Dꢀt†k dt ‡
kveꢀt†k dt:
2
2
0
0
Using the same argument as that one leading to (32) for the ®rst term and Tche-
bychev's inequality for the second one (combined with (27)), we can ®nd e3 U e2
such that for 0 < e U e3 the last sum is bounded above by c on the complement of
a set of probability less than g.
Lastly, we consider the event fo : sup0UtUT kY eꢀt†k V d=2g. By de®nition, the
process Y e satis®es
ꢄ
t
e
Y eꢀt† ˆ ‰bꢀXu~;v ;eꢀs†; uꢀs†† À bꢀX
ꢀs†; uꢀs††Š ds
u~;D;e
~
~
0
ꢄ
t
e
‡
‡
‰sꢀXu~;v ;eꢀs†† À sꢀXu~;D;eꢀs††Šveꢀs† ds
0
"
#
ꢂ
ꢃ
ꢄ
ꢄÁ
sꢀXu~;D;eꢀs†† veꢀs† À
veꢀr† dr Dꢀs† ds
t
0
0
ꢄ
ꢄ
t
t
e
‡ e1=2 sꢀXu~;v ;eꢀs†† dWꢀs† À e1=2 sꢀXu~;D;eꢀs††W Dꢀs† ds:
0
0
Lipschitz continuity and boundedness of bꢀÁ ; Á† and sꢀÁ† yield
ꢄ
t
kY eꢀt†k U kY eꢀs†kꢀK ‡ Kkveꢀs†k† ds
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
"
#
ꢂ
ꢃ
ꢄ
ꢄÁ
D
t
‡
sꢀXu~;D;eꢀs†† veꢀs† À
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢄ
ꢄ
t
t
ꢇ
e
sꢀXu~;v ;eꢀs†† dWꢀs† ‡ e1=2B kW Dꢀs†k ds:
1=2
ꢇ
‡ e
ꢇ
ꢇ
0
0

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
81
Using Gronwall's inequality we obtain
ꢀ
ꢁ
ꢄ
ꢄ
sup kY eꢀt†k U e1=2
B
T kW Dꢀs†k ds ‡ exp T ꢀK ‡ Kkveꢀs†k† ds
0UtUT
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
"
#
ꢂ
ꢂ
ꢃ
ꢄ
ꢄÁ
D
t
Â
sup
0UtUT
sꢀXu~;D;eꢀs†† veꢀs† À
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢃ
ꢄ
t
e
1=2
‡ sup
e
sꢀXu~;v ;eꢀs†† dWꢀs†
:
ꢇ
0UtUT
0
Similar estimates as those leading to (32) give e small enough so that
ꢀ
ꢁ
ꢄ
Px e1=2 T kW Dꢀs†k ds V
U g:
ꢀ33†
d
4B
0
Next, the basic properties of the stochastic integral enable us to write
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢀ
ꢁ
ꢄ
t
e
d
1=2
Px sup
0UtUT
e
sꢀXu~;v ;eꢀs†† dWꢀs† V
ꢇ
8R
0
(
)
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢄ
2
T
64R2e
d2
e
U
Ex
sꢀXu~;v ;eꢀs†† dWꢀs†
0
ꢀ
ꢁ
ꢄ
T
64R2e
d2
e
2
ˆ
Ex
ksꢀXu~;v ;eꢀs††k ds
0
64R2B2Te
U
< g
ꢀ34†
d2
for 0 < e < gd2=64R2B2T. Finally, we turn our attention to the set
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
(
"
#
)
ꢂ
ꢃ
ꢄ
ꢄÁ
D
t
d
o : sup
0UtUT
sꢀXu~;D;eꢀs†† veꢀs† À
veꢀr† dr ꢀs† ds V
:
ꢇ
ꢇ
ꢇ
ꢇ
8R
0
0
Obtaining bounds on the probability of this set involves approximating
sꢀXu~;D;eꢀÁ†† by functions which are piecewise constant on intervals of the form
‰ j=n; ꢀj ‡ 1†=n†, with j V 0 and n < y de®ned by (31). De®ne
ꢅ
ꢅ ꢆꢆ
ꢂ
ꢆ
j
j j ‡ 1
sun~;D;eꢀs† j s Xu~;D;e
for s A
;
:
n
n
n
Simple calculations lead to the following string of inequalities, valid except on a

Â
M. Boue and P. Dupuis
82
set of probability less than 2g (see (30) and (31)):
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
"
#
ꢂ
ꢃ
ꢄ
ꢄÁ
D
t
sꢀXu~;D;eꢀs†† veꢀs† À
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
0
0
ꢄ
U
t ksꢀXu~;D;eꢀs†† À sun~;D;eꢀs†k kveꢀs†k ds
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
"
#
ꢂ
ꢃ
ꢄ
ꢄÁ
D
t
‡
snu~;D;eꢀs† veꢀs† À
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢂ
ꢃ
ꢄ
ꢄÁ
D
t
‡
ksꢀXu~;D;eꢀs†† À snu~;D;eꢀs†k
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
0
0
ꢂ
ꢃ
ꢂ
ꢃ
ꢄ
ꢄ
1=2
1=2
T
T
2
2
U 2
ksꢀXu~;D;eꢀs†† À snu~;D;eꢀs†k ds
kveꢀs†k ds
0
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
"
#
ꢅ
ꢅ ꢆꢆ
ꢂ
ꢃ
ꢄ
ꢄÁ
‰ntŠÀ1
D
ꢀ j‡1†=n
X
j
‡
s Xu~;D;e
veꢀs† À
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
n
j=n
0
jˆ0
ꢇ
ꢇ
ꢇ
ꢇ
"
#
ꢅ
ꢅ
ꢆꢆ
ꢂ
ꢃ
ꢄ
ꢇ
ꢄÁ
D
t
‰ntŠ
ꢇ
‡
s Xu~;D;e
veꢀs† À
veꢀr† dr ꢀs† ds
ꢇ
ꢇ
ꢇ
ꢇ
n
‰ntŠ=n
0
ꢂ
ꢅ
ꢆ
ꢃꢂ
ꢃ
ꢄ
1=2
T
1
2
U 2T1=2m sꢀXu~;D;e†;
‡ 4nTBD1=2
kveꢀs†k ds
n
0
3d
U
:
ꢀ35†
16R
If e4 > 0 is such that (30) and (33)±(35) hold, then for 0 U e < e4 we have
Pxfsup0UtUT kY eꢀt†k V d=2g U 4g. We can conclude that for e < eà j e3 5 e4
and T and D de®ned as before, we have PxfWTe ;Dg < 7g. Choosing g <
‡
c=7ꢀI ꢀx† À 3c† gives (28) and thus completes the proof of (ii).
(iii) It follows from (i) and (ii) that I ꢀx† U IÀꢀx†: We omit the proof of
‡
I ꢀx† V IÀꢀx†. It can be found in Lemma A4 of [DK].
9
‡
Appendix
In this appendix we include two lemmas that are used in the proof of Theorem
4.6. The proof of Lemma A1 follows the proof of Lemma A2 in [DK]. Since the
proof there contains a small error in the estimates, we include a corrected version
here.
Lemma A1. Let I, d > 0, and a A G be given such that
inf Cxꢀa‰vŠ; v† V I:
v A N
Then there exists a0 A G such that for all v A N; a0‰vŠꢀÁ† is a piecewise constant

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
83
function (without loss of generality we take it to be right-continuous) and
Cxdꢀa0‰vŠ; v† V I:
ꢀ36†
Proof. For h > 0 let fui; i ˆ 1; . . . ; Jg be an h-net of the control space U, and let
fUi; i ˆ 1; . . . ; Jg be a Borel measurable partition of U such that the Hausdor¨
distance between fuig and Ui is less than h for i ˆ 1; . . . ; J. L etT j I=y. For
g > 0 and 0 U l U T=g À 1 de®ne
ꢄ
lg‡g
tꢀi; l; v† j
Ifa‰vŠꢀt† A U g dt:
i
lg
Note that for all v A N and 0 U l U T=g À 1,
J
X
tꢀi; l; v† ˆ g:
iˆ1
Now de®ne a0‰vŠꢀt† j u1 for 0 U t U g and
"
!
iÀ1
i
X
X
a0‰vŠꢀt† ˆ ui
for t A lg ‡
tꢀj; l À 1; v†; lg ‡
tꢀj; l À 1; v† ;
jˆ1
jˆ1
l ˆ 1; . . . ; T=g À 1. For t V T, a0‰vŠꢀt† can be de®ned in any arbitrary way as long
as it is a strategy. It is easy to verify that for every j A Cꢀ‰0; TŠ : Rd† and all
v A N,
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢄ
t
0
sup
0UtUT
‰bꢀjꢀs†; a‰vŠꢀs†† À bꢀjꢀs†; a ‰vŠꢀs††Š ds U hKT ‡ 2gB ‡ 2KTmꢀj; g†;
ꢇ
0
ꢀ37†
where mꢀj; Á† is the modulus of continuity of j over ‰0; TŠ.
To prove the desired result we compare the processes j and c de®ned by
_
j ˆ bꢀj; a‰vŠ† ‡ sꢀj†v;
jꢀ0† ˆ x;
and
0
_
c ˆ bꢀc; a ‰vŠ† ‡ sꢀc†v;
cꢀ0† ˆ x;
respectively. Using (37) and the inequality ab U ꢀa2 ‡ b2†=2 we obtain
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢄ
t
0
kjꢀt† À cꢀt†k U
‰bꢀjꢀs†; a‰vŠꢀs†† À bꢀjꢀs†; a ‰vŠꢀs††Š ds
0
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢄ
ꢄ
t
0
0
‡
‰bꢀjꢀs†; a ‰vŠꢀs†† À bꢀcꢀs†; a ‰vŠꢀs††Š ds
0
ꢇ
ꢇ
t
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
ꢇ
‡
‰sꢀjꢀs††vꢀs† À sꢀcꢀs††vꢀs†Š ds
0
ꢄ
t kjꢀs† À cꢀs†k ds
3K
U hKT ‡ 2gB ‡ 2KTmꢀj; g† ‡
2
0
ꢄ
t
K
2
‡
kjꢀs† À cꢀs†k kvꢀs†k ds;
2
0

Â
M. Boue and P. Dupuis
84
and by Gronwall's inequality
sup kjꢀt† À cꢀt†k U ꢀhKT ‡ 2gB ‡ 2KTmꢀj; g†† exp‰Kꢀ2T ‡ I†Š:
0UtUT
This quantity can be bounded above by d if h and g are small enough. We thus
conclude that if v A N is such that jꢀÁ† escapes from G before time T, then cꢀÁ†
must exit Gd before time T, and thus Cxdꢀa0‰vŠ; v† V I. On the other hand, if v is
such that jꢀÁ† has not escaped G before time T, then c has not escaped Gd either,
and Cxdꢀa0‰vŠ; v† V I. In either case (36) holds and the proof of the lemma is
complete.
9
Lemma A2. Let I < y and b A L be given such that
sup Cxꢀu; b‰uŠ† U I:
u A M
Then given any d > 0 there exist g > 0, B1 < y, and b0 A L such that for each
u A M, b0‰uŠꢀÁ† is constant on intervals of the form ‰ng; ng ‡ g†, n V 0,
kb0‰uŠꢀt†k U B1
ꢀa:s: in t†
ꢀ38†
and
CxÀ2dꢀu; b0‰uŠꢀt†† U I:
ꢀ39†
Proof. According to Lemma A1 in [DK] (with T ˆ I=y in place of T ˆ 1 there),
there exists a strategy b1 A L such that for all u A M, b1‰uŠꢀÁ† is bounded with
constant B1, Lipschitz continuous with constant B2, and
ꢄ
ꢄ
I=y
I=y
2
2
kb1‰uŠꢀt†k dt U
kb‰uŠꢀt†k dt:
0
0
Furthermore, if we consider the processes j and c de®ned by
_
j ˆ bꢀj; u† ‡ sꢀj†b‰uŠ;
jꢀ0† ˆ x;
cꢀ0† ˆ x:
_
c ˆ bꢀc; u† ‡ sꢀc†b1‰uŠ;
then
sup kjꢀt† À cꢀt†k U d:
0UtUI=y
It follows that for all u A M,
CxÀdꢀu; b1‰uŠ† U Cxꢀu; b‰uŠ† U I:
For g > 0, u A M, and n V 0 de®ne
b0‰uŠꢀt† j b1‰uŠꢀng†
if t A ‰ng; ng ‡ g†:
Then b0 is obviously a strategy which satis®es (38). To show (39), we compare the
processes j and c which are solutions to
_
j ˆ bꢀj; u† ‡ sꢀj†b1‰uŠ;
jꢀ0† ˆ x;
cꢀ0† ˆ x:
0
_
c ˆ bꢀc; u† ‡ sꢀc†b ‰uŠ;

Risk-Sensitive and Robust Escape Control for Degenerate Di¨usion Processes
85
We have
ꢄ
t
kjꢀt† À cꢀt†k U Kꢀ1 ‡ B1†kjꢀs† À cꢀs†k ds
0
ꢄ
‡
t ksꢀcꢀs††b1‰uŠꢀs† À sꢀcꢀs††b0‰uŠꢀs†k ds:
0
Thanks to the estimate
ꢄ
t ksꢀcꢀs††b1‰uŠꢀs† À sꢀcꢀs††b0‰uŠꢀs†k ds
0
ꢄ
I=g
ng‡g
ng
X
gIB2B
U
B
kb1‰uŠꢀs† À b1‰uŠꢀng†k ds U
;
y
nˆ1
we can ®nd g > 0 small enough so that dꢀj; c† U d. The Dominated Convergence
Theorem then implies that the costs for b1 and b0 are su½ciently close for g small
enough, and therefore (39) follows.
9
References
[BJP] R. K. Boel, M. R. James, and I. R. Petersen, Robust Filtering, Technical Report, The
Australian National University, Canberra, 1996.
Â
[BD] M. Boue and P. Dupuis, A variational representation for certain functionals of Brownian
motion, Ann. Probab., 26 (1998), 1641±1659.
[DMR] P. Dai Pra, L. Meneghini, and W. J. Runggaldier, Some connections between stochastic
control and dynamic games, Math. Control Signals Systems, 9 (1996), 303±326.
[DE] P. Dupuis and R. S. Ellis, A Weak Convergence Approach to the Theory of Large Deviations,
Wiley, New York, 1997.
[DK] P. Dupuis and H. J. Kushner, Minimizing escape probabilities: a large deviations approach,
SIAM J. Control Optim., 27 (1989), 432±445.
[DM] P. Dupuis and W. M. McEneaney, Risk-sensitive and robust escape criteria, SIAM J. Control
Optim., 35 (1997), 2021±2049.
[EK] R. J. Elliot and M. J. Kalton, The existence of value in di¨erential games, Mem. Amer. Math.
Soc., 126 (1972).
Â
Â
[FH] W. H. Fleming and D. Hernandez-Hernandez, Risk-sensitive control of ®nite state machines
on an in®nite horizon, I, SIAM J. Control Optim., 35 (1997), 1790±1810.
[FS] W. H. Fleming and P. E. Souganidis, On the existence of value functions of two-player, zero-
sum stochastic di¨erential games, Indiana Univ. Math. J., 38 (1989), 293±314.
Â
Â
[HM] D. Hernandez-Hernandez and S. I. Marcus, Risk sensitive control of Markov processes in
countable state space, Systems Control Lett., 29 (1996), 147±155.
[KS] I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, 2nd edn., Springer-
Verlag, New York, 1991.
[KF] A. N. Kolmogorov and S. F. Fomin, Elements of the Theory of Functions and Functional
Analysis, Graylock Press, Rochester, NY, 1957.
[ZDG] K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Prentice-Hall, Upper
Saddle River, NJ, 1996.