Elements of Dynamic Optimization ( PDFDrive )

ELEMENTS
OF DYNAMIC
OPTIMIZATION
Alpha C. Chiang
Professor of Economics
The University of Connecticut
New York
Caracas
New Delhi
Lisbon
Paris
St. Louis
London
San Juan
McGraw-Hill, Inc.
San Francisco
Madrid
Singapore
Mexico
Auckland
Sydney
· Milan
Tokyo
Bogota
Montreal
Toronto
ELEMENTS O F DYNAMIC OPTIMIZATION
International Editions 1992
Exclusive rights by McGraw-Hill Book Co-Singapore for manufacture and
export. This book cannot be re-exported from the country to which it is
consigned by McGraw-Hill.
Copyright
©
1992 by McGraw-Hill, Inc. All rights reserved. Except as
permitted under the United States Copyright Act of 1976, no part of this
publication may be reproduced or distributed in any form or by any means,
or stored in a data base or retrieval system, without the prior written
permission of the publisher.
5 6 7 8 9 0 KHL SW 9 6 5
Library of Congress Cataloging-in-Publicaton Data
Chiang, Alpha C., (date).
Elements of dynamic optimization
p.
em.
I
Alpha C. Chiang.
Includes bibliographical references and index.
ISBN 0-07-010911-7
1. Mathematical optimization.
I. Title.
HB143.7.C45
2. Economics, Mathematical.
1992
330'.01'51'- dc20
91-37803
This book was set in Century Schoolbook by Science Typographers, Inc.
The editor was Scott D. Stratford;
the production supervisor was Denise L. Puryear.
The cover was designed by John Hite.
Project supervision was done by Science Typographers, Inc.
Art was prepared electronically by Science Typographers, Inc.
When ordering this title, use ISBN 0-07-112568-X.
Printed in Singapore.
CONTENTS
Preface
Part 1
1
1.1
1.2
1.3
1.4
Part 2
2
2.1
2.2
2.3
2.4
2.5
3
3.1
3.2
3.3
3.4
Introduction
The Nature of Dynamic Optimization
xi
1
3
Salient Features of Dynamic Optimization Problems
Variable Endpoints and Transversality Conditions
The Objective Functional
Alternative Approaches to Dynamic Optimization
4
8
12
17
The Calculus of Variations
25
The Fundamental Problem of the Calculus
of Variations
27
The Euler Equation
Some Special Cases
Two Generalizations of the Euler Equation
Dynamic Optimization of a Monopolist
Trading Off Inflation and Unemployment
28
37
45
49
54
Transversality Conditions
for Variable-Endpoint Problems
59
The General Transversality Condition
Specialized Transversality Conditions
Three Generalizations
The Optimal Adjustment of Labor Demand
59
63
72
75
vii
viH
CONTENTS
4
4.1
4.2
4.3
4.4
5
5.1
5.2
5.3
5.4
5.5
6
6.1
6.2
6.3
Part 3
7
Second-Order Conditions
79
Second-Order Conditions
First and Second Variations
79
81
91
95
Infinite Planning Horizon
98
The Concavity ;convexity Sufficient Condition
The Legendre Necessary Condition
Methodological Issues of Infinite Horizon
The Optimal Investment Path of a Firm
The Optimal Social Saving Behavior
Phase-Diagram Analysis
The Concavity /Convexity Sufficient Condition
Again
98
103
111
117
130
Constrained Problems
133
Four Basic Types of Constraints
134
144
148
Some Economic Applications Reformulated
The Economics of Exhaustible Resources
Optimal Control Theory
159
Optimal Control: The Maximum Principle
161
7.1
7.2
7.3
7.4
7.5
The Simplest Problem of Optimal Control
162
167
177
181
7.6
7.7
The Political Business Cycle
8
8.1
8.2
8.3
8.4
8.5
9
9.1
9.2
9.3
9.4
10
10.1
The Maximum Principle
The Rationale of the Maximum Principle
Alternative Terminal Conditions
The Calculus of Variations and Optimal Control
Theory Compared
Energy Use and Environmental Quality
191
193
200
More on Optimal Control
205
An Economic Interpretation of the Maximum Principle
Problems with Several State and Control Variables
Antipollution Policy
205
210
214
221
234
Infinite-Horizon Problems
240
Transversality Conditions
240
244
253
264
The Current-Value Hamiltonian
Sufficient Conditions
Some Counterexamples Reexamined
The Neoclassical Theory of Optimal Growth
Exogenous and Endogenous Technological Progress
Optimal Control with Constraints
275
Constraints Involving Control Variables
275
'·
PRJ:P'A.CE
10.2
10.3
10.4
10.5
The Dynamics of a Revenue-Maximizing Firm
State-Space Constraints
Economic Examples of State-Space Constraints
Limitations of Dynamic Optimization
Answers to Selected Exercise Problems
Index
ix
292
298
307
313
315
323
PREFAC E
In recent years I have received many requests to expand my Fundamental
Methods of Mathematical Economics to include the subject of dynamic
optimization. Since the existing size of that book would impose a severe
space constraint, I decided to present the topic of dynamic optimization in a
separate volume. Separateness notwithstanding, the present volume can be
considered as a continuation of Fundamental Methods of Mathematical
Economics.
As the title Elements of Dynamic Optimization implies, this book is
intended as an introductory text rather than an encyclopedic tome. While
the basics of the classical calculus of variations and its modern cousin,
optimal control theory, are explained thoroughly, differential games and
stochastic control are not included. Dynamic programming is explained in
the discrete-time form; I exclude the continuous-time version because it
needs partial differential equations as a prerequisite, which would have
taken us far afield.
Although the advent of optimal control theory has caused the calculus
of variations to be overshadowed, I deem it inadvisable to dismiss the topic
of variational calculus. For one thing, a knowledge of the latter is indispens­
able for understanding many classic economic papers written in the calcu­
lus-of-variations mold. Besides, the method is used even in recent writings.
Finally, a background in the calculus of variations facilitates a better and
fuller understanding of optimal control theory. The reader who is only
interested in optimal control theory may, if desired, skip Part 2 of the
present volume. But I would strongly recommend reading at least the
following: Chap. 2 (the Euler equation), Sec. 4.2 (checking concavity1 con­
vexity), and Sec. 5.1 (methodological issues of infinite horizon, relevant also
to optimal control theory).
Certain features of this book are worth pointing out. In developing the
Euler equation, I supply more details than most other books in order that
the reader can better appreciate the beauty of the logic involved (Sec. 2.1).
xi
xii
PREFACE
In connection with infinite-horizon problems, I attempt to clarify some
common misconceptions about the conditions for convergence of improper
integrals (Sec. 5.1). I also try to argue that the alleged counterexamples in
optimal control theory against infinite-horizon transversality conditions
may be specious, since they involve a failure to recognize the presence of
implicit fixed terminal states in those examples (Sec. 9.2).
To maintain a sense of continuity with Fundamental Methods of
Mathematical Economics, I have written this volume with a comparable
level of expository patience, and, I hope, clarity and readability. The dis­
cussion of mathematical techniques is always reinforced with numerical
illustrations, economic examples, and exercise problems. In the numerical
illustrations, I purposely present the shortest-distance problem-a simple
problem with a well-known solution-in several different alternative formu­
lations, and use it as a running thread through the book.
In the choice of economic examples, my major criterion is the suitabil­
ity of the economic models as illustrations of the particular mathematical
techniques under study. Although recent economic applications are natural
candidates for inclusion, I have not shied away from classic articles. Some
classic articles are not only worth studying in their own right, but also turn
out to be excellent for illustrative purposes because their model structures
are uncluttered with secondary complicating assumptions. As a by-product,
the juxtaposition of old and new economic models also provides an interest­
ing glimpse of the development of economic thought. For instance, from the
classic Ramsey growth model (Sec. 5.3) through the neoclassical growth
model of Cass (Sec. 9.3) to the Romer growth model with endogenous
technological progress (Sec. 9.4), one sees a progressive refinement in the
analytical framework. Similarly, from the classic Hotelling model of ex­
haustible resources (Sec. 6.3) to the Forster models of energy use and
pollution (Sec. 7. 7 and Sec. 8.5), one sees the shift in the focus of societal
concerns from resource-exhaustion to environmental quality. A comparison
of the classic Evans model of dynamic monopolist (Sec. 2.4) with the more
recent model of Leland on the dynamics of a revenue-maximizing firm (Sec.
10.2) also illustrates one of the many developments in microeconomic
reorientation.
In line with my pedagogical philosophy, I attempt to explain each
economic model in a step-by-step manner from its initial construction
through the intricacies of mathematical analysis to its final solution. Even
though the resulting lengthier treatment requires limiting the number of
economic models pre<;ented, I believe that the detailed guidance is desirable
because it serves to minimize the trepidation and frustration often associ­
ated with the learning of mathematics.
In the writing of this book, I have benefited immensely from the
numerous comments and suggestions of Professor Bruce A. Forster of the
University of Wyoming, whose keen eyes caught many sins of commission
and omission in the original manuscript. Since I did not accept all his
i·
'
PREFACE
xiii
suggestions, however, I alone should be held responsible for the remaining
imperfections. Many of my students over the years, on whom I tried the
earlier drafts of this book, also helped me with their questions and reac­
tions. Scott D. Stratford, my editor, exerted the right amount of encourage­
ment and pressure at critical moments to keep me going. And the coopera­
tive efforts of Joseph Murphy at McGraw-Hill, and Sarah Roesser, Cheryl
Kranz, and Ellie Simon at Science Typographers, Inc., made the production
process smooth as well as pleasant. Thanks are also due to Edward T.
Dowling for ferreting out some typographical errors that lurked in the
initial printing of the book. Finally, my wife Emily again offered me
unstinting assistance on manuscript preparation. To all of them, I am
deeply grateful.
Alpha C. Chiang
ELEMENTS OF DYNAMIC OPTIMIZATION
PART
1
INTRODUCTION
CHAPTER
1
THE
NATURE
OF DYNAMIC
OPTIMIZATION
Optimization i s a predominant theme i n economic analysis. For this reason,
the classical calculus methods of finding free and constrained extrema and
the more recent techniques of mathematical programming occupy an impor­
tant place in the economist's everyday tool kit. Useful as they are, such tools
are applicable only to static optimization problems. The solution sought in
single
such problems usually consists of a
optimal magnitude for every
choice variable, such as the optimal level of output per week and the optimal
price to charge for a product. It does not call for a schedule of optimal
sequential action.
In contrast, a
optimization problem poses the question of
dynamic
what is the optimal magnitude of a choice variable in each period of time
within the planning period (discrete-time case) or at each point of time in a
T] (continuous-time case). It is even possible to
given time interval, say
[0,
consider an infinite planning horizon, so that the relevant time interval is
oo)-literally "from here to eternity." The solution of a dynamic opti­
[0,
mization problem would thus take the form of an
optimal time path
for
every choice variable, detailing the best value of the variable today, tomor­
row, and so forth, till the end of the planning period. Throughout this book,
we shall use the asterisk to denote optimality. In particular, the optimal
time path of a (continuous-time) variable y will
I
be denoted by
y*(t).
3
4
PART 1: INTRODUCTION
1.1
SALIENT FEATURES OF DYNAMIC
OPTIMIZATION PROBLEMS
Although dynamic optimization is mostly couched in terms of a sequence of
time, it is also possible to envisage the planning horizon as a sequence of
stages in an economic process. In that case, dynamic optimization can be
viewed as a problem of multistage decision making. The distinguishing
feature, however, remains the fact that the optimal solution would involve
more than one single value for the choice variable.
Multistage Decision Making
_
The multistage character of dynamic optimization can be illustrated with a
simple discrete example. Suppose that a firm engages in transforming a
certain substance from an i nitial state A (raw material state) into a termi­
nal state Z (finished product state) through a five-stage production process.
In every stage, the firm faces the problem of choosing among several
possible alternative subprocesses, each entailing a specific cost. The ques­
tion is: How should the firm select the sequence of subprocesses through
the five stages in order to minimize the total cost?
In Fig. 1.1, we illustrate such a problem by plotting the stages
horizontally and the states vertically. The initial state A is shown by the
leftmost point (at the beginning of state 1); the terminal state Z is shown by
the rightmost point (at the end of stage 5). The other points B, C, . . . , K
show the various intermediate states into which the substance may be
transformed during the process. These points (A, B, ... , Z) are referred to
as vertic es. To indicate the possibility of transformation from state A to
state B, we draw an arc from point A to point B. The other arc AC shows
!�
-·
State
0 '-----+----+--1- Stage
1
2
3
4
5
�'-y---'-y----1
----J'-y-----J
Stage 1
FIGURE 1.1
Stage
2
Stage 3
Stage 4
Stage 5
5
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
that the substance can also be transformed into state C instead of state B.
Each arc is assigned a specific value-in the present example, a cost-shown
in a circle in Fig. 1.1. The first-stage decision is whether to transform the
raw material into state B (at a cost $2) or into state C (at a cost of $4), that
is, whether to choose arc AB or arc AC. Once the decision is made, there
will arise another problem of choice in stage 2, and so forth, till state Z is
reached. Our problem is to choose a connected sequence of arcs going from
left to right, starting at A and terminating at Z, such that the sum of the
values of the component arcs is minimized. Such a sequence of arcs will
constitute an optimal path.
The example in Fig. 1.1 is simple enough so that a solution may be
found by enumerating all the admissible paths from A to Z and picking the
one with the least total arc values. For more complicated problems, how­
ever, a systematic method of attack is needed. This we shall discuss later
when we introduce dynamic programming in Sect. 1.4. For the time being,
let us just note that the optimal solution for the present example is the path
ACEHJZ, with $14 as the minimum cost of production. This solution serves
to point out a very important fact: A myopic, one-stage-at-a-time optimiza­
tion procedure will not in general yield the optimal path! For example, a
myopic decision maker would have chosen arc AB over arc AC in the first
stage, because the former involves only half the cost of the latter; yet, over
the span of five stages, the more costly first-stage arc AC should be selected
instead. It is precisely for this reason, of course, that a method that can take
into account the entire planning period needs to be developed.
The Continuous-Variable Version
The example in Fig. 1. 1 is characterized by a discrete stage variable, which
takes only integer values. Also, the state variable is assumed to take values
belonging to a small finite set, {A, B, . . , Z). If these variables are continu.
State
z
I
I
I
I
A
I
I
I
I
I
o�------_,--8�
T
FIGURE 1.2
PART 1: INTRODUCTION
6
ous, we may instead have a situation as depicted in Fig. 1.2, where, for
illustration, we have drawn only five possible paths from A to Z. Each
possible path is now seen to travel through an infinite number of stages in
the interval [0,T]. There is also an infinite number of states on each path,
each state being the result of a particular choice made in a specific stage.
For concreteness, let us visualize Fig. 1.2 to be a map of an open
terrain, with the stage variable representing the longitude, and the state
variable representing the latitude. Our assigned task is to transport a load
of cargo from location A to location Z at minimum cost by selecting an
appropriate travel path. The cost associated with each possible path de­
pends, in general, not only on the distance traveled, but also on the
topography on that path. However, in the special case where the terrain is
completely homogeneous, so that the transport cost per mile is a constant,
the least-cost problem will simply reduce to a shortest-distance problem.
The solution in that case is a straight-line path, because such a path entails
the lowest total cost (has the lowest path value). The straight-line solution
is, of course, well known-so much so that one usually accepts it without
demanding to see a proof of it. In the next chapter (Sec. 2.2, Example 4), we
shall prove this result by using the calculus of variations, the classical
approach to the continuous version of dynamic optimization.
For most of the problems discussed in the following, the stage variable
will represent time; then the curves in Fig. 1.2 will depict time paths. As a
concrete example, consider a firm with an initial capital stock equal to A at
time 0, and a predetermined target capital stock equal to Z at timeT. Many
alternative investment plans over the time interval [0,T] are capable of
achieving the target capital at time T. And each investment plan implies a
specific capital path and entails a specific potential profit for the firm. In
this case, we can interpret the curves in Fig. 1.2 as possible capital paths
and their path values as the corresponding profits. The problem of the firm
is to identify the investment plan-hence the capital path-that yields the
maximum potential profit. The solution of the problem will, of course,
depend crucially on how the potential profit is related to and determined by
the configuration of the capital path.
From the preceding discussion, it should be clear that, regardless of
whether the variables are discrete or continuous, a simple type of dynamic
optimization problem would contain the following basic ingredients:
1
initial point and a given terminal point;
2 a set of admissible paths from the initial point to the terminal point;
3 a set of path values serving as performance indices (cost, profit, etc.)
4
a given
associated with the various paths; and
a specified objective-either to maximize or to minimize the path value
or performance index by choosing the
optimal path .
CHAPTER 1, THE NATURE OF DYNAMIC OPTIMIZATION
7
The Concept of a Functional
The relationship between paths and path values deserves our close atten­
tion, for it represents a special sort of mapping-not a mapping from real
numbers to real numbers as in the usual fu nction, but a mapping from
paths (curves) to real numbers (performance indices). Let us think of the
paths in question as time paths, and denote them by y 1(t), Y n(t), and so on.
Then the mapping is as shown in Fig. 1.3, where V1, Vn represent the
associated path values. The general notation for the mapping should there­
fore be V(y(t)]. But it must be emphasized that this symbol fundamentally
differs from the composite-function symbol g( f(x )]. In the latter, g is a
function of f, and f is in turn a function of x; thus, g is in the final
analysis a function of x. In the symbol V(y(t)), on the other hand, the y(t)
component comes as an integral unit-to indicate time paths-and there­
fore we should not take V to be a function of t. Instead, V should be
understood to be a function of "y(t)" as such.
To make clear this difference, this type of mapping is given a distinct
name: fu nctio nal . To further avoid confusion, many writers omit the "(t)"
part of the symbol, and write the functional a s V[y] or V{y), thereby
underscoring the fact that it is the change in the position of the entire y
path-the variatio n in they path-as against the change in t, that results
in a change in path value V. The symbol we employ is V[y ]. Note that when
the symbol y is used to indicate a certain state, it is suffixed, and appears as,
say, y(O) for the initial state or y(T ) for the terminal state. In contrast, in
the path connotation, the t in y(t) is not assigned a specific value. In the
following, when we want to stress the specific time interval involved in a
Set of admissible paths
(curves)
Set
of path values
(real line)
FIGURE 1.8
. PART 1:
8
$
$
MC
INTRODUCTION
MC
PI t-------J'--.!"'RI
P2._------�'--t- MR2
1
I
I
I
I
I
I
_._.-4�>-Q
Q'2 Q\
0 _______
0
__
I
.____._!
...._ Q
_
_
(b)
(a)
FIGURE
I
I
I
I
1.4
path or a segment thereof, we shall use the notation y[O, T] or y[O, T ). More
often, however, we shall simply use the y(t) symbol, or use the term "y
path". The optimal time path is then denoted by y*(t), or they* path.
As an aside, we may note that although the concept of a functional
takes a prominent place primarily in dynamic optimization, we can find
examples of it even in elementary economics. In the economics of the firm,
the profit-maximizing output is found by the decision rule MC = MR
(marginal cost = marginal revenue). Under purely competitive conditions,
where the MR curves are horizontal as in Fig. 1.4a, each MR curve can be
represented by a specific, exogenously determined price, P0• Given the MC
curve, we can therefore express the optimal output as Q* = Q*(P0), which
is a function, mapping a real number (price) into a real number (optimal
output). But when the MR curves are downward-sloping under imperfect
competition, the optimal output of the firm with a given MC curve will
depend on the specific position of the MR curve. In such a case, since the
output decision involves a mapping from curves to real numbers, we in fact
have something in the nature of a functional: Q* Q*[MR]. It is, of
course, precisely because of this inability to express the optimal output as a
function of price that makes it impossible to draw a supply curve for a firm
under imperfect competition, as we can for its competitive counterpart.
=
1 .2 VARIABLE ENDPOINTS
AND TRANSVERSALITY
CONDITIONS
In our earlier statement of the problem of dynamic optimization, we simpli­
fied matters by assuming a given initial point (a given ordered pair (0, A)}
and a given terminal point [a given ordered pair (T, Z)]. The assumption of
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
9
a given initial point may not be unduly restrictive, because, in the usual
problem, the optimizing plan must start from some specific initial position,
say, the current position. For this reason, we shall retain this assumption
throughout most of the book. The terminal position, on the other hand, may
very well turn out to be a flexible matter, with no inherent need for it to be
predetermined. We may, for instance, face only a fixed terminal time, but
have complete freedom to choose the terminal state (say, the terminal
capital stock). On the other hand, we may also be assigned a rigidly specified
terminal state (say, a target inflation rate), but are free to select the
terminal time (when to achieve the target). In such a case, the terminal
point becomes a part of the optimal choice. In this section we shall briefly
discuss some basic types of variable terminal points.
We shall take the stage variable to be continuous time. We shall also
retain the symbols 0 andT for the initial tim e and terminal tim e, and the
symbols A and Z for the initial and terminal states. When no confusion can
arise, we may also use A and Z to designate the initial and terminal points
(ordered pairs), especially in diagrams.
Types of Variable Terminal Points
As the first type of variable terminal point, we may be given a fixed terminal
timeT, but a free terminal state. In Fig. 1.5a, while the planning horizon is
fixed at time T, any point on the vertical line t T is acceptable as a
terminal point, such as Z1, Z2, and Z3• In such a problem, the planner
obviously enjoys much greater freedom in the choice of the optimal path
and, as a consequence, will be able to achieve a better-or at least no worse
-optimal path value, V*, than if the terminal point is rigidly specified.
This type of problem is commonly referred to in the literature as a
fixed-time-horizon p robl em, or fixed-tim e probl em, meaning that the termi­
nal time of the problem is fixed rather than free. Though explicit about the
time horizon, this name fails to give a complete description of the problem,
since nothing is said about the terminal state. Only by implication are we to
understand that the terminal state is free. A more informative characteriza­
tion of the problem is contained in the visual image in Fig. 1.5a . In line with
this visual image, we shall alternatively refer to the fixed-time problem as
the vertical-terminal -line probl em .
T o give an economic example of such a problem, suppose that a
monopolistic firm is seeking to establish a (smooth) optimal price path over
a given planning period, say, 12 months, for purpose of profit maximization.
The current price enters into the problem as the initial state. If there is no
legal price restriction in force, the terminal price will be completely up to
the firm to decide. Since negative prices are inadmissible, however, we must
eliminate from consideration all P < 0. The result is a truncated v ertical
term inal lin e, which is what Fig. 1.5a in fact shows. If, in addition, an
=
10
PART 1: INTRODUCTION
y
y
t=T
T
(a)
(b)
y
Z=,P(T)
(c)
FIGURE 1.5
official price ceiling is expected to be in force at the terminal time t = T,
then further truncation of the vertical terminal line is needed.
The second type of variable-terminal-point problem reverses the roles
played by the terminal time and terminal state; now the terminal state Z is
stipulated, but the terminal time is free. In Fig. 1.5b, the horizontal line
y = Z constitutes the set of admissible terminal points. Each of these,
depending on the path chosen, may be associated with a different terminal
time, as exemplified by T1, T 2, and T3• Again, there is greater freedom of
choice as compared with the case of a fixed terminal point. The task of the
planner might, for instance, be that of producing a good with a particular
quality characteristic (steel with given tensile strength) at minimum cost,
but there is complete discretion over the length of the production period.
This permits the rise of a lengthier, but less expensive production method
that might not be feasible under a rushed production schedule.
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
11
This type of problem i s commonly referred t o as a fixed-endpoint
A possible confusion can arise from this name because the word
"endpoint" is used here to designate only the terminal state Z, not the
entire endpoint in the sense of the ordered pair (T, Z). To take advantage of
the visual image of the problem in Fig. 1.5b, we shall alternatively refer to
the fixed-endpoint problem as the horizontal-terminal-line problem .
Turning the problem around, it is also possible in a problem of this
type to have minimum production time (rather than minimum cost) as the
objective. In that case, the path with T1 as the terminal time becomes
preferable to the one ending at T2, regardless of the relative cost entailed.
This latter type of problem is called a time-optimal problem.
In the third type of variable-terminal-point problem, neither the termi­
nal time T nor the terminal state Z is individually preset, but the two are
tied together via a constraint equation of the form Z = c/J(T). As illustrated
in Fig. 1.5c, such an equation plots as a terminal curve (or, in higher
dimension, a terminal surface) that associates a particular terminal time
(say, T1) with a corresponding terminal state (say, Z1). Even though the
problem leaves both T and Z flexible, the planner actually has only one
degree of freedom in the choice of the terminal point. Still, the field of
choice is obviously again wider than if the terminal point is fully prescribed.
We shall call this type of problem the terminal-curve (or terminal-surface)
problem .
problem .
To give an economic example of a terminal curve, we may cite the case
of a custom order for a product, for which the customer is interested in
having both ( 1) an early date of completion, and (2) a particular quality
characteristic, say, low breakability. Being aware that both cannot be
attained simultaneously, the customer accepts a tradeoff between the two
considerations. Such a tradeoff may appear as the curve Z = c/J(T) in Fig.
1.5c, where y denotes breakability.
The preceding discussion pertains to problems with finite planning
horizon, where the terminal time T is a finite number. Later we shall also
encounter problems where the planning horizon is infinite (T--+ oo) .
Transversality Condition
The common feature of variable-terminal-point problems is that the planner
has one more degree of freedom than in the fixed-terminal-point case. But
this fact automatically implies that, in deriving the optimal solution, an
extra condition is needed to pinpoint the exact path chosen. To make this
clear, let us compare the boundary conditions for the optimal path in the
fixed- versus the variable-terminal-point cases. In the former, the optimal
path must satisfy the boundary (initial and terminal) conditions
y(O) =A
and
y(T)
=
Z
In the latter case, the initial condition
( T, A, and Z all given)
y(O) =A still applies by assumption.
12
PART 1 , INTRODUCTION
But since T andjor Z are now variable, the terminal condition y(T) = Z is
no longer capable of pinpointing the optimal path for us. As Fig. 1.5 shows,
all admissible paths, ending at Z1, Z2, or other possible terminal positions,
equally satisfy the condition y(T) Z. What is needed, therefore, is a
terminal condition that can_conclusively distinguish the optimal path from
the other admissible paths. Such a condition is referred to as a transversal­
ity condition, because it normally appears as a description of how the
optimal path crosses the terminal line or the terminal curve (to "transverse"
means to "to go across").
=
Variable Initial Point
Although we have assumed that only the terminal point can vary, the
discussion in this section can be adapted to a variable initial point as well.
Thus, there may be an initial curve depicting admissible combinations of the
initial time and the initial state. Or there may be a vertical initial line t = 0,
indicating that initial time 0 is given, but the initial state is unrestricted. As
an exercise, the reader is asked to sketch suitable diagrams silnilar to those
in Fig. 1.5 for the case of a variable initial point.
If the initial point is variable, the characterization of the optimal path
must also include another transversality condition in place of the equation
y(O) = A, to describe how the optimal path crosses the initial line or initial
curve.
EXERCISE 1.2
1
2
Sketch suitable diagrams similar to Fig. 1.5 for the case of a variable initial
point.
In Fig. 1.5a, suppose that y represents the price variable. How would an
official price ceiling that is expected to take effect at time t = T affect the
diagram?
3 In Fig. 1.5c, let y denote heat resistance in a product, a quality which takes
longer production time to improve. How would you redraw the terminal
curve to depict the tradeoff between early completion date and high heat
resistance?
1.3 THE OBJECTIVE FUNCTIONAL
The Integral Form of Functional
An
optimal path is, by definition, one that maximizes or minimizes the path
value V[y ]. Inasmuch as any y path must perforce travel through an
interval of time, its total value would naturally be a sum. In the discrete-
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
13
stage framework of Fig. 1.1, the path value is the sum of the values of its
component arcs. The continuous-time counterpart of such a sum is a
definite integral, J;{ (arc value) dt. But how do we express the "arc value"
for the continuous case?
To answer this, we must first be able to identify an "arc" on a
continuous-time path. Figure 1.1 suggests that three pieces of information
are needed for arc identification: (1) the starting stage (time), (2) the
starting state, and (3) the direction in which the arc proceeds. With continu­
ous time, since each arc is infinitesimal in length, these three items are
represented by, respectively: (1) t, (2) y(t), and (3) y' (t) = dy jdt. For
instance, on a given path y1, the arc associated with a specific point of time
t0 is characterized by a unique value y1(t0) and a unique slope y{(t0). If
there exists some function, F, that assigns arc values to arcs, then the value
of the said arc can be written as F[t0, y1(t0), y {( t0 )] . Similarly, on another
path, y11, the height and the slope of the curve at t = t0 are y11(t0) and
yJ(t0), respectively, and the arc value is F[t0, y11(t0), y1{(t0)]. It follows that
the general expression for arc values is F[t, y(t), y'(t)}, and the path-value
functional-the sum of arc values-can generally be written as the definite
integral
(1.1)
V[yJ
=
f TF [ t , y ( t ) , y'(t) ] dt
0
It bears repeating that, as the symbol V[y] emphasizes, it is the variation in
they path (y1 versus y11) that alters the magnitude of V. Each different y
path consists of a different set of arcs in the time interval [0, T], which,
through the arc-value-assigning function F, takes a different set of arc
values. The definite integral sums those arc values on each y path into a
path value.
If there ¥e two state variables, y and z, in the problem, the arc values
on both the y and z paths must be taken into account. The objective
functional should then appear as
( 1 .2)
V[y, z ]
=
foTF [ t , y ( t ) , z(t),y'( t ) , z' ( t ) ] dt
A problem with an objective functional in the form of (1.1) or ( 1.2)
constitutes the standard problem. For simplicity, we shall often suppress
the time argument (t) for the state variables and write the integrand
function more concisely as F(t, y, y') or F(t, y, z, y', z').
A Microeconomic Example
(1.1) may arise, for instance, in the
case of a profit-maximizing, long-range-planning monopolistic firm with a
dynamic demand function Qd = D(P, P'), where P' = dPjdt. In order to
A functional of the standard form in
14
PART 1: INTRODUCTION
set Q, = Qd (to allow no inventory accumulation or decumulation), the
firm's output should be Q = D(P, P'), so that its total-revenue function is
R =PQ =R(P,P')
Assuming that the total-cost function depends only on the level of output,
we can write the composite function
C
=
C(Q) = C[D(P, P')]
It follows that the total profit also depends on P and P':
7r = R - C
Summing
functional
7r
= R(P,
P')- C[D(P, P')]
=
7r(P, P')
over, say, a five-year period, results in the objective
t71'(P, P') dt
0
t
which conforms to the general form of (1.1), except that the argument
in
the F function happens to be absent. However, if either the revenue
function or the cost function can shift over time, then that function should
contain t as a separate argument; in that case the 71' function would also
have t as an argument. Then the corresponding objective functional
t7T
(t, P, P') dt
0
would be exactly in the form of (1.1). As another possibility, the variable t
can enter into the integrand via a discount factor e-pt.
To each price path in the time interval [0, 5), there must correspond a
particular five-year profit figure, and the objective of the firm is to find the
optimal price path P*[O, 5] that maximizes the five-year profit figure.
A Macroeconomic Example
Let the social welfare of an economy at any time be measured by the utility
from consumption, U = U(C). Consumption is by definition that portion of
output not saved (and not invested). If we adopt the production function
Q = Q(K, L), and assume away depreciation, we can then write
C
= Q(
K,
L) - I
= Q(K, L)
- K'
where K' = I denotes net investment. This implies that the utility function
be rewritten as
can
U(C)
=
U[Q(K, L)- K']
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
15
If the societal goal is to maximize the sum of utility over a period
[0, T], then its objective functional takes the form
fTU [ Q( K, L) - K') dt
0
This exemplifies the functional in (1.2), where the two state variables y and
refer in the present example to K and L, respectively.
Note that while the integrand function of this example does contain
both K and K' as arguments, the L variable appears only in its natural
form unaccompanied by L'. Moreover, the t argument is absent from the F
function, too. In terms of (1.2), the F function contains only three argu­
ments in the present example: F[y(t), z(t),y'(t)], or F[K, L, K'].
z
Other Forms of Functional
Occasionally, the optimization criterion in a problem may not depend on
any intermediate positions that the path goes through, but may rely exclu­
sively on the position of the terminal point attained. In that event, no
definite integral arises, since there is no need to sum the arc values over an
interval. Rather, the objective functional appears as
V [y)
(1.3)
= G[T, y( T ) ]
where the G function is based on what happens at the terminal time T
only.
It may also happen that both the definite integral in (1.1) and the
terminal-point criterion in (1.3) enter simultaneously in the objective func­
tional. Then we have
(1.4)
V[y]
=
f0TF(t ,y(t) ,y' ( t) ] dt + G (T,y( T ) ]
where the G function may represent, for instance, the scrap value of some
capital equipment. Moreover, the functionals in (1.3) and ( 1.4) can again be
expanded to include more than one state variable. For example, with two
state variables y and z, (1.3) would become
(1.5)
V[y, z]
= G [ T,y( T ) , z(T)]
A problem with the type of objective functional i n (1.3) is called a
problem of Mayer. Since only the terminal position matters in V, it is also
known as a terminal-control problem. If (1.4) is the form of the objective
functional, then we have a problem of BoZza.
PART 1: lNTRODUCTION
16
Although the problem of Bolza may seem to be the more general
formulation, the truth is that the three types of problems-standard,
Mayer, and Bolza-are all convertible into one another. For example, the
functional (1. 3) can be transformed into the form (1.1) by defining a new
variable
( 1 . 6)
z( t)
=
G[t, y( t)]
It should be noted that it is
function in (1.6). Since
with initial condition
"t"
rather than
"T"
z( O) = 0
that appears in the
G
( 1 .7)
(z'( t ) dt =z ( t ) i� = z ( T) - z ( O)
0
= G[T,y(T)]
=
z ( T)
[by the initial condition ]
[by ( 1 .6) ]
the functional in (1. 3), G[T, y(T)], can be replaced by the integral in (1.7) in
the new variable z(t). The integrand, z'(t) = dG[t, y(t)]jdt, is easily recog­
nized as a special case of the function F[t, z(t), z'( t)], with the arguments t
and z( t) absent; that is, the integral in (1.7) still falls into the general form
of the objective functional (1.1). Thus we have transformed a problem of
Mayer into a standard problem. Once we have found the optimal z path, the
optimal y path can be deduced through the relationship in (1.6).
By a similar procedure, we can convert a problem of Bolza into a
standard problem; this will be left to the reader. An economic example of
this type of problem can be found in Sec. 3.4. In view of this convertibility,
we shall deem it sufficient to couch our discussion primarily in terms of the
standard problem, with the objective functional in the form of an integral.
EXERCISE 1.3
1
In a so-called "time-optimal problem," the objective is to move the state
variable from a given initial value y(O) to a given terminal value y(T) in
the least amount of time. In other words, we wish to minimize the
functional V[y] = T - 0.
(a) Taking it as a standard problem, write the specific form of the F
function in (1.1) that will produce the preceding functional.
(b) Taking it as a problem of Mayer, write the specific form of the G
function in (1.3) that will produce the preceding functional.
2 Suppose we are given a function D(t) which gives the desired level of the
state variable at every point of time in the interval [0, T ]. All deviations
from D(t), positive or negative, are undesirable, because they inflict a
negative payoff (cost, pain, or disappointment). To formulate an appropriate
dynamic minimization problem, which of the following objective functionals
, QJIAPTBJl l :
17
THE NATURE OF DYNAMIC OPTIMIZATION
would be acceptable? Why?
(a) f[!y(t) - D(t)) dt
(b) /0T[y(t) - D(t))2 dt
(c) f[ID(t) - y(t)P dt
(d) f:iD(t) - y(t)l dt
3
4
Transform the objective functional (1.4) of the problem of Bolza into the
format of the standard problem, as in (L 1) or (1.2). [ Hint: Introduce a new
variable z(t) = G[t, y(t)), with initial condition z(O)
0.]
=
Transform the objective functional (1.4) of the problem of Bolza into the
format of the problem of Mayer, as in (1.3) or (1.5). [Hint: Introdu�e a new
variable z(t) characterized by z'(t) = F[t, y(t), y'(t)), with initial condition
z(O)
= 0.)
1.4 ALTERNATIVE APPROACHES TO
DYNAMIC OPTIMIZATION
To tackle the previously stated problem of dynamic optimization, there are
three major approaches. We have earlier mentioned the calculus of varia­
tions and dynamic programming. The remaining one, the powerful modern
generalization of variational calculus, goes under the name of optimal
control theory. We shall give a brief account of each.
,,
The Calculus of Variations
Dating back to the late 17th century, the calculus of variations is the
classical approach to the problem. One of the earliest problems posed is that
of determining the shape of a surface of revolution that would encounter
the least resistance when moving through some resisting medium (a surface
of revolution with the minimum area).1 Issac Newton solved this problem
and stated his results in his Principia, published in 1687. Other mathe­
maticians of that era (e.g., John and James Bernoulli) also studied problems
of a similar nature. These problems can be represented by the following
general formulation:
( 1.8)
=
j TF[ t, y( t ) , y' (t)] dt
Maximize or minimize
VJ:y]
subject to
y(O) =A
and
y( T )
1This problem will be discussed in Sec. 2.2, Example
0
= Z
3.
(A given)
( T, Z given)
18
PART [, INTRODUCTION
Such a problem, with an integral functional in a single state variable, with
completely specified initial and terminal points, and with no constraints, is
known as the fundamental problem (or simplest problem) of calculus o£
variations.
In order to make such problems meaningful, it is necessary that the
functional be integrable (i.e., the integral must be convergent). We shall
assume this condition is met whenever we write an integral of the general
form, as in (1.8). Furthermore, we shall assume that all the functions that
appear in the problem are continuous and continuously differentiable. This
assumption is needed because the basic methodology underlying the calcu­
lus of variations closely parallels that of the classical differential calculus.
The main difference is that, instead of dealing with the differential dx that
changes the value of y = f(x), we will now deal with the " variation" of an
entire curve y(t) that affects the value of the functional V[y ]. The study of
variational calculus will occupy us in Part 2.
Optimal Control Theory ·
The continued study of variational problems has led to the development of
the more modern method of optimal control theory. In optimal control
theory, the dynamic optimization problem is viewed as consisting of three
(rather than two) types of variables. Aside from the time variable t and the
state variable y(t), consideration is given to a control variable u(t). Indeed,
it is the latter type of variable that gives optimal control theory its name
and occupies the center of stage in this new approach to dynamic optimiza­
tion.
To focus attention on the control variable implies that the state
variable is relegated to a secondary status. This would be acceptable only if
the decision on a control path u(t) will, once given an initial condition on y,
unambiguously determine a state-variable path y(t) as a by-product. For
this reason, an optimal control problem must contain an equation that
relates y to u :
dy
dt
= f [ t , y(t) , u(t)]
Such an equation, called an equation of motion (or transition equation or
state equation), shows how, at any moment of time, given the value of the
state variable, the planner's choice of u will drive the state variable y over
time. Once we have found the optimal control-variable path u*(t), the
equation of motion would make it possible to construct the related optimal
state-variable path y*(t).
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
19
The optimal control problem corresponding
tions problem (1.8) is as follows:
Maximize or minimize
( 1 .9)
=
f [ t, y(t), u ( t)J
y(O) = A
y( T )
and
the calculus-of-varia­
V[ u ] = foTF [ t , y(t), u ( t ) ) dt
y'( t)
subject to
to
= Z
(A given)
(T, Z given)
Note that, in (1.9), not only does the objective functional contain u as an
argument, but it has also been changed from V[y] to V[ u ]. This reflects the
fact that u is now the ultimate instrument of optimization. Nonetheless,
this control problem is intimately related to the calculus-of-variations prob­
lem (1.8). In fact, by replacing y'(t) with u(t) in the integrand in (1.8), and
adopting the differential equation y'(t) = u(t) as the equation of motion, we
immediately obtain ( 1.9).
The single most significant development in optimal control theory is
known as the maximum principle. This principle is commonly associated
with the Russian mathematician L. S. Pontryagin, although an American
mathematician, Magnus R. Hestenes, independently produced comparable
work in a Rand Corporation report in 1949.2 The powerfulness of that
principle lies in its ability to deal directly with certain constraints on the
control variable. Specifically, it allows the study of problems where the
admissible values of the control variable u are confined to some closed,
bounded convex set �. For instance, the set � may be the closed interval
[0, 1 ], requiring 0 � u(t) � 1 throughout the planning period. If the marginal
propensity to save is the control variable, for instance, then such a con­
straint, 0 � s(t) � 1 , may very well be appropriate. In sum, the problem
2The maximum principle is the product of the joint efforts of L. S. Pontryagin and his
associates V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko, who were jointly
awarded the 1962 Lenin Prize for Science and Technology. The English translation of their
work, done by K. N. Trirogoff, is
The Mathematical Theory of Optimal Processes, Interscience,
New York, 1962.
Hestenes' Rand report is titled A General Problem in the Calculus of Variations with
Applications to Paths of Least Time. But his work was not easily available until he published a
paper that extends the results of Pontryagin: "On Variational Theory and Optimal Control
Theory," Journal of SIAM, Series A, Control , Vol. 3, 1 96 5, pp. 23-48. The expanded version of
this work is contained in his book
Calculus of Variations and Optimal Control Theory, Wiley,
New York, 1966 .
Some writers prefer to call the principle
the minimum principle, which is the more
appropriate name for the principle in a slightly modified formulation of the problem. We shall
use the original name, the maximum principle.
20
PART 1: INTRODUCTION
addressed by optimal control theory is (in its simple form) the same as in
(1.9), except that an additional constraint,
u ( t)
E
�
for 0 :::;;
t ::5 T
may be appended to it. In this light, the control problem (1.9) constitutes a
special (unconstrained) case where the control set � is the entire real line.
The detailed discussion of optimal control theory will be undertaken in
Part 3.
Dynamic Programming
Pioneered by the American mathematician Richard Bellman,3 dynamic
programming presents another approach to the control problem stated in
(1.9). The most important distinguishing characteristics of this approach
are two: First, it embeds the given control problem in a family of control
problems, with the consequence that in solving the given problem, we are
actually solving the entire family of problems. Second, for each member of
this family of problems, primary attention is focused on the optimal value of
the functional, V*, rather than on the properties of the optimal state path
y*(t) (as in the calculus of variations) or the optimal control path u*(t) (as
in optimal control theory). In fact, an optimal value function-assigning an
optimal value to each individual member of this family of problems-is used
as a characterization of the solution.
All this is best explained with a specific discrete illustration. Referring
to Fig. 1.6 (adapted from Fig. 1.1), let us first see how the "embedding" of a
problem is done. Given the original problem of finding the least-cost path
from point A to point Z, we consider the larger problem of finding the
least-cost path from each point in the set {A, B, C, . . . , Z} to the terminal
point Z. There then exists a family of component problems, each of which is
associated with a different initial point. This is, however, not to be confused
with the variable-initial-point problem in which our task is to select one
initial point as the best one. Here, we are to consider every possible point as
a legitimate initial point in its own right. That is, aside from the genuine
initial point A, we have adopted many pseudo initial points ( B, C, etc). The
problem involving the pseudo initial point Z is obviously trivial, for it does
not permit any real choice or control; it is being included in the general
problem for the sake of completeness and symmetry. But the component
3Richard E. Beilman, Dynamic Programming, Princeton University Press, Princeton, NJ,
1957.
CHAPTER
1 : THE NATURE OF DYNAMIC OPTIMIZATION
21
State
A
o
.,
I
I
I
I
I
____ ____ _
f
.L__ _ ;
1
'---y---1
Stage
..
2
f. ___
- ---,.-- +
3
+
4
5
'---y---1'-----y-J
Stage 4
1
. ,. s�
Stage 5
FIGURE 1.6
problems for the other pseudo initial points are not trivial. Our Origi­
nal problem has thus been " embedded" in a family of meaningful problems.
Since evecy component problem has a unique optimal path value, it is
possible to write an optimal value function
V*
=
V*( i )
( i = A , B, . . . , Z)
which says that we can determine an optimal path value for evecy possible
initial point. From this, we can also construct an optimal policy function,
which will tell us how best to proceed from any specific initial point i, in
order to attain V*(i) by the proper selection of a sequence of arcs leading
from point i to the terminal point Z.
The purpose of the optimal value function and the optimal policy
function is easy to grasp, but one may still wonder why we should go to the
trouble of embedding the problem, thereby multiplying the task of solution.
The answer is that the embedding process is what leads to the development
of a systematic iterative procedure for solving the original problem.
Returning to Fig. 1 .6, imagine that our immediate problem is merely
that of determining the optimal values for stage 5, associated with the three
initial points I, J, and K. The answer is easily seen to be
( 1 .10)
V* ( I ) = 3
V*(J)
=
1
V* ( K )
=2
Having found the optimal values for I, J, and K, the task of finding the
least-cost values V*(G) and V*(H) becomes easier. Moving back to stage 4
and utilizing the previously obtained optimal-value information in (1.10), we
can determine V *(G) as well as the optimal path GZ (from G to Z) as
PART 1: INTRODUCTION
22
follows:
( 1 . 11)
V* ( G ) = min{value of arc GI + V* ( I ) , value of arc GJ
= min{ 2
+
3, 8 ·+ 1 } = 5
+
V*( J)}
[The optimal path GZ is GIZ.]
The fact that the optimal path from G to Z should go through I is
indicated by the arrow pointing away from G; the numeral on the arrow
shows the optimal path value V*(G). By the same token, we find
( 1 . 12)
V* ( H ) = min{ value of arc HJ + V* ( J ) , value of arc HK + V* ( K) }
= min{4 + 1 , 6 + 2 ) = 5
[The optimal path HZ is HJZ.]
Note again the arrow pointing from H toward J and the numeral on it. The
set of all such arrows constitutes the optimal policy function, and the set of
all the numerals on the arrows constitutes the optimal value function. With
the knowledge of V*(G) and V*(H), we can then move back one more stage
to calculate V*(D), V*(E), and V*(F )-and the optimal paths DZ, EZ,
and FZ-in a similar manner. And, with two more such steps, we will be
back to stage 1, where we can determine V*(A) and the optimal path AZ,
that is, solve the original given problem.
The essence of the iterative solution procedure is captured in Bellman's
principle of optimality, which states, roughly, that if you chop off the first
arc from an optimal sequence of arcs, the remaining abridged sequence
must still be optimal in its own right-as an optimal path from its own
initial point to the terminal point. If EHJZ is the optimal path from E to
Z, for example, then HJZ must be the optimal path from H to Z.
Conversely, if HJZ is already known to be the optimal path from H to Z,
then a longer optimal path that passes through H must use the sequence
HJZ at the tail end. This reasoning is behind the calculations in ( 1 . 1 1) and
(1.12). But note that in order to apply the principle of optimality and the
iterative procedure to delineate the optimal path from A to Z, we must find
the optimal value associated with every possible point in Fig. 1.6. This
explains why we must embed the original problem.
Even though the essence of dynamic programming is sufficiently clari­
fied by the discrete example in Fig. 1.6, the full version of dynamic program­
ming includes the continuous-time case. Unfortunately, the solution of
continuous-time problems of dynamic programming involves the more ad­
vanced mathematical topic of partial differential equations. Besides, partial
differential equations often do not yield analytical solutions. Because of this,
we shall not venture further into dynamic programming in this book. The
rest of the book will be focused on the methods of the calculus of variations
and optimal control theory, both of which only require ordinary differential
equations for their solution.
CHAPTER 1: THE NATURE OF DYNAMIC OPTIMIZATION
23
EXERCISE 1.4
1
From Fig. 1.6, find V*(D), V*( E), and V*( F). Determine the optimal
paths DZ, EZ, and FZ.
2
On the basis of the preceding problem, find V * ( B ) and V*(C). Determine
the optimal paths BZ and CZ.
3 Verify the statement in Sec. 1 . 1 that the minimum cost of production for
the example in Fig. 1.6 (same as Fig. 1.1) is $14, achieved on the path
ACEHJZ.
4
Suppose that the arc values in Fig. 1.6 are profit (rather than cost) figures.
For every point i in the set {A, B, . . . , Z}, find
(a)
(b)
the optimal (maximum-profit) value V* ( i ), and
the optimal path from i to Z.
. 2
PART
THE CALCULUS
OF VARIATIONS
- ·.
- ,.
.
.
'
·: ·
CHAPTER
2
THE
FUNDAMENTAL
PROBLEM OF
THE CALCULUS
OF VARIATIONS
We shall begin the study of the calculus of variations with the fundamental
problem:
( 2. 1)
Maximize or minimize
V [y )
subject to
y(O) = A
and
y( T )
=
=
j0TF[ t, y ( t) , y' ( t ) ) dt
Z
( A given)
( T, Z given)
The maximization and minimization problems differ from each other in the
second-order conditions, but they share the same first-order condition.
The task of variational calculus is to select from a set of admissible y
paths (or trajectories) the one that yields an extreme value of V[y ]. Since
the calculus of variations is based on the classical methods of calculus,
requiring the use of first and second derivatives, we shall restrict the set of
admissible paths to those continuous curves with continuous derivatives. A
smooth y path that yields an extremum (maximum or minimum) of V[y] is
called an extremal. We shall also assume that the integrand function F is
twice differentiable.
27
PART 2: THE CALCULUS.OF VARIATIONS
28
y
FIGURE 2.1
In locating an extremum of V[ y ], one may be thinking of either an
absolute (global) extremum or a relative (local) extremum. Since the calcu­
lus of variations is based on classical calculus methods, it can directly deal
only with relative extrema. That is, an extremal yields an extreme value of
V only in comparison with the immediately " neighboring" y paths.
2.1 THE EULER EQUATION
The basic first-order necessary condition in the calculus of variations is the
Euler equation. Although it was formulated as early as 1744, it remains the
most important result in this branch of mathematics. In view of its impor­
tance and the ingenuity of its approach, it is worthwhile to explain its
rationale in some detail.
With reference to Fig. 2.1, let the solid path y*(t) be a known
extremal. We seek to find some property of the extremal that is absent in
the (nonextremal) neighboring paths. Such a property would constitute a
necessary condition for an extremal. To do this, we need for comparison
purposes a family of neighboring paths which, by specification in (2.1), must
pass through the given endpoints (0, A) and (T, Z). A simple way of
generating such neighboring paths is by using a perturbing curve, chosen
arbitrarily except for the restrictions that it be smooth and pass through the
points 0 and T on the horizontal axis in Fig. 2.1, so that
( 2 .2)
p( O)
=
p(T)
=
0
We have chosen for illustration one with relatively small p values and small
slopes throughout. By adding Ep(t) to y*(t), where E is a small number, and
by varying the magnitude of E, we can perturb the y*(t) path, displacing it
to various neighboring positions, thereby generating the desired neighbor­
ing paths. The latter paths can be denoted generally as
( 2.3)
y(t) = y* (t) + Ep(t)
[implying y'(t) = y*' ( t ) + Ep ' (t)]
f
,F
CHAPTER 2: THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
29
with the property that, as e -> 0, y(t) -> y*(t). To avoid clutter, only one of
these neighboring paths has been drawn in Fig. 2.1.
The fact that both y*(t) and p(t) are given curves means that each
value of e will determine one particular neighboring y path, and hence one
particular value of V[y ]. Consequently, instead of considering V as a
functional of the y path, we can now consider it as a function of the
variable e-V(e). This change in viewpoint enables us to apply the familiar
methods of classical calculus to the function V = V(e). Since, by assump­
tion, the curve y*(t)-which is associated with e 0-yields an extreme
value of V, we must have
=
-
dV
de
(2.4)
\
.�o
= 0
This constitutes a defining property o f the extremal. It follows that
dV1 de = 0 is a necessary condition for the extremal.
As written, however, condition (2.4) is not operational because it
involves the use of the arbitrary variable E as well as the arbitrary perturb­
ing function p(t). What the Euler equation accomplishes is to express this
necessary condition in a convenient operational form. To transform (2.4)
into an operational form, however, requires a knowledge of how to take the
derivatives of a definite integral.
Differentiating a Definite Integral
Consider the definition integral
(2.5)
I( x )
=
tF( t, x ) dt
a
where F(t, x ) is assumed to have a continuous derivative Fx(t, x) in the
time interval [a, b]. Since any change in x will affect the value of the F
function and hence the definite integral, we may view the integral as a
function of x-I(x). The effect of a change in x on the integral is given by
the derivative formula:
( 2 .6)
dl
d =
X
b
J Fx(t, x ) dt
a
[ Leibniz 's rule]
In words, to differentiate a definite integral with respect to a variable x
which is neither the variable of integration (t) nor a limit of integration
(a or b), one can simply differentiate through the integral sign with respect
to x.
The intuition behind Leibniz's rule can be seen from Fig. 2.2a, where
the solid curve represents F(t, x), and the dotted curve represents the
displaced position of F(t, x) after the change in x. The vertical distance
between the two curves (if the-change is infinitesimal) measures the partial
.. .
30
PART 2: THE CALCULUS OF VARIATIONS
F
F
0
0
b
a
(a)
1
a
(b)
F(t, x)
b
F
F(t,x)
0
b
a
(c)
FIGURE 2.2
derivative F/t, x) at each value of t. It follows that the effect of the change
in x on the entire integral, dljdx, corresponds to the area between the two
curves, or, equivalently, the definite integral of Fx(t, x) over the interval
[a, b]. This explains the meaning of (2.6).
The value of the definite integral in (2.5) can also be affected by a
change in a limit of integration. Defining the integral alternatively to be
J( b, a)
( 2 .7)
=
fF(t, x) dt
a
we have the following pair of derivative formulas:
(2 .8)
iJJ
(2 .9)
iJJ
F ( t, x ) i t-b = F( b, x )
ab =
aa
= - F (t, x ) l t -a = - F( a , x )
In words, the derivative of a definite integral with respect to its upper limit
31
CHAPTER 2 : THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
o f integration b is equal to the integrand evaluated a t t =
derivative with respect to its lower limit of integration a is the
the integrand evaluated at
t = a.
b; and the
negative of
In Fig. 2.2b, an increase in b is reflected in a rightward displacement
of the right-hand boundary of the area under the curve. When the displace­
ment is infinitesimal, the effect on the definite integral is measured by the
value of the F function at the right-hand boundary-F(b, x). This provides
the intuition for (2.8). For an increase in the lower limit, on the other hand,
the resulting displacement, as illustrated in Fig.
2.2c,
ment of the left-hand boundary, which
the area under the curve.
This is why there is a negative sign
reduces
in (2.9).
is a rightward move­
The preceding derivative formulas can also be used in combination. If,
for instance, the definite integral takes the form
K( X )
( 2 . 10)
where
x
=
fb(x)F( t, X ) dt
a
F, but also affects the
(2.6) and (2.8), to get the
not only enters into the integrand function
upper limit of integration, then we can apply both
total derivative
( 2.11)
dK
b(x)
- = f Fx ( t, x ) dt + F [ b(x) , x ] b'( x )
dX
a
The first term on the right, an integral, follows from (2.6); the second ter-m,
representing the chain
EXAMPLE 1
dK db( x )
·
db( x ) �
The derivative of
rule,
EXAMPLE 2
J;e-x dt
(2.8).
with respect to
x
is, by Leibniz's
Similarly,
.!!_
dx
EXAMPLE 3
is based on
}2f3x1 dt = J2[3 !_x1 dt = £23tx1-1 dt
dx
To differentiate
J:x3t2 dt
with respect to
x
which appears in
the upper limit of integration, we need the chain rule as well as
result is
d
dx
i2x3t2 dt = [-d i2x
l d(2x)
3t2 dt -- = 3(2x } 2
0
d(2x}
0
dx
•
(2.8).
2 = 24x 2
The
PART 2, THE CALCULUS OF VARIA'nONS
32
Development of the Euler Equation
For ease of understanding, the Euler equation will be developed in four
steps.
Step i Let us first express V in terms of E, and take its derivative.
Substituting (2.3) into the objective functional in (2.1), we have
[
]
. V( e ) = (F t, y* ( t) + ep(t) , y*' (t) + ep' (t) dt
----- .____....
0
y(t)
y'(t)
obtain the derivative dVjd e , Leibniz's rule tells u s t o differentiate
( 2.12 )
To
through the integral sign:
( 2 .13)
[ by ( 2 .3 ) ]
Breaking the last integral in (2.13) into two separate integrals, and setting
dV1de = 0, we get a more specific form of the necessary condition for an
extremal as follows:
( 2 . 1 4)
While this form of necessary condition is already free of the arbitrary
variable e , the arbitrary perturbing curve p(t ) is still present along with its
derivative p ' (t). To make the necessary condition fully operational, we must
also eliminate p(t) and p '(t ) .
Step ii To that end, we first integrate the second integral in (2.14) by
parts, by using the formula:
( 2.15 )
Let
f.t-b du = vu ,t-b f.t-b dv
t-a V
t-a - t-a U
v = Fy'
and
[ u = u(t), v = v(t)}
u = p(t). Then we have
dv
dFy'
dv = dt
=
dt
dt
dt
and
du
=
du
dt = p' ( t ) dt
dt
Substitution of these expressions into (2.15)-with
a =
0 and
b = T-
CHAPTER 2:
TQE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
- 33
gives us
( 2 . 16)
since the first term to the right of the first equals sign must vanish under
assumption (2.2). Applying (2. 16) to (2.14) and combining the two integrals
therein, we obtain another version of the necessary condition for the
extremal:
( 2 . 17)
Step iii Although p'(t) is no longer present in (2. 17), the arbitrary p(t)
still remains. However, precisely because p(t) enters in an arbitrary way,
we may conclude that condition (2. 1 7) can be satisfied only if the bracketed
expression [ FY - dF .jdt] is made to vanish for every value of t on the
Y
extremal; otherwise, the integral may not be equal to zero for some admissi­
ble perturbing curve p(t). Consequently, it is a necessary condition for an
extremal that
( 2 . 18)
Fy -
d
dt
- F., = 0
for all
J
t
E
[0, T ]
[Euler equation]
Note that the Euler equation is completely free of arbitrary expressions, and
thus be applied as soon as one is given a differentiable F(t, y, y')
function.
The Euler equation is sometimes also presented in the form
can
( 2 . 18' )
which is the result of integrating (2. 18) with respect to
t.
Step iv The nature of the Euler equation (2. 18) can be made clearer when
we expand the derivative dFy•/dt into a more explicit form. Because F is a
function with three arguments (t, y, y'), the partial derivative Fy' should
also be a function of the same three arguments. -The-total derivative dFY.jdt
-therefore consists of three terms:
dFy'
dt
--
aFy' aFy' dy aFy' dy'
+
- + -at
ay dt
ay' dt
= F1y + Fyy•Y' ( t ) + Fy•y•Y" ( t)
=
-
-
34
PART
2: THE CALCULUS OF VARIATIONS
Substituting this into (2. 18), multiplying through by - 1, and rearranging,
we arrive at a more explicit version of the Euler equation:
Fy•y•Y"(t) + Fyy'Y' ( t) + Fey' - Fy = 0
( 2 . 19)
for all
[ Euler equation ]
t E [0, T ]
This expanded version reveals that the Euler equation is in general a
second-order nonlinear differential equation. Its general solution will thus
contain two arbitrary constants. Since our problem in (2.1) comes with two
boundary conditions (one initial and one terminal), we should normally
possess sufficient information to definitize the two arbitrary constants and
obtain the definite solution.
EXAMPLE
4
Find the extremal of the functional
V(y] =
with boundary conditions
have the derivatives
fo\ 12ty + y'2 ) dt
y(O) = 0
and
y(2) = 8.
FY. = 2y'
By
(2.19),
Since
F = 12ty + y'2,
we
and
the Euler equation is
2y" (t) - 12t = 0
which, upon integration, yields
or
y"(t) = 6t
y'(t) = 3t 2 + c1,
and
[ general solution]
To definitize the arbitrary constants c1 and c2 , we first set t = 0 in the
general solution to get y(O) = c2 ; from the initial condition, it follows that
c2 = 0. Next, setting t = 2 in the general solution, we get y(2) = 8 + 2c1;
from the terminal condition, it follows that c1 = 0. The extremal is thus the
cubic function
y*(t) = t3
EXAMPLE 5
[ definite solution ]
Find the extremal of the functional
V[y] =
with boundary conditions
3t + (y' )ll2 • Thus,
' - 1 /2
Fy' = l(
2 y )
f [ st + (y') 1 12] dt
1
y(1) = 3
and
y(5) = 7.
'
- /2
Fy'y' = - . ( y' ) 3
Here we have
and
F=
35
CHAPTER 2 : THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
The Euler equation (2.19) now reduces to
The only way to satisfy this equation is to have a constant y', in order that
y" = 0. Thus, we write y'(t) = c1, which integrates to the solution
[general solution 1
To definitize the arbitrary constants c1 and c 2 , we first set t = 1 to find
y(1) = c1 + c 2 = 3 (by the initial condition), and then set t = 5 to find
y(5) = 5c1 + c2 7 (by the terminal condition). These two equations give us
c1 = 1 and c2 = 2. Therefore, the extremal takes the form of the linear
=
function
y* ( t )
EXAMPLE 6
=t+2
[definite solution1
Find the extremal of the functional
= t(t + y 2 + 3y') dt
with boundary conditions y(O) 0 and y(5) = 3. Since F
V[y1
have
=
0
=
t
+
y2
+
3y', we
and
By (2.18), we may write the Euler equation as 2y
y* ( t )
=0
= 0, with solution
Note, however, that although this solution is consistent with the initial
condition y(O) = 0, it violates the terminal condition y(5) 3. Thus, we
must conclude that there exists no extremal among the set of continuous
curves that we consider to be admissible.
This last example is of interest because it serves to illustrate that
certain variational problems with given endpoints may not have a solution.
More specifically, it calls attention to one of two peculiar results that can
arise when the integrand function F is linear in y'. One result, as illus­
trated in Example 6, is that no solution exists. The other possibility, shown
in Example 7, is that the Euler equation is an identity, and since it is
automatically satisfied, any admissible path is optimal.
=
EXAMPLE 7
Find the extremal of the functional
PART 2o THE CALCULUS OF VARIATIONS,
36
with boundary conditions y(O) = a and
y(t) = {3. With F = y', we have
and
d
dt
- Fy, =
O
It follows that the Euler equation (2. 18) is always satisfied. In this example,
it is in fact clear from straight integration that
V ( y ] = [y(t ) ] � = y(T) - y(O) = {3 - a
The value of V depends only on the given terminal and initial states,
regardless of the path adjoining the two given endpoints.
The reason behind these peculiarities is that when F is linear in y', Fy'
is a constant, and Fy'y' = 0, so the first term in the Euler equation (2.19)
vanishes. The Euler equation then loses its status as a second-order differ­
ential equation, and will not provide two arbitrary constants in its general
solution to enable us to adapt the time path to the given boundary condi­
tions. Consequently, unless the solution path happens to pass through the
fixed endpoints by coincidence, it cannot qualify as an extremal. The only
circumstance under which a solution can be guaranteed for such a problem
(with F linear in y' and with fixed endpoints) is when FY = 0 as well,
which, together with the fact that Fy' = constant (implying dFy'jdt = 0),
would turn the Euler equation (2.18) into an identity, as in Example 7.
EXERCISE 2.1
1
In discussing the differentiation of definite integrals, no mention was made
of the derivative with respect to the variable t. Is that a justifiable
omission?
Find the derivatives of the following definite integrals with respect to
2
I = J.bx4 dt
3
I = J.be -"'' dt
4 I = Jo2xe' dt
5
I = j02"'tex dt
6
V[y 1
Find the extremals, if any, of the following functionals:
7
8
=
g(ty
+
2y'2) dt,
with
y(O)
=
1 and
V[y 1 = gtyy' dt, with y(O) = 0 and y(l) =
V[y 1 = j;(2ye ' + y2
+
y(l) = 2
1
y'2) dt, with y(O) = 2 and y(2)
9 V[y1 = J;(y2 + t2y') dt, with y(O) = 0 and y(2) = 2
=
2e2 + e-2
x:
CHAPTER 2, THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
2.2
37
SOME SPECIAL CASES
We have written the objective functional in the general form f;{F(t, y, y') dt,
in which the integrand function F has three arguments: t, y, and y'. For
some problems, the integrand function may not contain all three argu­
ments. For such special cases, we can derive special versions of the Euler
equation which may often (though not always) prove easier to solve.
Special case 1: F
y, implying that F:t
with the solution
F( t, y' )
=
In this special case, the F function is free of
= 0. Hence, the Euler equation reduces to dF:t.jdt = 0,
Fy' = constant
It may be noted that Example 5 of the preceding section falls under
this special case, although at that time we just used the regular Euler
equation for its solution. It is easy to verify that the application of (2.20)
will indeed lead to the same result. Here is another example of this special
case.
(2.20)
EXAMPLE 1 Find the extremal of the functional
with boundary conditions y(O)
=
y(1)
F = ty' + y'2
(2.20) gives us t + 2y (t )
'
=
and
= constant, or
y'( t)
=
1. Since
Ft'
:
=
t + 2y'
- 4t + c1
Upon direct integration, we obtain
y* ( t )
=
- tt2 + c1t + c2
[general solution]
=
With the help of the boundary conditions y(O) = y(1) 1, it is easily verified
that c1 t and c
1. The extremal is therefore the quadratic path
2
- i't2 + tt + 1
y*(t)
[definite solution]
=
=
=
Special case II: F = F(y, y' )
Since F is free of t in this case, we have
�:t' = 0, so the Euler equation (2.19) simplifies to
F:t' t'y"(t) + F:t:t•Y'(t) - F:t
:
=
0
The solution to this equation is by no means obvious, but it turns out that if
we multiply through by y', the left-hand-side expression in the resulting
38
PART z, THE CALCULUS OF VARIATIONS
new equation will be exactly the derivative
d(y'Fy' - F)jdt, for
d
d
d F(y, y')
)
(y'Fy
F)
=
(y'Fy
.
•
dt
dt
dt
= Fy•Y'� + y'(FyyY' + Fh.y") - (Fyy' + Fy·Y" )
y'(Fy•y·Y" + Fyy•Y' - Fy)
Consequently, the Euler equation can be written as d(y'Fy' - F)jdt = 0,
with the solution y'Fy' - F = constant, or, what amounts to be the same
=
thing,
F - y'Fy' = constant
( 2 .21)
This result-the simplified Euler equation already integrated once:-is
a
first-order differential equation, which may under some circumstances be
easier to handle than the original Euler equation (2. 19). Moreover, in
analytical (as against computational) applications, (2.21) may yield results
that would not be discernible from (2.19), as will be illustrated in Sec. 2.4.
EXAMPLE 2
Find the extremal of the functional
with boundary conditions
y(O) = 1 and y(
and
direct substitution in
7T
/2) = 0. Since
Fy' - 2y'
=
(2.21) gives us
y ' 2 + y 2 = constant
This last constant is nonnegative because the left-hand-side terms are all
squares; thus we can justifiably denote it by
2
a ,
where
a
is a nonnegative
real number.
2
The reader will recognize that the equation
+
= a can be
plotted as a circle, as in Fig. 2.3, with radius a and with center at the point
of origin. Since
is plotted against
in the diagram, this circle constitutes
the phase line for the differential equation. Moreover, the circular loop
y'2 y2
y'
y
shape of this phase line suggests that it gives rise to a cyclical time path, 1
1
Phase diagrams are explained in Alpha C. Chiang, Fundamental Methods of Mathematical
Economics, 3d ed., McGraw-Hill, New York, 1984, Sec. 14.6. The phase line here is similar to
phase line C in Fig. 14.3 in that section; the time path it implies is shown in Fig. 14.4c.
CHAPTER 2: THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
39
y'(t)
FIGURE 2.3
with the y values bounded in the closed interval [ - a, a], as in a cosine
function with amplitude a and period 27T. Such a cosine function can be
represented in general by the equation
y(t) = a cos( bt + c)
where, aside from the amplitude parameter a , we also have two other
parameters b and c which relate to the period and the phase of the
function, respectively. In our case, the period should be 27T; but since the
cosine function shows a period of 27T jb (obtained by setting the bt term to
27T ), we infer that b = 1. But the values of a and c are still unknown, and
they must be determined from the given boundary conditions.
At t = 0, we have
y(O) = a cos c = 1
[by the initial condition]
When t = 7T/2, we have
Y( i) = a cos ( i + c ) = 0
[by the terminal condition]
To satisfy this last equation, we must have either a = 0 or cos( 7T/2 + c) = 0.
But a cannot be zero, because otherwise a cos c cannot possibly be equal to
1. Hence, cos( 7T /2 + c) must vanish, implying two possibilities: either c 0,
or c 7T. With c = 0, the equation a cos c = 1 becomes a cos 0 = 1 or
a(1) = 1, yielding a 1. With c = 7T, however, the equation a cos c = 1
becomes a( - 1) = 1, giving us a = - 1, which is inadmissible because we
have restricted a to be a nonnegative number. Hence, we conclude that the
boundary conditions require a = 1 and c = 0, and that the time path of y
which qualifies as the extremal is
=
=
=
y * ( t ) = cos t
The same solution can, as we would expect, also be obtained in a
straightforward manner from the original Euler equation (2.18) or (2.19).
PART 2, THE CALCULUS OF VARIATIONS
40
After normalizing and rearranging, the Euler equation can be expressed
the following homogeneous second-order linear differential equation:
as
y" + y = O
Since this equation has complex characteristic roots r1 , r
2
solution takes the form of2
y* ( t )
= a
=
± i, the general
cos t + f3 sin t
The boundary conditions fix the arbitrary constants a and f3 at the values
of 1 and 0, respectively. Thus, we end up with the same definite solution:
y * ( t) = cos t
For this example, the original Euler equation (2.18) or (2.19) turns out
to be easier to use than the special one in (2.21). We illustrate the latter, not
only to present it as an alternative, but also to illustrate some other
techniques (such as the circular phase diagram in Fig. 2.3). The reader will
have to choose the appropriate version of the Euler equation to apply to any
particular problem.
EXAMPLE 3 Among the curves that pass through two fixed points A and
Z, which curve will generate the smallest surface of revolution when rotated
about the horizontal axis? This is a problem in physics, but it may be of
interest because it is one of the earliest problems in the development of the
calculus of variations. To construct the objective functional, it may be
helpful to consider the curve AZ in Fig. 2.4 as a candidate for the desired
extremal. When rotated about the t axis in the stipulated manner, each
point on curve AZ traces out a circle parallel to the xy plane, with its center
on the t axis, and with radius R equal to the value of y at that point. Since
the circumference of such a circle is 27TR (in our present case, 27Ty ) all we
have to do to calculate the surface of revolution is to sum (integrate) the
circumference over the entire length of the curve AZ.
An expression for the length of the curve AZ can be obtained with the
help of Fig. 2.5. Let us imagine that M and N represent two points that are
located very close to each other on curve AZ. Because of the extreme
proximity of M and N, arc MN can be approximated by a straight line,
with its length equal to the differential ds. In order to express ds in terms
of the variables y and t, we resort to Pythagoras' theorem to write (ds)2
(dy )2 + (dt)2• It follows that (ds)2j(dt)2 = (dy )2 j(dt)2 + 1. Taking the
,
=
2
For an explanation of complex roots, see Alpha C. Chiang, Fundamental Metlwds of Mathe­
matical Economics, 3d ed., McGraw-Hill, New York, 1984, Sec. 15.3. Here we have h = 0, and
h
v = 1; thus, e ' = 1 and vt = t.
CHAP'l'Bil 2: THE FUNDAMENTAL PROBLEM OF CALCULUS OF VAlUATIONS
y
41
z
FIGURE 2.4
�- root o'ft both sides, we get
ds
dt
-
=
.vhich yields the desired expression for
ds in terms of y and t as follows:
[arc length]
fhe entire length of curve
AZ
must then be the integral of
ds, namely
f:(l + y'2 ) 1 12 dt. To sum the circumference 2'1Ty over the length of curve
AZ
will therefore result in the functional
V [y ]
=
2'1Tfzy(l
a
+
0
1/2
.....
y
-
+ y'2)
-
t
.
PIGURE lU
dt
42
PART 2, THE
CALCULUS OF VARIATIONS
The reader should note that, while it is legitimate to " factor out" the "2'1T"
(constant) part of the expression 2'1TR, the
(variable) part must remain
within the integral sign.
"y"
For purposes of minimization, we may actually disregard the constant
to _be the expression for the
function, with
2'1T and take
y(1
+ y'2)112
F_y = yy ( 1 +y,2)-l/2
F
'
Since
F is free of t, we may apply (2.21) to get
This equation can be simplified by takin� the following steps: (1) multiply
through by (1 +
(2) cancel out
and
on the left-hand side,
in terms of y and c, and (4) take the
(3) square both sides and solve for
square root. The result is
y' 2)112 ,
y'2 yy'
-yy'2
y'(= dydt ) "!:.c .. Jy2 - c2
cdy
...jy2 c2 dt
=
or
--=
=
= =
_
y
In this last equation, we note that the variables
and
have been
separated, so that each side can be integrated by itself. The right-hand side
t
t + k,
poses no problem, the integral of
being in the simple form
where k
is an arbitrary constant. But the left-hand side is more complicated. In fact,
to attempt to integrate it "in the raw" would take too much effort; hence,
one should consult prepared tables of integration formulas to find the
result:3
dt
f cdy
�
vr
= c ln
- c�
t+k
Equating this result with
(y + � )
c
+ c1
(and subsuming the constant
c1
under the
3See, for example, CRC Standard Mathematical Tables, 28th ed., ed. by William H. Beyer, CRC
Press, Boca Raton, FL, 1987, Formula 157. The c in the denominator of the log expression on
the right may be omitted without affecting the validity of the formula, because its presence or
absence merely makes a difference amounting to the constant - c In c, which can be absorbed
into the arbitrary constant of integration c1, at any rate. However, by inserting the c in the
denominator, our result will come out in a more symmetrical form.
43
CHAPTER 2: THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
constant k), we have
ln
( y + Jy2 - c2 )
c
e <t+ k )/c = y + VY 2 - c2
or
=
t+k
c
--
[ by definition of natural log ]
c
Multiplying both sides by c, subtracting y from both sides, squaring both
sides and canceling out the y2 term, and solving for y in terms of t, we
finally find the desired extremal to be
y* ( t )
c
= - [ e<t + k )jc
2
+
e
- ( t +k)/ c
]
[ general solution ]
where the two constants c and k are to be definitized by using the boundary
conditions.
This extremal is a modified form of the so-called catenary curve,
( 2.23)
[catenary ]
whose distinguishing characteristic is that it involves the average of two
exponential terms, in which the exponent of one term is the exact negative
of the exponent of the other. Since the positive-exponent term gives rise to a
curve that increases at an increasing rate, whereas the negative-exponent
term produces an ever-decreasing curve, the average of the two has a
general shape that portrays the way a flexible rope will hang on two fixed
pegs. (In fact, the name catenary comes from the Latin word catena ,
meaning "chain.") This general shape is illustrated by curve AZ in Fig. 2.4.
Even though we have found our extremal in a curve of the catenary
family, it is not certain whether the resulting surface of revolution (known
as a catenoid) has been maximized or minimized. However, geometrical and
intuitive considerations should make it clear that the surface is indeed
minimized. With reference to Fig. 2.4, if we replace the AZ curve already
drawn by, say, a new AZ curve with the opposite curvature, then a larger
surface of revolution can be generated. Hence, the catenoid cannot possibly
be a maximum.
Special case III: F = F(y') When the F function depends on y' alone,
many of the derivatives in (2.19) will disappear, including Fyy'' Fty • and FY .
In fact, only the first term remains, so the Euler equation becomes
'
( 2.24)
=
To satisfy this equation, we must either have y"(t) 0 or Fy'y' 0. If
= c1 and y(t) c1t + c2, indicating that the
y"(t) = 0, then obviously y'(t)
=
=
44
PART
2: THE CALCULUS OF VARIATIONS
general solution is a two-parameter family of straight lines. If, on the other
hand, Fy'y' 0, then since Fy'y' is, like F itself, a function of y' alone, the
solution of Fy'y' = 0 should appear as specific values of y'. Suppose there are
one or more real solutions y' ki, then we can deduce that y = kit + c,
which again represents a family of straight lines. Consequently, given an
integrand function that depends on y' alone, we can always take its ex­
tremal to be a straight line.
=
=
·
EXAMPLE 4
Find the extremal of the functional
( 2 .25)
with boundary conditions y(O) = A and y(T) = Z. The astute reader will
note that this functional has been encountered in a different guise in
Example 3. Recalling the discussion of arc length leading to (2.22), we know
that (2.25) measures the total length of a curve passing through two given
points. The problem of finding the extremal of this functional is therefore
that of finding the curve with the shortest distance between those two
points.
The integrand function, F (1 + y' 2) 112, is dependent on y' alone. We
can therefore conclude immediately that the extremal is a straight line. But
if it is desired to examine this particular example explicitly by the Euler
equation, we can use (2.18). With FY 0, the Euler equation is simply
dFY.jdt = 0, and its solution is Fy' constant. In view of the fact that
Fy' = y'/(1 + y'2 )112, we can write (after squaring).
=
=
· ,_ :M\llqp)yjng both aides by
.qan -� y' iJ1 �rms of
.
.
•·
.
.
.:
·.
,
=
and factoring out y', we
(1 + y' 2) rearranging,
2
as follows: y'
c 2j(l - c 2). Equivalently,
,
c
y' =
=
(
1
c
-
c z) 1/2
= constant
Inasmuch as y' (t), the slope of y(t), is a constant, the desired extremal y*(t)
must be a straight line.
Strictly speaking, we have only found an "extremal" which may either
maximize or minimize the given functional. However, it is intuitively obvi­
ous that the distance between the two given points is indeed minimized
rather than maximized by the straight-line extremal, because there is no
such thing as " the longest distance" between two given points.
:>:,.
CHAPTER
2' THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
45
Special case IV: F = F( t, y ) In this special case, the argument y ' i s
missing from the F function. Since w e now have Fy' = 0, the Euler equa­
tion reduces simply to F
0, or, more explicitly,
Y
=
Fy( t , y )
The fact that the derivative
=
0
y' does not appear in this equation means that
the E uler equation is not a differential equation. The problem is degenerate.
Since there are no arbitrary constants in its solution to be definitized in
accordance with the given boundary conditions, the extremal may not
satisfy the boundary conditions except by sheer coincidence.
This situation is very much akin to the case where the F function is
linear in the argument
y' (Sec.
2.1,
Example
6).
The reason is that the
function F(t, y) can be considered as a special case of F(t, y, y') with y'
entering via a single additive term, Oy', with a zero coefficient. Thus, F(t, y)
is, i n a special sense, "linear" i n
y'.
EXERCISE 2.2
Find the extremals of the following functionals:
1
V[y]
2
V[y]
3 V[y]
4
5
6
V[y]
V[y]
V[y]
=
g(t2
=
f;7y'3 dt, with y(O)
=
fcJ<y + yy' + y' + h'2) dt, with y(O) = 2 and y(l)
=
fabr3y'2 dt
=
=
+
y'2) dt, with y(O) = 0 and y(l) = 2
= 9 and y(2)
=
11
=
5
(Find the general solution only.)
JcJ<y2 + 4yy' + 4y'2) dt, with y(O) = 2e 1 12
f0" 12(y 2 - y'2) dt, with y(O)
= 0 and
and
y( 1T/2) =
y(l) = 1 + e
1
2.3 TWO GENERALIZATIONS OF THE
EULER EQUATION
The previous discussion of the Euler equation is based on an integral
functional with the integrand F(t, y, y'). Simple generalizations can be
made, however, to the case of several state variables and to the case where
higher-order derivatives appear as arguments in the
F function.
The Case of Several State Variables
With
n
(2.26)
> 1
state variables in a given problem, the functional becomes
.;' ,'_: �··· -.
46
PART 2: THE CALCULUS OF VARIATIONS
and there will be a pair of initial condition and terminal condition for each
of the n state variables.
Any extremal y/ ( ), (j = 1, . . . , n ), must by definition yield the ex­
treme path value relative to all the neighboring paths. One type of neighbor­
ing path arises from varying _only one of the functions y ) at a time, say,
y 1( , while all the other
functions are kept fixed. Then the functional
depends only on the variation in y1( ) as if we are dealing with the case of a
single state variable. Consequently, the Euler equation (2.18) must still hold
as a necessary condition, provided we change the y symbol to y1 to reflect
the new problem. Moreover, this procedure can be similarly used to vary the
other yi functions, one at a time, to generate other Euler equations. Thus,
for the case of n state variables, the single Euler equation (2.18) should be
replaced by a set of n simultaneous Euler equations:
t
t)
( 2 .27)
}t
y}t)
FYj - dtd FY/ = 0
t
for all
t E [0, T ]
(j
=
.
1 , 2, . . , n)
[Euler equations]
These n equations will, along with the boundary conditions, enable us to
determine the solutions y 1 ( ) ,
Although (2.27) is a straightforward generalization of the Euler equa­
tion (2. 18)-replacing the symbol y by yi-the same procedure cannot be
used to generalize (2.19). To see why not, assume for simplicity that there
are only two state variables, y and in our new problem. The
function
will then be a function with five arguments,
y, y',
and the partial
derivatives
and
will be, too. Therefore, the total derivative of
with respect to will include five terms:
y',
* t . . . , Yn*(t).
z,
F(t, z, z'),
F
Fy'
Fz'•
t
F/t, y, z, z')
d t
dt Fy.( , y, z ,y', z') = Fty' + Fyy•Y'( t ) + Fzy•z'(t) + Fy•yY"( t) + Fzyz" ( t)
with a similar five-term expression for dF jdt. The expanded version of
•.
simultaneous Euler equations corresponding
more complicated than (2. 19) itself:
(2 .28)
to
(2.19) thus looks much
FHy"(t) + Fz'y'z"( t ) + Fyy•Y'( t ) + Fzy'z'( t ) + Fty' - FY = 0
Fyz•Y"( t ) + Fz'z'z"( t ) + Fyz•Y'( t ) + Fzz'z'( t ) + F;z' - Fz = 0
for all t
E
EXAMPLE 1
Find the extremal of
From the integrand, we find that
Fy
=
1
Fy'
=
2y'
F11 =
2z'
[0, T ]
CHAPTER
2:
Thus, by
47
THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
(2.27), we have two
simultaneous Euler equations
2y"
z
11-2
"
0
0
=
=
�
or
"
y =
or
Z, - !2
The same result can also be obtained from (2.28).
In this particular case, it happens that each of the Euler equations
·
contains one variable exclusively. Upon integration, the first yields
+
and hence,
�t c1,
y* ( t) it2 + c1t + c2
z
z* ( t) tt2 + c3t + c4
(c1, , c4)
y(O), z(O), y(T),
y'
=
=
Analogously, the extremal of
is
=
The four arbitrary constants
can be definitized with the help of
. • .
four boundary conditions relating to
and
z(T).
The Case of Higher-Order Derivatives
another generalization, consider a functional that contains high-order
derivatives of
Generally, this can be written as
As
y(t).
V[y] = f TF(t,y,y',y", . . . ,y<n>) dt
( 2 .29)
0
Since many derivatives are present, the boundary conditions should in this
case prescribe the initial and terminal values not only of
but also of the
derivatives
total of
2n
y',y",
.. . . , up to and including the derivative
boundary conditions.
y,
y<n-I>,
making a
F
To tackle this case, we first note that an
function with a single state
and derivatives of y up to the n th order can be transformed into
variable
y
an equivalent form containing n state variables and their first-order deriva­
tives only. In other words, the functional in (2.29) can be transformed into
the form in (2.26). Consequently, the Euler equation (2.27) or (2.28) can
again be applied. Moreover, the said transformation can automatically take
care of the boundary conditions as well.
EXAMPLE 2
Transform the functional
y(O)
y(T)
y'(O)
y'(T)
A,
= Z,
= a, and
/3 ,
with boundary conditions
into the format o f (2.26). T o achieve this task, we only have t o introduce a
new variable
z y'
==
=
[ implying
z' y"]
==
=
..
.
PART
Then we cah rewrite the integrand as
2: THE CALCIJLUSOF VARIAT�
y
which now contains two state variables and z, with no derivatives higher
than the first order. Substituting the new F into the functional will turn
the latter into the format of
What about the boundary conditions? For the original state variable
= A and
= Z can be kept intact. The other two
the conditions
conditions for
can be directly rewritten as the conditions for the new
= a and
state variable z:
= {3. This complete the transformation.
(2.26).
y,
y(T)
y(O)
y'
z (O)
z(T)
Given the functional (2.29), it is also possible to deal with it directly
instead of transforming it into the format of (2.26). By a procedure similar
to that used in deriving the Euler equation, a necessary condition for an
extremal can be found for (2.29). The condition, known as the Eu ler­
Poisson equation, is
d2
dn
d
+ ( - l} n FY - - FY. + -Fy (2.30)
dt
dt2 "
dtn FYc., 0
=
· · ·
for all
[ Euler-Poisson equation )
t E [0, T)
2n.
This equation is in general a differential equation of order
Thus its
solution will involve
arbitrary constants, which can be definitized with
the help of the
boundary conditions.
2n
EXAMPLE 3
2n
Find an extremal of the functional in Example
have
FY =
F,.
2ty + y'
=
2. Since we
2y"
the Euler-Poisson equation is
dy + -d22y" = 0
2ty + y' - dt dt2
or
2ty + 2y<•> - 0
which is a fourth-order differential equation.
EXERCISE 2.3
1
2
Find the extremal of
y(l) = y'(l) = 1.
Find the extremal of
only).
V[y] = gu
V[y, z]
=
+ y"2) dt, with
y(O) = 0
J:<y' 2 + z'2 + y'z') dt
and
y'(O) =
(general solution
1·.
.
OHAPTER
2: THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
49
3 Find the extremal of V[y, z] = j0"'12(2yz + y'2 + z'2) dt, with y(O) = z(O) =
0 and y(rr/2) = z(rr/2) = 1.
4 Example 3 of this section shows that, for the problem stated in Example 2,
a necessary condition for the extremal is 2ty + 2y<4l = 0. Derive the same
result by applying the definition
that problem.
2.4
z
=
y' and the Euler equations
(2.27) to
DYNAMIC OPTIMIZATION
OF A MONOPOLIST
Let us turn now to economic applications of the Euler equation. As the first
example, we shall discuss the classic Evans model of a monopolistic firm,
one of the earliest applications of variational calculus to economics. 4
A Dynamic Profit Function
Consider a monopolistic firm that produces a single commodity with a
quadratic total cost function5
C = aQ 2 + [3Q + y
(2.31)
( a , {3 , y > O)
Since no inventory is considered, the output Q is always set equal to the
quantity demanded. Hence, we shall use a single symbol Q(t) to denote both
quantities. The quantity demanded is assumed to depend not only on price
P(t), but also on the rate of change of price P'(t):
( a , b > O ; h *- 0)
(2.32)
Q = a - bP( t) + hP'(t)
The firm's profit is thus
1r = PQ - C
= P ( a - bP + hP' ) - a( a - bP + hP') 2 - f3(a - bP + bP') - y
which is a function of P and P'. Multiplying out the above expression and
collecting terms, we have the dynamic profit function
(2.33) 1r(P, P') = -b(l + ab)P 2 + ( a + 2aab + f3b) P
- ah2P' 2 - h(2aa + f3 ) P' + h ( l + 2ab) PP'
[profit function]
- ( aa2 + {3a + y)
4G. C. Evans, " The Dynamics ofMonopoly," American Mathematical Monthly, February 1924,
pp.
77-83. Essentially
author,
the same material appears in Chapters 14 and 15 of a book by the same
Mathematical Introduction to Economics, McGraw-Hill, New York, 1930, pp. 143-153.
6This quadratic cost function plots as a U-shaped curve. But with
f3 positive, only the
right-hand segment of the U appears in the first quadrant, giving us an upward-sloping
total-cost curve for
Q
� 0.
50
PART
2:
THE CALCULUS OF VARIATIONS
The Problem
P
The objective of the firm is to find an optimal path of price
that
maximizes the total profit fi over a finite time period [0, T ]. This period is
assumed to be sufficiently short to justify the assumption of fixed demand
and cost functions, as well as- the omission of a discount factor. Besides, as
the first approach to the problem, both the initial price
and the terminal
price
are assumed to be given.
The objective of the monopolist is therefore to
P0
Pr
fi(P] = j0T1r(P, P') dt
P( O) = P0
(P0 given)
P(T ) = Pr (T, Pr given)
Maximize
subject to
(2.34)
and
The Optimal Price Path
(2.34)
Although
pertains to Special Case II, it turns out that for computa­
tion with specific functions it is simpler to use the original Euler equation
(2.19), where we should obviously substitute 7T for F, and
for y. From
the profit function
it is easily found that
P
(2.33),
7Tp = -2b(l + ab)P + (a + 2aab + {3b) + h(l + 2ab ) P'
= -2ah2P' - h (2aa + /3) + h(l + 2ab ) P
= h ( l + 2ab )
7TP'P' = - 2ah2
·
7TP'
and
7Tpp•
These expression turn (2. 19)-after normalizing-into the specific form
+ f3b
(2.35 ) P" - b(la+h2ab) p = - a + 2aab
2
2a h
[Euler equation]
This is a second-order linear differential equation with constant
ficients and a constant term, in the general format of
y
"
coef­
+ a iy' + a 2y = a a
Its general solution is known to be6
6
This type of differential equation is discussed in Alpha C. Chiang, Fundamental Methods of
Mathematical Econcmics, 3d ed., McGraw-Hill, New York, 1984, Sec. 15.1.
CHAPTER
2:
51
THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
where the characteristic .roots
r
1
and
r2 take the values
r1, r2 = t { -a1 ± Jai - 4 a 2 )
and the particular integral y is
Thus, for the Euler equation of the present model (where a1 =
(2 . 36)
where
_
p
and
=
b ( 1 + ab)
ah2
a + 2aab + f3b
____;__
2b( 1 + ab)
0),
we have
[ characteristic roots J
(particular integral]
_
_
_
Note that the two characteristic roots are real and distinct under our sign
specifications. Moreover, they are the exact negatives of each other. We may
therefore let r denote the common absolute value of the two roots, and
rewrite the solution as
(2.36')
(general solution ]
The two arbitrary constants A1 and A2 in
the boundary conditions P(O) = P0 and
(2.36' ) can be definitized via
P(T) = PT . When we set t = 0 and
t = T, successively, in (2.36'), we get two simultaneous equations in the two
variables A1 and A2:
Po = A l + A z + p
PT = Al e rT + A 2e- rT + p
with solution values
(2 .37)
A1
-
-
-P -p
- ) e rT
( Pr p
1 - e 2 rT
o :.. -� - _!__....:..,
...;.�
·
A2 -
P0 - P - (PT - P) e - rT
1
e -2 r T
_
The determination of these two constants completes the solution of the
problem, for the entire price path
P*(t)
is now specified in terms of the
known parameters T, P0, PT ,
{3, y,
and
Of these parameters, all
have specified signs except
But inasmuch as the parameter
enters into
the solution path (2.36') only through r, and only in the form of a squared
term
we can see that its algebraic sign will not affect our results,
h.
a,
h2 ,
although its numerical value will.
a, b,
h.
h
52
PART 2: THE CALCULUS OF VARIATIONS
At this moment, we are not equipped to discuss whether the solution
path indeed maximizes (rather than minimizes) profit. Assuming that it
indeed does, then a relevant question is: What can we say about the general
configuration of the price path (2.36')? Unfortunately, no simple answer can
be given. If Pr > P0, the monopolist's price may optimally rise steadily in
the interval [0, T ] from P0 ··to Pr or it may have to be lowered slightly
before rising to the level of Pr at the end of the period, depending on
parameter values. In the opposite case of P0 > Pr, the optimal price path is
characterized by a similar indeterminacy. Although this aspect of the prob­
lem can be pursued a little further (see Exercise 2.4, Probs. 3 and 4), specific
parameter values are needed before more definite ideas about the optimal
price path can be formed.
A More General View of the Problem
Evans' formulation specifies a quadratic cost function and a linear demand
function. In a more general study of the dynamic-monopolist problem by
Tintner, these functions are left unspecified. 7 Thus, the profit function is
.. simply written as rr(P, P'). In such a general formulation, it turns out that
:':\ formula (2.21) [for Special Case II] can be used to good advantage. It directly
yields a simple necessary condition
rr -
( 2 .38 )
arr
P'aP'
= c
that can he given a clear-cut economic interpretation.
To see this, first look into the economic meaning of the constant c. If
profit rr does not depend on the rate of change of price P'-that is, if we are
dealing with the static monopoly problem as a special case of the dynamic
model-then arr;aP' = 0, and (2.38) reduces to rr = c . So the constant c
represents the static monopoly profit. Let us therefore denote it by 7T8
(subscript s for static). Next, we note that if (2.38) is divided through by rr,
the second term on the left-hand side will involve
arr P'
aP' -.;;:- = E ,.p'
which represents the partial elasticity of rr with respect to P'. Indeed, after
performing the indicated division, the equation can he rearranged into the
7
Gerhard Tintner,
" Monopoly
over Time,"
Econometrica , 1937, pp. 160- 170. Tintner also
tried to generalize the Evans model to the case where rr depends on higher-order derivatives of
P, but the economic meaning of the result is difficult to interpret.
CHAPTER
2:
THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
optimization rule
€-rr p'
( 2 .38' )
7Ts
= 1 - 7T
This rule states that the monopolistic firm should always select the price in
such a way that the elasticity of profit with respect to the rate of change of
price be equal to 1 - 7T5j7T. The reader will note that this analytical result
would not have emerged so readily if we had not resorted to the special-case
formula (2.21).
It is of interest to compare this elasticity rule with a corresponding
elasticity rule for the static monopolist. Since in the latter case profit
depends only on P, the first-order condition for profit maximization is
simply d 7TjdP = 0. If we multiply through by Pj7T, the rule becomes
couched in terms of elasticity as follows: E.,P = 0. Thus, while the state
monopolist watches the elasticity of profit with respect to price and sets it
equal to zero, the dynamic monopolist must instead watch the elasticity of
profit with respect to the rate of change of price and set it equal to
1 - 7TJ7T.
The Matter of the Terminal Price
The foregoing discussion is based on the assumption that the terminal price
given. In reality, however, the firm is likely to have discretionary
control over P(T) even though the terminal time T has been preset. If so, it
will face the variable-terminal-point situation depicted in Fig. 1.5a, where
the boundary condition P( T) = Pr must be replaced by an appropriate
transversality condition. We shall develop such transversality conditions in
the next chapter.
P( T ) is
EXERCISE 2.4
1
If the monopolistic firm in the Evans model faces a static linear demand
(h = 0), what price will the firm charge for profit maximization? Call that
price P., and check that it has the correct algebraic sign. Then compare the
values of P. and P, and give the particular integral in (2.36) an economic
interpretation.
A1
A2
should indeed have the values shown in (2.37).
2 Verify that
and
3 Show that the extremal P*(t) will not involve a reversal in the direction of
price movement in the time interval [0, T ] unless there is a value 0 < t0 < T
4
A1er1• A 2e-rto
A1
such that
=
[i.e., satisfying the condition that the first two
terms on the right-hand side of (2.36') are equal at t = t0].
If the coefficients
and A 2 in (2.36' ) are both positive, the P*(t) curve
will take the shape of a catenary. Compare the location of the lowest point
54
PART 2: THE CALCULUS OF VARIATIONS
5
on the price curve in the following three cases: (a) A1 > A2, (b) A1 = A2,
and (c) A1 < A2• Which of these cases can possibly involve a price reversal
in the interval [0, T ]? What type of price movement characterizes the
remaining cases?
Find the Euler equation for the Evans problem by using formula (2.18).
6 If a discount rate p > t:J is used to convert the profit 7T(P, P' ) at any point
of time to its present value, then the integrand in (2.34) will take the
general form F = 7T(P, P')e - P1 •
·�
(a) In that case, is formula (2.21) still applicable?
F expression to derive the new Euler
equation.
(c) Apply formula (2.18) to derive the Euler equation, and express the
result as a rule regarding the rate of growth of 7Tp• ·
(b) Apply formula (2.19) to this
2.5 TRADING OFF INFLATION AND
UNEMPLOYMENT
The economic maladies of both inflation and unemployment inflict social
losses. When a Phillips tradeoff exists between the two, what would be the
best combination of inflation and unemployment over time? The answer to
this question may be sought through the calculus of variations. In this
section we present a simple formulation of such a problem adapted from a
paper by Taylor.8 In this formulation, the unemployment variable as such is
not included; instead, it is proxied by (Y1 - Y)-the shortfall of current
national income Y from its full-employment level Y1 .
The Social Loss Function
Let the economic ideal consist of the income level Y1 coupled with the
inflation rate 0. Any deviation, positive or negative, of actual income Y from
Y1 is considered undesirable, and so is any deviation of actual rate of
inflation p from zero. Then we can write the social loss function as follows:
( 2.39)
(a > 0)
Because the deviation expressions are squared, positive and negative devia­
tions are counted the same way (cf. Exercise 1.3, Prob. 2). However, Y
deviations and p deviations do enter the loss function with different weights,
in the ratio of 1 to a, reflecting the different degrees of distaste for the two
types of deviations.
8Dean Taylor, "Stopping Inflation in the Dornbusch Model: Optimal Monetary Policies with
Alternate Price-A(ijustment Equations," Journal ofMacroecorwmics, Spring 1989, pp. 199-216.
In our adapted version, we have altered the planning horizon from oo to a finite T.
lI
l
55
CHAPTER 2: THE FUNDAMENTAL PROBLEM OF CALCULUS OF VARIATIONS
The expectations-augmented Phillips tradeoff between
is captured in the equation
(Yr - Y) and p
p = -f3(Yr - Y) + 1r ( f3 > 0)
(2.40)
where 7T, unlike its usage in the preceding section, now means the expected
rate of inflation. The formation of inflation expectations is assumed to be
adaptive:
'
7T
(2.41)
( �;) =j(p
=
- 1T
)
(0 < j :5 1)
p
If the actual rate of inflation
exceeds the expected rate of inflation 1T
(proving 7T to be an underestimate), then 1r' >
and 1T will be revised
0,
upward; if, on the other hand, the actual rate p falls short of the expected
rate 1T (proving 1T to be an over estimate), then 1r' <
and 1T will be revised
downward.
The last two equations together imply that
which
can
be rearranged as
- 7T
(2.40), (2.42) yields
1T
(2.43)
'
p = -;- + 7T
J
(2.42)
(2.43) into (2.39), we are able to express the loss
And, by plugging
and
function entirely in terms of
(2 .44)
'
{3j
Yr - Y = -
(2 .42)
When substituted into
0,
A(1T, 1r')
= ( {3j
7T'
1T
and
1r':
)2 ( j )2
+ a
1T'
+
1T
[loss function]
The Problem
The problem for the government is then to find an optimal path of
[0, T).
1T
that
minimizes the total social loss over the time interval
The initial
(current) value of 1T is given at 1r0, and the terminal value of 1r, a policy
target, is assumed to be set at 0. To recognize the importance of the present
over the future, all social losses are discounted to their present values via a
positive discount rate p.
PART 2, THE CALCULUS OF VARIATIONS
In view of these considerations, the objective of the policymaker is to
.>
''�
A[ 1T )
Minimize
(2.45)
subject to
�
r:
and
=
·
. .--·-:� ·:- •
j TA ( 1r, 1r')e-pt dt
0
!
l
l
( 1r0 > 0 given)
1r(O) = 1r0
1r(T) 0
(T given)
=
.
I
d'
f
The Solution Path
On the basis of the loss function
yields the first derivatives
'
(2.44), the integrand function A(1T, 1T')e7',
. with second derivatives
and
(
,
F
-rrr.
=
)
2a
-e-pt
j
a -pt
1 + af32 , Ft1T' - - 2p
f3 2 · 2 1T + . 1T e
-
J
J
Consequently, formula (2.19) gives us (after simplification) the specific
necessary condition
(2 .46)
1r" - p1r' - il1r = 0 [�uler equation)
af32j (p + j ) > 0
where n
1 + af3
2
==
Since this differential equation is homogeneous, its particular integral
is zero, and its general solution is simply its complementary function:
(2.47)
[general solution)
The characteristic roots are real and distinct. Moreover, inasmuch
square root has a numerical value greater than
we know that
p,
(2.48)
as
the
and
'
L.
!
CHAPTER 2: THE
FUNDAMENTAL I'ROBLEM OF CALCULUS OF VARIATIONS
57
FIGURE 2.6
t
=
To definitize the arbitrary constants A 1 and A 2 , we successively set
t=
in (2.4 7), and use the boundary conditions to get the pair of
0 and
relations
T
A I + A2
Al
=
'1T o
e r,T + A 2 e rzT =
0
Solved simultaneously, these relations give us
(2.49)
Because of the signs of r1 and r2 in
(2.48),
we know that
(2.50)
From the sign information in
(2.50) and (2.48), we can deduce that the
2.6. With a
7r* (t) path should follow the general configuration in Fig.
t
negative A1 and positive r1, the A1er, component of the path emerges as
the mirror image (with reference to the horizontal axis) of an exponential
t
growth curve. On the other hand, the A 2 e rz component is, in view of the
positive A2 and negative r2, just a regular exponential decay curve. The '7T*
path, the sum of the two component curves, starts from the point
7T0)-where 1r0 = A1 + A2-and descends steadily toward the point
(0,
(T, 0), which is vertically equidistant from the two component curves.
fact that the
7r *
path shows a steady downward movement
can
The
also be
58
PART 2: THE CALCULUS OF VARIATIONS
verified from the derivative
?T*' ( t )
=
1
r1A1er,t + r2A2er•t < 0
Having found ?T*(t) and ?T*'(t), we can also derive some conclusions
regarding p and (Yr - Y). F?r these, we can simply use the relations in
(2.43) and (2.42), respectively.
EXERCISE
2.5
1
Verify the result in (2.46) by using Euler equation (2.18).
2
Let
(a)
(b)
S
the
/:lA<7T,
objective
7r')e-P1 dt.
functional
in
problem
(2.45) be changed to
Do you think the solution of the problem will be different?
Can you think of any advantage in including a coefficient
!
in the
integrand?
Let the terminal condition in problem
0
<
(a)
(b)
7Tr < 7ro.
(2.45) be changed to 1r(T) =
What would be the values of A1 and A2?
Can you ambiguously evaluate the signs of A1 and A2?
7Tr,
j
CHAPTER
3
TRANSVERSALITY
CONDITIONS
FOR
VARIABLE­
ENDPOINT
PROBLEMS
·
The Euler equation-the basic necessary condition in the calculus of varia­
tions-is normally a second-order differential equation containing two arbi­
trary constants. For problems with fixed initial and terminal points, the two
given boundary conditions provide sufficient information to definitize the
two arbitrary constants. But if the initial or terminal point is variable
(subject to discretionary choice), then a boundary condition will be missing.
Such will be the case, for instance, if the dynamic monopolist of the
preceding chapter faces no externally imposed price at time T, and can treat
the Pr choice as an integral part of the optimization problem. In that case,
the boundary condition P(T ) = Pr will no longer be available, and the void
must be filled by a transversality condition. In this chapter we shall develop
the transversality conditions appropriate to various types of variable termi­
nal points.
3.1 THE GENERAL TRANSVERSALITY
CONDITION
For expository convenience, we shall assume that only the terminal point is
variable. Once we learn to deal with that, the technique is easily extended to
the case of a variable initial point.
59
PART 2: THE CALCULUS OF VARIATIONS
60
The Variable-Terminal-Point Problem
Our new objective is to
Maximize or minimize
subject to
(3.1)
V{y] t0 F(t,y,y') dt
y(O) A
( A given)
y(T) Yr ( T, yr free)
=
=
and
=
This differs from the previous version of the problem in that the terminal
time
and terminal state Yr are now " free" in the sense that they have
become a part of the optimal choice process. It is to be understood that
although
is free, only positive values of are relevant to the problem.
To develop the necessary conditions for an extremal, we shall, as
before, employ a perturbing curve
and use the variable to generate
neighboring paths for comparison with the extremal. First, suppose that
is the known optimal terminal time. Then any value of in the immediate
neighborhood of
can be expressed as
T
T
T
p(t),
.
E
T
T*
T*
T T* + o lT
where E represents a small number, and llT represents an arbitrarily
chosen (and fixed) small change in T. Note that since T* is known and llT
is a prechosen magnitude, T can be considered as a function of E, T(E), with
( 3 .2)
=
derivative
dT llT
dE =
(3.3)
E
The same i s used i n conjunction with the perturbing curve
generate neighboring paths of the extremal
y*(t):
[ implying y'(t) y*'(t) + Ep'(t)]
p(t) to
y(t) y*(t) + Ep(t)
However, although the p (t) curve must still satisfy the condition p(O) = 0
(3 .4)
=
=
[see (2.2)] to force the neighboring paths to pass through the fixed · initial
point, the other condition-p(tY= 0-should now be dropped, because Yr is
free. By substituting (3.4) into the functional
we get a function
akin to (2.12), but since
is a function of by (3.2), the upper limit of
integration in the function will also vary with
V[y],
V
E:
V(E) foT(E)F[t, y*(t) + Ep(t) , y*'(t) + Ep'(t) ) dt
(3 .5)
y'(t)
y(t)
Our problem is to optimize this V function with respect to E.
T
=
E
V(E)
CHAPTER 3: TRANSVERSAL!TY CONDJ'I'!ONS FOR VARIABLE-ENDPOINT PROBLEMS
61
Deriving the General Transversality
Condition
The first-order necessary condition for an extremum in V is simply dV1d c
= 0. This latter derivative is a total derivative taking into account the
direct effect of E on V, as well as the indirect effect of E on V through the
upper limit of integration T
_
Step i The definite integral (3.5) falls into the general form of (2.10). The
derivative dVjdE is therefore, by (2. 11),
(3.6)
dV
dc
=
fT<•> aF
dt
0
ac
+
dT
F [ T , y( T ) , y'( T ) ] dE
The integral on the right closely resembles the one encountered in (2.13)
during our earlier development of the Euler equation. In fact, much of the
derivation process leading to (2.17) still applies here, with one exception.
The expression [FY.p(t)J& in (2. 16) does not vanish in the present problem,
but takes the value [ FY.p(t)]1_ T [ Fy,]t - Tp(T), since we have assumed that
p(O) = 0 but p(T) i= 0. With this amendment to the earlier result in (2. 17),
we have
=
By (3.3), we can also write
Second term in ( 3.6)
=
[ F ] 1 = 7- .6. T
Substituting these into (3.6), and setting dV/dE = 0, we obtain the new
condition
Ofthe three terms on the left-hand side of (3. 7), each contains its own
independent arbitrary element: p(t) (the entire perturbing curve) in the
first term, p(T) (the terminal value on the perturbing curve) in the second,
and tiT (the arbitrarily chosen change in T ) in the third. Thus we cannot
presume any offsetting or cancellation of terms. Consequently, in order to
satisfy the condition (3. 7), each of the three terms must individually be set
equal to zero.
It is a familiar story that when the first term in (3.7) is set equal to
zero, the Euler equation emerges, as in (2.18). This establishes the fact that
the Euler equation remains valid as a necessary condition in the variable­
endpoint problem. The other two terms, which relate only to the terminal
time T, are where we should look for the transversality condition.
62
y
PART 2: THE CALCULUS OF VARIATIONS
I
����-�- � - - - - - � - .
.
I
: ,;
· '· ' ' '
·'
?:'' 1
,.,, .,..
� -- -- -- --
_.. ..-
....
.... .... (7')
p
Z"
· :.
l} y'(T) 4r :
--·
,.... .....
A J..-��
p(t)
I
..
·
•
·,
.
.;· ·:,
1
I
I
I
I Z
I
,.... .J
,..
I
I ll T I
�
�
�
----------------�--�.----0 �-
T
FIGURE 3.1
Step ii To this end, we first get rid of the arbitrary quantity p(T) by
transforming it into terms of AT and Ayr-the changes in T and Yr• the
two principal variables in the variable-terminal-point problem. This can be
done with the help of Fig. 3.1. The · AZ' curve represents a neighboring path
obtained by perturbing the AZ path by Ep(t), with E set equal to one for
convenience. Note that while the two curves share the same initial point
because p(O) = 0, they have different heights at t = T because p(T ) 'f. 0 by
construction of the perturbing curve. 1 The magnitude of p(T), shown by
the vertical distance ZZ', measures the direct change in Yr resulting from
the perturbation. But inasmuch as T can also be altered by the amount
E AT ( = AT since E = 1), the AZ' curve should, if A T > 0, be extended out
2
to Z". As a result, Yr is further pushed up by the vertical distance between
Z' and Z". For a small AT, this second change in Yr can be approximated
by y'( T ) AT. Hence, the total change in Yr from point Z to point Z" is
Ayr = p ( T ) + y'( T ) A T
Rearranging this relation allows us t o express p ( T ) i n terms o f Ayr and
AT:3
( 3 .8)
p ( T ) = Ayr - y'( T ) AT
1
Technically, the point T on the horizontal axis in Fig. 3.1 should be labeled T*. We are
omitting the • for simplicity. This is justifiable because the result that this discussion is
leading to-(3.9)-is a transversality condition, which, as a standard practice, is stated in
terms of T (without the * ) anyway.
2
Aithough we are initially interested only in the solid portion of the AZ path, the equation for
that path should enable us also to plot the broken portion as an extension of AZ. The
perturbing curve is then applied to the extended version of AZ.
3
The result in (3.8) is valid even if £ is not set equal to one as in Fig. 3.1.
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
63
Step iii Using (3.8) to eliminate p(T) in (3. 7), and dropping the first term
in (3.7), we finally arrive at the desired general transversality condition
( 3.9)
This condition, unlike the Euler equation, is relevant only to one point of
time, T. Its role is to take the place of the missing terminal condition in the
present problem. Depending on the exact specification of the terminal line
or curve, however, the general condition (3.9) can be written in various
specialized forms.
3.2
SPECIALIZED TRANSVERSALITY
CONDITIONS
In this section we consider five types of variable terminal points: vertical
terminal line, horizontal terminal line, terminal curve, truncated vertical
terminal line, and truncated horizontal terminal line.
Vertical Terminal Line
(Fixed-Time-Horizon Problem)
The vertical-terminal-line case, as illustrated in Fig. 1.5a, involves a fixed
T. Thus tlT 0, and the first term in (3.9) drops out. But since tlyr is
arbitrary and can take either sign, the only way to make the second term in
(3.9) vanish for sure is to have Fy' 0 at t T. This gives rise to the
transversality condition
=
=
=
( 3 1 0)
.
which is sometimes referred to as the natural boundary condition.
Horizontal Terminal Line .
(Fixed-Endpoint Problem)
For the horizontal-terminal-line case, as illustrated in Fig. 1.5b, the situa­
tion is exactly reversed; we now have tlyr = 0 but tlT is arbitrary. So the
second term in (3.9) automatically drops out, but the first does not. Since
tlT is arbitrary, the only way to make the first term vanish for sure is to
have the bracketed expression equal to zero. Thus the transversality condi­
tion is
( 3 . 1 1)
It might be useful
to
give an economic interpretation to
(3.10) and
(3. 11). To fix ideas, let us interpret F(t, y, y') as a profit function, where y
represents capital stock, and
y' represents net investment. Net investment
64
PART 2: THE CALCULUS OF VARIATIONS
entails taking resources away from the current profit-making business
operation, so as to build up capital which will enhance future profit. Hence,
there exists a tradeoff between current profit and future profit. At any time
t, with a given capital stock y, a specific investment decision-say, a
decision to select the investment rate y'0-will result in the current profit
F(t, y, y'0). As to the effect o£ the investment decision on future profits, it
enters through the intermediary of capital. The rate of capital accumulation
is y'0; if we can convert that into a value measure, then we can add it to the
current profit F(t, y, y'0) to see the overall (current as well as future) profit
implication of the investment decision. The imputed (or shadow) value to
the firm of a unit of capital is measured by the derivative Fy' · This means
that if we decide to leave (not use up) a unit of capital at the terminal time,
it will entail a negative value equal to -Fy'· Thus, at t = T, the value
measure of y'0 is -y'0FY'o· Accordingly, the overall profit implication of the
decision to choose the investment rate y'0 is F(t, y, y'0) - y0F:Io · The gen­
eral expression for this is F - y'Fy'• as in (3. 11).
Now we can interpret the transversality condition (3.11) to mean that,
in a problem with a free terminal time, the firm should select a T such that
a decision to invest and accumulate capital will, at t = T, no longer yield
any overall (current and future) profit. In other word!>, all the profit
opportunities should have been fully taken advantage of by the optimally
chosen terminal time. In addition, (3. 10)-which can equivalently be writ­
ten as [ - Fy,]t- T = 0-instructs the firm to avoid any sacrifice of profit that
will be incurred by leaving a positive terminal capital. In other words, in a
free-terminal-state problem, in order to maximize profit in the interval
[0, T ] but not beyond, the firm should, at time T, us up all the capital it ever
accumulated.
Terminal Curve
With a terminal curve Yr = l/J(T ), as illustrated in Fig. 1.5c, neither 11Yr
nor 11T is assigned a zero value, so neither term in (3.9) drops out.
However, for a small arbitrary 11T, the terminal curve implies that 11yr
¢' 11T. So it is possible to eliminate 11yr in (3.9) and combine the two terms
into the form
=
Since 11T is arbitrary, we can deduce the transversality condition
( 3 . 12)
[ F + ( cf>' - y' ) Fy· L - r = 0
Unlike the last two cases, which involve a single unknown in the
terminal point (either Yr or T ), the terminal-curve case requires us to
determine both Yr and T. Thus two relations are needed for this purpose.
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
65
The transversality condition (3.12) only provides one; the other is supplied
by the equation Yr = cf>(T).
Truncated Vertical Terminal Line
The usual case of vertical terminal line, with !lT - 0, specializes (3.9) to
( 3.13 )
When the line is truncated-restricted by the terminal condition Yr � Ymin
where Ymin is a minimum permissible level of y-the optimal solution can
have two possible types of outcome: y/ > Yrnin or y/ = Ymin · If y/ > Yrnin •
the terminal restriction is automatically satisfied; that is, it is nonbinding.
Thus, the transversality condition is in that event the same as (3. 10):
for Yr* > Y min
( 3 .14)
The basic reason for this is that, under this outcome, there are admissible
neighboring paths with terminal values both above and below y/, so that
llyr = Yr - Yr* can take either sign. Therefore, the only way to satisfy
(3.13) is to have Fy' = 0 at t = T.
The other outcome, y/ = Ymin • on the other hand, only admits neigh­
boring paths with terminal values Yr � y/. This means that !lyT is no
longer completely arbitrary (positive or negative), but is restricted to be
nonnegative. Assuming the perturbing curve to have terminal value p(T ) >
0, as in Fig. 3.1, !lyT � 0 would mean that t: � 0 (t: = 1 in Fig. 3.1). The
nonnegativity of t: , in turn, means that the transyersality condition
(3.13)-which has its roots in the first-order condition dVIdE = 0- must
, be changed to an inequality as in the Kuhn-Tucker conditions.4 For a
maximization problem, the � type of inequality is called for, and (3.13)
should become
( 3. 15)
And since
( 3. 16 )
!lyr � 0, (3.15) implies condition
for Yr* = Ymin
. .. ·- ,
· .:-
Combining (3.14) and (3.16), we may write the following summary
statement of the transversality condition for a maximization problem:
Y/ � Ymin
( 3 . 17)
( y/ - Ym;n) [ Fy· L -r = 0
[for maximization of V}
4
For an explanation of the Kuhn-Tucker conditions, see Alpha C . Chiang, Fundamental
Methods of Mathematical Economics, 3d ed., McGraw-Hill, New York, 1984, Sec. 2 1 .2.
' �
·.;: .
66
PART 2: THE CALCULUS OF VARIATIONS
If the problem is instead to minimize V, then the inequality sign in
(3.15) must be reversed, and the transversality condition becomes
[ F,. L - T ;:>: 0
( 3 . 17')
(y/ - Ymin ) [ Fy' L - T = 0
[ for minimization
of
V]
The tripartite condition (3.17) o r (3.17') may seem complicated, but its
application is not. In practical problem solving, we can try the [
=
= 0
condition in (3. 14) first, and check the resulting
value. If
Fy•l, T
y/ � Ymin•
then the terminal restriction is satisfied, and the problem solved. If Yr* <
Ymin• on the other hand, then the optimal Yr lies below the permissible
range, and [ Fy•l,_r fails to reach zero on the truncated terminal line. So, in
y/
order to satisfy the complementary-slackness condition in (3.17) or (3. 17'),
we just set
Ymin • treating the problem as one with a fixed terminal
point
Yr* =
(T, Ymin ).
Truncated Horizontal Terminal Line
The horizontal terminal line may be truncated by the restriction
:s;
where
max
represents a maximum permissible time for completing a
task-a deadline. The analysis of such a situation is very similar to the
truncated vertical terminal line just discussed. By analogous reasoning, we
can derive the following transversality condition for a maximization prob­
lem:
T
T
(3.18 )
[ F - y'Fy•L � r ;:>: 0
( T * - Tmax ) [ F - y'Fy•L - r = 0
If the problem is t o
minimize V,
T*
:S;
Tmw<>
Tmax
[ for maximization of V ]
the first inequality in (3.18) must be
changed, and the transversality condition is
( 3 . 18')
[ F - y'F,.j , _ T :s; 0
( T * - Tmax ) [ F - y'Fy·L -r 0
T*
:S;
Tmax
[ for minimization of V ]
=
EXAMPLE 1 Find the curve with the shortest distance between the point
and the line
= 2
t. Referring to Fig. 3.2, we see that this is a
problem with a terminal curve
= 2 - T, but otherwise it is similar to
the shortest-distance problem in Example 4 of Sec. 2.2. Here, the problem
is to
(0, 1)
y
-
y(T)
T
Minimize
V[y ] = 1 (1
subject to
y(O)
and
y( T )
0
= 1
=
/
+ y'2 ) 1 2 dt
2-T
CHAPTER
3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
67
y
'. {� r .
y = 2 - ': :
0 ._------4---�-2
FIGURE 3.2
It has previously been shown that with the given integrand, the Euler
equation yields a straight-line extremal, say,
y* = at + b
with two arbitrary constants a and b. From the initial condition y(O) = 1,
we can readily determine that b = 1. To determine the other constant, a,
we resort to the transversality condition (3.12). Since we have
¢'
=
-1
the transversality condition can be written as
(at t = T )
1/
2
2
Multiplying through by ( 1 + y ' )
and simplifying, we can reduce this
equation to the form y' = 1 (at t = T ). But the extremal actually has a
constant slope, y*' =
at all values of t. Thus, we must have a = 1. And
a,
the extremal is therefore
y* ( t) = t + 1
As shown in Fig. 3.2, the extremal is a straight line that meets the
terminal line at the point <t 1-V. Moreover, we note that the slopes of the
terminal line and the extremal are, respectively, - 1 and + 1. Thus the two
lines are orthogonal (perpendicular) to each other. What the transversality
condition does in this case is to require the extremal to be orthogonal to the
terminal line.
EXAMPLE 2
Find the extremal of the functional
PART 2: THE CALCULUS OF VARIATIONS
68
y
(6, 10)
10 +------=��
1
--- -------+-0 '---·
6
FIGURE 3.3
with boundary conditions y(O) = 1, Yr = 10, and T free. Here we have a
horizontal terminal line as depicted in Fig. 3.3.
The general solution of the Euler equation for this problem has
previously been found to be a quadratic path (Sec. 2.2, Example 1):
y*(t)
=
- {t2 +
c1t
+
c2
Since y(O) = 1, we have c2 L But the other constant c1 must be defini­
tized with the help of the transversality condition (3. 1 1). Since F = ty' + y'2,
so that Fy' = t + 2y', that condition becomes
=
ty' + y'2 - y' ( t + 2y')
=
(at t = T )
0
which reduces to -y'2 0 or y' = 0 at t = T . That is, the extremal is
required to have a zero slope at the terminal time. To meet this condition,
we differentiate the general solution to get y*'(t), then set t = T, and let
y*'(T) = 0. The result is the equation - T/2 + c1 = 0; it follows that
c1 = Tj2. However, we still cannot determine the specific value of c1
without knowing the value of T.
To find T, we make use of the information that Yr = 10 (horizontal
terminal line). Substituting c 1 = T/2 and c2 = 1 into the general solution,
setting t T, and letting the resulting expression take the stipulated value
10, we obtain the equation
=
=
- iT 2 + iT2 + 1
=
10
The solution values of T are therefore ± 6. Rejecting the negative value, we
end up with T* = 6. Then it follows that c1 = 3, and the extremal is
y* ( t)
=
- it2 + 3t
+
1
. �..:
This time path is shown in Fig. 3.3. AB required by the transversality
condition, the extremal indeed attains a zero slope at the terminal point
(6, 10). Unlike in Example 1, where the transversality condition translates
I
'
69
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
into an orthogonality requirement, the transversality condition
in the pre­
sent example dictates that the extremal share a common slope with the
given horizontal terminal line at
t == T.
Application to the Dynamic Monopoly
Model
Let us now consider the Evans model of a dynamic monopolist as a problem
with a fixed terminal time T, but a variable terminal state
Pr � Pmin· With
the vertical terminal line thus truncated, the appropriate transversality
condition is
For simplicity, we shall in this discussion assign specific
(3.17).
numerical values to the parameters, for otherwise the solution expressions
will become too unwieldy.
Let the cost function and demand function be
[ i .e . , a = -fo , {3 = 0, 'Y = 1000]
[i . e . , a = 160, b = 8, h = 100]
c = -foQ 2 + 1000
Q = 160 - BP + lOOP'
·. ..... :
Then the profit function becomes
7T =
PQ - C = 416P - 14.4P 2
+
260PP' - 1000P' 2 - 3200P' - 3500
which implies that
(3.19)
7Tp•
=
260P - 2000P' - 3200
This is the derivative needed in the transversality condition.
:: . ···
Since the postulated parameter values yield the characterilltie
,
.
and particular integral [see
(2.36)]:
p
=
'mow ·
14�
the general solution of the Euler equation is
(3.20)
If we further assume the boundary conditions to be
Pr = 15�
then, according to
(2.37),
the constants
T=2
and
A1
and
A2
should, in the fixed­
terminal-point problem, have the values (after rounding):
A1 = 6.933
A 2 = - 9 .933
(3.20)
P*(2) = 15�
The reader can verify that substitution of these two constants into
'
does (apart from rounding errors) produce the terminal price
as required.
70
PART
2: THE CALCULUS OF VARIATIONS
Now adopt a variable tenninal state PT � 10. Using the transversality
condition (3.17), we first set the TTp· expression in (3.19) equal to zero at
t = T = 2. Upon normalizing, this yields the condition
( 3.21 )
P'( T ) - 0.13P( T ) = - 1.6
The P(T) term here refers to the general solution P*(t) in (3.20)
evaluated at t = T = 2. And the P'(T) is the derivative of (3.20) evaluated
at the same point of time. That is,
P(T ) = Ale o.24 + A 2 e -o.24 + 14�
P' ( T ) = 0.12 A 1e0·24 - 0.12 A2 e-0 24
'l'hus,
(3.21)
can
be rewritten more specifically as
- 0.01A1e0·24 - 0.25A 2e -0·24 = 0.2778
To solve for A1 and A 2 , we need to couple this condition with. the oonmtioll
that governs the initial point,
A 1 + A2 = - 3
obtained from (3.20) by setting t = 0 and equating the result to P
The solution values of A1 and A 2 turn out to be (after rounding): 0
and
AI = 4.716
A 2 = - 7.716
=
11�.
giving us the definite solution
(3.22)
P* ( t) = 4 . 716e0 ·121 - 7 .716e-0· 121 + 1 4�
It remains to check whether this solution satisfies the terminal speci­
fication PT � 10. Setting t = T = 2 in (3.22), we find that P*(2) = 14.37.
Since this does meet the stipulated restriction, the problem is solved.5
The Inflation-Unemployment Tradeoff
Again
The inflation-unemployment problem in Sec. 2.5 has a fixed terminal point
that requires the expected rate of inflation, TT, to attain a value of zero at
the given terminal time T: TT(T) = 0. It may be interesting to ask: What
will happen if the terminal condition comes as a vertical terminal line
instead?
5While Evan's original treatment does include a discussion of the variable terminal price, it
neither makes use of the transversality condition nor gives a completely determined price path
for that case. For his alternative method of treatment, see his
Economics, McGraw-Hill, New York, 1930, pp. 150-153.
Mathematical Introduction to
71
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
From (2.4 7), the general solution of the Euler equation is
(3.23)
=
.,_
The initial condition, still assumed to be
t 0 in the general solution) that
(3.24)
'TT(O) = '!To
T,
[Fy,]t� T =
F . = 2 ( 1 f3+z.ia2{321T' + �'!T)e-pt
=0
j
>
..
0, requires (by �ting
'
' H
:
With a vertical terminal line at the given
however, we now must use the
transversality condition
0. In the tradeoff model, this takes the
form
(3.25)
Tf
·
(at
t = T)
which can be satisfied i f and only if the expression in the parentheses is
zero. Thus, the transversality condition can be written (after simplification)
as
+
= 0
(3.25')
'TT( T)
1T1(T)
1T'(T) a'TT( T)
{3 2j
where a = a
1 + a/32
The
and
expressions are, of course, to be obtained from the
general solution 'TT* (t) and its derivative, both evaluated at t = T. Using
(3.23), we can thus give condition (3.25') the specific form
(3.25")
When (3.24) and (3.25") are solved simultaneously, we finally find the
definite values of the constants
and
A1
(3.26)
A2 :
1To__:___;
(rl ; + a)er•T
A2 - (rl + a)er•
T - (rz + a)e r•T
----=
-
,----=-.
These can be used to turn the general solution into the definite solution.
Now that the terminal state is endogenously determined, what value of
'!T* (T) emerges from the solution? To see this, we substitute (3.26) into the
general solution (3.23), and evaluate the result at t
The answer turns
out to be
= T.
'TT* (T)
=
0
In view of the given loss function, this requirement-driving the rate of
expected inflation down to zero at the terminal time-is not surprising. And
PART 2: THE CALCULUS OF VARIATIONS
72
since the earlier version of the problem in Sec. 2.5 already has the target
value of 7T' set at 7T'r = 0, the switch to the vertical-terminal-line formula­
tion does not modify the result.
EXERCISE 3.2
1
For the functional V[y I = J;{(t2 + y'2) dt, the general solution to the
Euler equation is y*(t) c1t + c2 (see Exercise 2.2, Prob. 1).
(a) Find the extremal if the initial condition is y(O) = 4 and the terminal
condition is T = 2, YT free.
(b) Sketch a diagram showing the initial point, the terminal point, and the
extremal.
How will the answer to the preceding problem change, if the terminal
condition is altered to: T = 2, YT :2: 3?
Let the terminal condition in Prob. 1 be changed to: YT 5, T free.
(a) Find the new extremal. What is the optimal terminal time T*?
(b) Sketch a diagram showing the initial point, the terminal line, and the
extremal.
(a) For the functional V[y ) = f T(y'2 jt3) dt, the general solution to the
Euler equation is y*(t ) = c 1 t2 + c2 (see Exercise 2.2, Prob. 4). Find the
extremal(s) if the initial condition is y(O) = 0, and the terminal
condition is YT = 3 - T .
(b) Sketch a diagram to show the terminal curve and the extremal(s).
The discussion leading to condition (3.16) for a truncated vertical terminal
line is based on the assumption that p(T) > 0. Show that if the perturbing
curve is such that p(T) < 0 instead, the same condition (3.16) will emerge.
For the truncated-horizontal-terminal-line problem, use the same line of
reasoning employed for the truncated vertical terminal line to derive
transversality condition (3.18).
=
2
3
4
5
6
=
3.3 THREE GENERALIZATIONS
What we have learned about the variable terminal point can be generalized
in three directions.
A Variable Initial Point
If the initial point is variable, then the boundary condition y(O) = A no
longer holds, and an initial transversality condition is needed to fill the void.
Since the transversality conditions developed in the preceding section can be
applied,
to the case of variable initial points, there is no
need for further discussion. If a problem has
the initial and terminal
mutatis mutandis,
both
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
73
points variable, then two transversality conditions have to be used to
definitize the two arbitrary constants that arise from the Euler equation.
The Case of Several State Variables
When several state variables appear in the objective functional, so that the
integrand is
F(t , y l , · , yn , Y1 1 •
·
·
• · '
, y'n )
the general (terminal) transversality condition (3.9) must be modified into
. . +y:FYJL-r � T
+ [ FY {L= r �Y tr + . . · [ FY.: L - r �Yn r = 0
[ F - (y;Fy{ +
(3.27)
·
+
n=
It should be clear that (3.9) constitutes a special case of (3.27) with
1.
Given a fixed terminal time, the first term in (3.27) drops out because
� T = 0. Similarly, if any variable Yi has a fixed terminal value, then
�YiT 0 and the jth term in (3 . 27) drops out. For the terms that remain,
however, we may expect all the � expressions to represent independently
determined arbitrary quantities. Thus, there cannot be any presumption
that the terms in (3.27) can cancel out one another. Consequently, each
term that remains will give rise to a separate transversality condition.
The following examples illustrate the application of (3.27) when
2,
with the state variables denoted by y and z. The general transversality
condition for
2 is
=
n=
n=
EXAMPLE 1
Assume that T is fixed, but Yr and Zr are both free. Then
we have
0, but tlyr and tlzr are arbitrary. Eliminating the first term
in (3.27') and setting the other two terms individually equal to zero, we get
the transversality conditions
!J. T =
and
which the reader should compare with (3.10).
Suppose that the terminal values of y and · i Bre"� tib
·�
'
1 ·' ;
.. ..;
satisfy the restrictions
EXAMPLE 2
'
Yr
= c/J( T )
and
Zr
=
, .·. ·:
. ,.
'
··
·
·
1/J(T)
Then we have a pair of terminal curves. For a small
following to hold:
and
'
tJ. T,
we may expect the
PART 2,
74
Using tlulse to eliminate
t:.yr
t:. zr
and
[ F + ( c/>' - y') Fy' +
THE CALCULUS OF VARIATIONS
in (3.27'), we obtain
( rf/ - z' ) F.·L� r t:.T = o
Be� *T is arbitrary, the transversality condition emerges
[F +
( c/>' - y')Fy
+
as
( rf/ - z') F.·L-r = 0
which the reader should compare with (3.12).
With this transversality condition, the terminal curves Yr = cf>(T ) and
Zr = 1/!(T), and the initial conditions y (O) = Yo and z(O) = z0 , we now have
five equations to determine T* as well as the four arbitrary constants in the
two Euler equations.
The Case of Higher Derivatives
When the functional V[y] has the integrand F(t, y, y', . . . , y<n >), the (termi­
nal) transversality condition again requires modification. In view of the
rarity of high-order derivatives in economic applications, we shall state here
the general transversality condition for the case of F(t, y, y', y" ) only:
(3.28)
[ F - y'Fy' - y"Fy" + y' :t Fy· ] t:.T
+ [ Fy. - t Fy" ] t:.yr + [ Fy"J r-r .t:.Y'r = O
t-T
d
d
t-T
The new symbol appearing in this condition, t:.y'T• means the change
in the terminal slope of the y path when it is perturbed. In terms of Fig.
3.1, t:.y'r would mean the difference between the slope of the AZ" path at
Z" and the slope of the AZ path at Z. If the problem specifies that the
terminal slope must remain unchanged, then t:.y'T = 0, and the last term in
(3.28) drops out. If the terminal slope is free to vary, then the last term will
call for the condition Fy'' = 0 at t T.
=
EXERCISE 3.3
1
'
For the functional V[y I = f{<y + yy
y' + !y' 2 ) dt, the general solution
of the Euler equation is y* ( t ) = !t2 + c1t + c (see Exercise 2.2, Prob. 3).
2
If we have a vertical initial line at t = 0 and a vertical terminal line at
t = 1, write out the transversality conditions, and use them to definitize
the constants in the general solution.
+
2 Let the vertical initial line in the preceding problem be truncated with the
restriction y*(O) 2: 1, but keep the terminal line unchanged.
(a) Is the original solution still acceptable? Why?
(b) Find the new extremal.
75
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
3
In a problem with the functional f;{F(t, y, z, y', z') dt, suppose that y(O)
A, z(O) = B, Yr = C, zr = D, T free (A, B, C, D are constants).
=
(a) How many transversality condition(s) does the problem require? Why?
(b)
Write out the transversality condition(s)_
4 In the preceding problem, suppose that we have instead y(O) = A, z(O)
Yr
=
C, and Zr
=
t/J(T), T free ( A , B, C are constants).
=
B,
(a) How many transversality condition(s) does the problem require? Why?
(b) Write out the transversality condition(s).
3.4 THE OPTIMAL ADJUSTMENT OF
LABOR DEMAND
Consider a firm that has decided to raise its labor input from L0 to a yet
undetermined level Lr after encountering a wage reduction at t = 0. The
adjustment of labor input is assumed to entail a cost C that varies with
U(t), the rate of change of L. Thus the firm has to decide on the best speed
of adjustment toward Lr as well as the magnitude of Lr itself. This is
the essence of the labor adjustment problem discussed in a paper by
Hamermesh.6
The Problem
For simplicity, let the profit of the firm be expressed by the general function
as illustrated in Fig. 3.4. The labor input is taken to
be the sole determinant of profit because we have subsumed all aspects of
production and demand in the profit function. The cost of adjusting L is
assumed to be
rr(L), with rr"(L) < 0,
bL' 2 + k
( b > 0 , and k > 0 when L' * 0)
Thus the net profit at any time is rr(L) - C(L').
(3 .29)
C ( L' )
=
The problem of the firm is to maximize the total net profit n over time
during the process of changing the labor input. Inasmuch as it must choose
not only the optimal Lr, but also an optimal time T* for completing the
adjustment, we have both the terminal state and terminal time free. An­
other feature to note about the problem is that n should include not only
the net profit from t 0 to t = T* (a definite integral), but also the
capitalized value of the profit in the post-T* period, which is affected by the
choice of Lr and T, too. Since the profit rate at time T is rr(Lr). its
present value is rr(Lr)e-p1, where p is the given discount rate and T is to
be set equal to T*- So the capitalized value of that present value is,
=
6 Daniel S. Hamermesh, " Labor Demand and the Structure of Adjustment Costs," American
Economic Review, September 1989, pp. 674-689.
PART z, THE CALCULUS OF VARIATIONS
76
1C
TC(L)
0
..________
L
FIGURE 3.4
:=::�=:�::��:;:� �:::·.::::�:�:w: I.
(3.30)
0
L( O) L 0
L( T ) Lr
=
subject to
and
(L0 given)
(Lr > L 0 free, T free)
=
p '
.
If the last term in the functional were a constant, we could ignore it in
the optimization process, because the same solution path would emerge
either way. But since that term-call it z( T )-varies with the optimal
choice of
and T, we must explicitly take it into account. From our
earlier discussion of the problem of Mayer and the problem of Bolza, we
Lr
have learned that
z(T)
(3.31)
Thus, by defining
z( t)
=
{0 z'(t) dt + z(O)
1
= -rr(
L ) e -pt
p
so that
we can write the post-T* term in
(3.3Y )
_:rr ( Lr) e-pT
p
=
=
(3.30) as
fT[ -rr(L)
0
z'(t)
+
[see (1 .7)]
[ -rr( L)
]
+
; 1T' (L)L' ] e -pt
_:rr'(L ) L' e-pt dt + _:rr(L0)
p
p
Substitution of (3.3Y) into the functional in (3.30) yields, after combining
the two integrals, the new but equivalent function
77
CHAPTER 3: TRANSVERSALITY CONDITIONS FOR VARIABLE-ENDPOINT PROBLEMS
While this functional still contains an extra term outside of the integral,
that term is a constant. So it affects only the optimal n value, but not the
optimal
path, nO{ the optimal values of
and T.
L
Lr
The Solution
To find the Euler equation, we see from
[ -bL'2 - k + ;1T'(L)L}-pt
F=
Thus, from formula
( 3 .33)
(3.32) that
(2.18), we get the condition
1T'(L)
L" - pL' + --2b
=
0
[ Euler equation]
The transversality condition in this problem is twofold. Both
Lr
and
T being free, we must satisfy both transversality conditions (3.10) and
(3.11).7 The condition [ Fvlt = T = 0 means that
1T'(L) = 0
L' - --2pb
(3.34)
And the condition
[ F - L'FL']t-T
( 3.35)
=
or
(at
t = T)
0 means (after simplification) that
( at t
= T)
L
where we take the positive square root because
is supposed to increase
from L0 to
The two transversality conditions, plus the initial condition
L(O) = £0, can provide the information needed for definitizing the arbitrary
constants in the solution path, as well as for determining the optimal
and T. In using (3.34) and (3 . 35), i t i s understood that the
symbol refers
to the derivative of the general solution of the Euler equation, evaluated at
Lr.
t
=
L'
T.
Lr
'IT(L).
T o obtain a specific quantitative solution, i t i s necessary t o specify the
form of the profit function
Since our primary purpose of citing this
example is to illustrate a case with both the terminal state and the terminal
7Technically, condition (3.17) for a truncated vertical terminal line should be used in lieu of
(3. 10). However, since Lr sho ld be strictly greater than L0, the complementary-slackness
u
requirement would reduce (3.17) to (3. 10).
PART 2o THE CALCULUS OF VARIATIONS
78
time free, and to illustrate the
problem
of Bolza, we · shall not delve into
specific quantitative solution.
EXERCISE 3.4
1 (a) From the two
transversality conditions (3.34) and (3.35), deduce the
location of the optimal terminal state L/ with reference to Fig. 3.4.
(b) How would an increase in p affect the location of L/? How would an
increase in b or k affect L/?
(c) Interpret the economic meaning of your result in (b).
2 In the preceding problem, let the profit function be
1r ( L )
=
(a) Find the value of L/.
2 mL
-
(O < n < m )
nL2
(b) At what L value does .,. reach a peak?
(c) In light of (b), what can you say about the location of Lr* in relation
to the .,.(L) curve?
3 (a) From the transversality condition (3.35), deduce the location of the
optimal terminal time T * with reference to a graph of the solution
path L*(t).
(b) How would an increase in k affect the location of T * ? How would an
increase in b affect T*?
. .
-
; .__ .
CHAPTER
4
SECOND-ORDER
CONDITIONS
Our discussion has so far concentrated on the identification of the
extremal(s) of a problem, without attention to whether they maximize or
minimize the functional V[y ]. The time has come to look at the latter aspect
of the problem. This involves checking the second-order conditions. But we
shall also discuss a sufficient condition based on the concept of concavity
and convexity. A simple but useful test known as the Legendre necessary
conditions for maximum and minimum will also be introduced.
4.1
SECOND-ORDER CONDITIONS
By treating the functional V[y] as a function V( E ) and setting the first
derivative dVId t:. equal to zero, we have derived the Euler equation and
transversality conditions as first-order nece�sary conditions for an extremal.
To distinguish between maximization and minimization problems, we can
take the second derivative d 2 Vjdt:. 2, and use the following standard
second-order conditions in calculus:
Second-order necessary conditions:
[ for maximization of V ]
[ for minimization of V]
79
80
PART
2: THE CALCULUS OF VARIATIONS
Second-order sufficient conditions:
d2V
de2 < 0
d2V >
de2 0
[for maximization of V]
[for minimization of V]
-
The Second Derivative of V
To find d2Vjde2, we differentiate the dVjde expression in (2.13) with
respect to e, bearing in mind that (1) all the partial derivatives of F( t, y, y')
are, like F itself, functions of t, y, and y', and (2) y and y' are, in turn, both
functions of e, with derivatives
( 4 . 1)
dy
de = p(t)
and
dy
de p'(t)
- =
'
[by (2.3) ]
Thus, we have
( 4 2)
.
( )
d zV = :!_ dV
de2 de de
=
l[ :
T
:!..._
de }n{ [ FYp(t) + FY.p'(t)] dt
0
: ]
= T p(t) e FY + p'(t) e Fy' dt
In view of the fact that
[by Leibniz's rule]
[ by (4. 1 }]
and, similarly,
the second derivative (4 .2) emerges (after simplification)
(4.2')
lo
as
d2V
de2 = T [ FyyP2(t) + 2Fyy•P (t)p'(t) + Fy'y'p'2(t)] dt
.·,. ,·.�. ....., ,·;'
The Quadratic-Form Test
The second derivative in (4.2') is a definite integral with a quadratic form as
its integrand. Since t spans the interval [0, T ], we have, of course, not one,
but an infinite number of quadratic forms in the integral. Nevertheless, if it
, ,
:
. . ..
'
81
CHAPTER 4: SECOND-ORDER CONDITIONS
can b e established that the quadratic form-with FYY ' Fyy'• and Fy'y' evalu­
ated on the extremal-is negative definite for every t, then
< 0,
and the extremal maximizes
Similarly, positive definiteness of the
quadratic form for every t is sufficient for minimization of V. Even if we
can only establish sign semi definiteness, we can at least have the second­
order necessary conditions checked.
For some reason, however, the quadratic-form test was totally ig­
nored in the historical development of the classical calculus of variations.
In a more recent development (see the next section), however, the concav­
ity;convexity of the integrand function F is used in a sufficient condition.
While concavity;convexity does not per se require differentiability, it is true
that if the F function does possess continuous second derivatives, then its
concavity1 convexity can be checked by means of the sign semidefiniteness
of the second-order total differential of F. So the quadratic-form test
definitely has a role to play in the calculus of variations.
V.
d 2VjdE 2
4.2 THE CONCAVITY I CONVEXITY
SUFFICIENT CONDITION
A Sufficiency Theorem for Fixed-Endpoint
Problems
Just as a concave (convex) objective function in a static optimization prob­
lem is sufficient to identify an extremum as an absolute maximum (mini­
mum), a similar sufficiency theorem holds in the calculus of variations:
For the fixed-endpoint problem (2.1), if the integrand function
is concave in the variables (y, y'), then the Euler
( 4 .3) equation is sufficient for an absolute maximum of V[y ].
Similarly, if F(t, y, y ' ) is convex in (y, y'), then the Euler
equation is sufficient for an absolute minimum of V[y ].
F(t, y, y')
It should be pointed out that concavity/convexity in (y, y ' ) means concav­
ity;convexity in the two variables y and y' jointly, not in each variable
separately.
We shall give here the proof of this theorem for the concave case.1
Central to the proof is the defining property of a differentiable concave
function: The function F(t, y, y') is concave in (y, y') if, and only if, for any
This proof is due to Akira Takayama. See his Mathematical Economics, 2d ed., Cambridge
University Press, Cambridge, 1985, pp. 429-430.
1
PART 2o THE CALCULUS OF VARIATIONS
82
pair of distinct points in the domain, (t,y* , y*') and (t, y, y'), we have
( 4.4 ) F( t, y, y' ) - F( t,y* , y *')
s Fy( t , y* , y*') (y - y* ) + Fy• ( t, y* , y*') (y' - y* ' )
Fy( t, y* , y*') €p ( t ) + Fy.( t, y * , y*')€p'( t )
[ by (2 .3)]
Here y*(t) denotes the optimal path, and y(t) denotes any other path. By
integrating both sides of (4.4) with respect to t over the interval [0, T ], we
=
obtain
( 4 .5) V[y ] - V[y* ]
!T [ Fy( t, y* , y*')p( t ) + Fy. ( t, y* , y*' ) p' ( t ) j dt
T [
d
= € � p (t) Fy(t , y* , y*' ) - Fl( t, y * , y*' ) ] dt
dt
s
€
0
[integration of the Fy• ( t, y* , y*') p' ( t )
term by parts as in (2. 16)]
= 0 [ since y * ( t) satisfies the Eul�r equation (2.18)]
In other words, V (y ] s V[y*), where y(t) can refer to any other path. We
have thus identified y*(t) as a V-maximizing path, and at the same time
demonstrated that the Euler equation is a sufficient condition, given the
assumption of a concave F function. The opposite case of a convex F
function for minimizing V can be proved analogously.
Note that if the F function is strictly concave in (y, y' ), then the weak
inequality ::;; in (4.4) and (4.5) will become the strict inequality < . The
result, V[y] < V(y* ], will then establish V[y* ] to be a unique absolute
maximum of V. By the same token, a strictly convex F will make V[y*) a
unique absolute minimum.
Generalization to Variable Terminal
Point
The proof of the sufficiency theorem (4. 3) is based on the assumption of
fixed endpoints. But it can easily be generalized to problems with a vertical
terminal line or truncated vertical terminal line.
To show this, first recall that the integration-by-parts process in (2.16)
originally produced an extra term [FY.p(t)Jr, which later drops out because
it reduces to zero. The reason is that, with fixed endpoints, the perturbing
curve p(t) is characterized by p(O) = p(T ) =
When we switch to the
problem with a variable terminal point, with T fixed but y(T) free, p(T) is
no longer required to be zero. For this reason, we must admit an extra term
0.
[by (2.3)]
83
CHAPTER 4: SECOND-ORDER CONDITJONS
on the right-hand side of the second and the third lines of
rearrangement,
(4.5). Upon
(4.5) now becomes
FY.
y(t) (y - y* )
where
is to be evaluated along the optimal path, and
represents
the deviation of any admissible neighboring path
from the optimal path
y* (t).
If the last term in the last inequality is zero, then obviously the
original conclusion-that
V[y* ]
is an absolute maximum-still stands.
Moreover, if the said term is negative, we can again be sure that
[ Fiy - y* )]t=T
[F/y - y* )]t-T·
V[y * ]
constitutes an absolute maximum. It is only when
is posi­
tive that we are thrown into doubt. In short, the concavity condition on
F(t, y, y')
in
(4.3)
only needs to be supplemented in the present case by a
nonpositivity condition on the expression
But this supplementary condition is automatically met when the
transversality condition is satisfied for the vertical-terminal-line problem,
namely,
[Fy.]t = T 0. As
[Fy.Jt = T
=
calls for either
for the truncated case, the transversality condition
=
0
(when the minimum acceptable terminal value
= Ym in (when that terminal value is binding,
Ymin is nonbinding), or
thereby in effect turning the problem into one with a fixed terminal point).
y*
Either way, the supplementary condition is met. Thus, if the integrand
function
F
is concave (convex) in the variables
(y, y')
in a problem with a
vertical terminal line or a truncated vertical terminal line, then the Euler
equation plus the transversality condition are sufficient for an absolute
maximum (minimum) of
V[y ].
Checking Concavity ; convexity
For any function (, whether differentiable or not, the concavity feature can
be checked via the basic defining property that, for two distinct points
and
v
in the domain, and for
0 < 8 < 1, F
Of( u ) + ( 1 - 0) f( v )
For convexity, the inequality
�
is concave if and only if
u
f [ O u + ( 1 - O)v]
� i s reversed t o �
.
Checking this property
can, however, be very involved and tedious. For our purposes, since
F(t, y, y')
is already assumed to have continuous second derivatives, we can take the
simpler alternative of checking the sign definiteness or semidefiniteness of
the quadratic form
( 4 .6)
This quadratic form can, of course, be equivalently rewritten
(4.6' )
as
84
PART 2: THE CALCULUS OF VARIATIONS
which would fit in better with the ensuing discussion of the Legendre
condition. Once the sign of the quadratic form is ascertained, inferences can
readily be drawn regarding concavity ;convexity as follows: The
y, y')
function is concave (convex) in (y, y') if, and only if, the quadratic form q is
everywhere negative (positive) semidefinite; and the
function is strictly
concave (strictly convex) if (but not only if) q is everywhere negative
(positive) definite.
It should be noted that concavity1convexity is a global concept. This
is why it is related to absolute extrema. This is also why the quadratic form
q is required to be everywhere negative (positive) semidefinite for concavity
(convexity) of F, meaning that the sign semidefiniteness of q should hold
for all points in the domain (in the yy' space) for all t. This is a stronger
condition than the earlier-mentioned quadratic-form criterion relating
to (4.2'), because in the latter, the second derivatives of F are to be evalu­
ated on the extremal only. From another point of view, however, the
concavity;convexity sufficient condition is less strong; it only calls for sign
semi definiteness, whereas, for (4.2'), a sufficient condition would require a
definite sign.
The sign definiteness and semidefiniteness of a quadratic form can be
checked with determinantal and characteristic-root tests. Since these tests
are applicable not only in the present context of the calculus of variations,
but also in that of optimal control theory later, it may prove worthwhile to
present here a self-contained explanation of these tests.
F
F(t,
The Determinantal Test for Sign
Definiteness
The determinantal test for the sign definiteness of the quadratic form
the simplest to use. We first write the discriminant of q,
( 4.7)
IDI
- J FFy'y'
yy'
=
q
is
[ from ( 4.6' )]
and then define the following two principal minors:
and
( 4.8)
The test is
( 4.9)
Negative defmiteness of q
Positive definiteness of q
ID11 < 0, ID2 1 > o
ID11 > 0, ID2 1 > 0
(everywhere in the domain)
85
CHAPTER 4: SECOND-ORDER CONDITIONS
Though easy to apply, this test may be i:werly stringent. It is inttmded for
strict concavity;convexity, whereas the sufficient condition (4.3) merely
stipulates weak concavity ;convexity.
The Determinantal Test for Sign
Semidefiniteness
The determinantal test for the sign semi definiteness of q requires a larger
number of determinants to be examined, because under this test we must
consider all the possible ordering sequences in which the variables of the
quadratic form can be arranged. For the present two-variable case, there are
two possible ordering sequences for the two variables-(y' , y) and (y, y').
Therefore, we need to consider only one additional discriminant besides
(4.7), namely,
( 4 . 10)
IDol
=I FyFYY'y FyFyy'y''
I
[from ( 4.6)]
whose principal minors are
(4.11)
ID01 I
= I Fy) Fyy
=
ID021
and
= I FyFYY'y
FyyFy'y'
'I
ID 1 1
ID02I
For notational convenience, let us refer to the 1 X 1 principal minors
and
and
collectively as I D 1 I , and the 2 X 2 principal minors
collectively as D . Further, let us use the notation I Di l � 0 to mean that
each member of I Di l is � 0. Then the test for sign semidefiniteness is as
follows:
ID01 I
( 4 . 12)
I 21
ID21
Negative semidefiniteness of q
l D 1 1 5: o ,
Positive semidefiniteness of q
ID I I �
ID2 1
�o
0 , ID21
�0
( everywhere in the domain),
The Characteristic-Root Test
The alternative to the use of determinants is to apply the characteristic-root
test, which can reveal sign semidefiniteness as well as sign definiteness all at
once. To use this test, first write the characteristic equation
[ from ( 4 . 7 ) ]
( 4 . 13)
Then sobe for the characteristic roots
r1
and
r2 • The test revolves aro\Uld
86
PART
2: THE CALCULUS OF VARIATIONS
the signs of these roots as follows:
Negative definiteness of q
Positive definiteness of q
Negative semidefiniteness of q
<
=
r1
=
r1 >
0, r2
>
0
=
r1
s
0 , r2
s
0
0, r2 < 0
( at least one root = 0)
Positive semidefiniteness of q
(at least one root
=
0)
For this test, it is useful to remember that the two roots are tied to the
determinant I D2 1 in (4.8) by the following two relations:2
( 4.15)
( 4 . 1 6)
r1
+ r2 = sum of principal-diagonal elements = Fy'y' + FYY
r1r2 = ID2 I
For one thing, these two relations provide a means of double-checking the
calculation of r1 and r2• More importantly, they allow us to infer that, if
ID2 1 < 0, then r1 and r2 must be opposite in sign, so that q must be
indefinite in sign. Consequently, once ID2 1 is found to be negative, there is
no need to go through the steps in (4.13) and (4.14).
The shortest-distance problem (Example 4 of Sec. 2.2) has
•
the integrand function F = (1 + y '2)112 Since there is only one variable, y ' ,
in this function, it· is easy to check the second-order condition. In so doing,
bear in mind that the square root in the F expression is a positive square
root because it represents a distance, which cannot be negative. This point
becomes important when we evaluate the second derivative.
The first derivative of F is, by the chain rule,
EXAMPLE 1
2 - 1 /2
2y'
Fy' = i { 1 + y' )
=
(1 + y'2 )
- 1/2 '
y
Further differentiation yields
- 1/2
r2 - 3/2 r2
Fh' = - (l + y )
y + ( l + yr2 )
/2
= (1 + y'2 ) - 3 [ - y'2 + (1 + y' 2 ) ]
= (1 + y'2 ) -3
2
/2
>0
[positive square root]
See the discussion relating to local stability analysis in Sec. 18.6 of Alpha C. Chiang,
Fundamental Methods of Mathematical Economics, 3d ed., McGraw-Hill, New York, 1984.
87
CHAPTER 4: SECOND-ORDER CONDITIONS
The positive sign for Fy'y'-positive for all y' values and for all t-means
that F is strictly convex in the only variable y' everywhere. We can thus
conclude that the extremal found from the Euler equation yields a unique
minimum distance between the two given points.
EXAMPLE 2
Is the Euler equation sufficient for maximization or mini­
mization if the integrand of the objective functional is F(t, y , y') = 4y 2 +
4Y.Y' + y'2?
To check the concavity;convexity of F, we first find the second
derivatives of F. Since
FY
= By + 4y'
and
Fy'
= 4y + 2y'
the required second derivatives are
Fy'y'
=2
Thus, with reference to
(4.8), we have
The quadratic form
q is not positive definite.
However, with reference to (4.11), we find that
So the condition for positive semidefiniteness in (4.12) is satisfied.
To illustrate the characteristic-root test, we first write the chtp"acteris­
tic equation
I
4
r2
8 - r = - lOr = 0
r1 = 10 and r2 = 0. [This result, incidentally,
Its two roots are
confirms
and (4.16)_) Since the roots are unconditionally nonnegative, q is
everywhere positive semidefinite according to (4.14). It follows that the F
function is convex, and the Euler equation is indeed sufficient for the
minimization of V[y ].
(4.15)
EXAMPLE 3 Let us now check whether the Euler equation is sufficient for
profit maximization in the dynamic monopoly problem of Sec. 2.4. The
profit function (2.33) gives rise to the second derivatives
7T'p•p•
=
- 2ah2
7T'p•p
= 7T'pP' = h ( l + 2ab)
7T'pp
= - 2b(l + ab)
Using the determinantal test (4.9), we see that even though
here) is negative as required for the negative definiteness of q,
ID1 1 ( rrp•p•
ID 2 1 is also
88
negative:
PART 2: THE CALCULUS OF VARIATIONS
17Tp•
7Tpp•p•
Hence, by (4. 16), the characteristic roots r1 and r2 have opposite signs. The
F function is not concave, and the Euler equation is not sufficient for profit
maximization.
As we shall see in the next section (Example 3 ), however, the problem
does satisfy a second-order necessary condition for a maximum.
:;�:�:::.:::.::::�:�l=
b!&, the oomavity ond oonvoxity
sufficiency conditions are still applicable. But, in that case, the F function
must be concavejconvex (as the case may be) in all the n variables and
their derivatives (y1,
, yn, y{, . . . , y;), jointly. For the characteristic-root
test, this would mean a higher-degree polynomial equation to solve. Once
the roots are found, however, we need only to subject all of them to sign
restrictions similar to those in (4. 14). In the determinantal test for sign
definiteness, there would be a larger discriminant with more principal
minors to be checked. But the technique is well known. a
With regard to the determinantal test for sign semi definiteness, the
procedure for the n-variable case would become more complex. This is
because the number of principal minors generated by different ordering
sequences of the variables will quickly multiply. Even with only two state
variables, say, (y, z), concavity;convexity must now be checked with regard
to as many as four variables (y, z, y', z' ). For notational simplicity, let us
number these four variables, respectively, as the first, second, third, and
fourth. Then we can arrange the second-order derivatives of F into the
following 4 X 4 discriminant:
• • .
(4. 17)
IDI
=
Fu
F21
Fal
F•1
F12
F22
F1a
F2a
Fa2
F42
Faa
F4a
F1•
F2•
Fa4
F44
=
FYY
Fzy
Fy'y
Fz'y
Fyz
F••
Fy'z
F
•
• •
Fyy'
Fzy'
Fyz'
F•••
Fy'y'
Fz'y'
Fy'z'
Fz'z'
•1f.·
·_
... ,.: _ .
Since each of the variables can be taken as the '' first" variable in a difterent
3See, for example, Alpha C. Chiang, Fundamental Methods of Mathematical Economics, 3d ed.,
McGraw-Hill, New York, 1984, Sec. 1 1 .3.
-�
89
CHAPTER 4: SECOND-ORDER CONDITIONS
1X 1
ordering sequence, there exist four possible
principal minors:
( 4.18)
11\ 1 -
We shall denote these 1 X 1 principal minors collectively by
The total number of possible 2 X 2 principal minors is 12, but only six
distinct values can emerge from them. These values can be calculated from
the following six principal minors:
( 4 . 19)
I
I
FY•
F••
FYY
F.Y
I I
I I
F.. Fzy'
Fy' z Fy'y'
FYY Fyy'
Fy'y Fy'y'
F..
F
•
• •
F••.
F
• •
• •
I I
I I
FYY Fyz'
Fz'y Fz'z'
Fy'y'
Fz'y'
I
Fy'z'
Fz'z'
l
i·.'
The reason why the other six can be ignored is that the principal minor with
in the diagonal is always equal in value to the one with the two
variables in reverse order, ( Fj1, Fi), in the diagonaL The 2 X 2 principal
�
minors in (4.19) will be collectively referred to as
As to 3 X 3 principal minors, the total number available is 24. From
these, however, only four distinct values can arise, and they can be calcu­
lated from the following principal minors:
(Fii• Fj)
I D2 1 .
( 4.20)
>.
Fyz
F.,
F
Fyz
F••
Fy'z
Fyy'
Fzy'
Fy'y'
FYY
Fzy
Fz'y
FYY Fyy'
Fy'y Fy'y'
Fz'y Fz'y'
Fyz'
Fy'z'
Fz'z'
F•• Fzy' F.,.
Fy'z Fy'y' Fy'z'
Fz'y' F
F,
FYY
Fzy
Fy'y
•
•
• •
Fyz'
F
Fz'z'
•
••
• •
• •
•
We can ignore the others -because the principal minor with ( Fu , Fj1 , Fkk) in
the diagonal always has the same value as those with F;i, Fj1 , Fkk arranged
in any other order in the diagonal. We shall refer to the principal minors in
(4.20) collectively as
Finally, we need to consider only one 4 X 4 principal minor,
in (4.17).
which is identical in value with
One question naturally arises: How does one ascertain the number of
distinct values that can arise from principal minors of various dimensions?
The answer is that we can just apply the formula for the number of
combinations of n objects taken r at a time, denoted by C;':
ID3 1 .
I D4 1 ,
ID I
( 4.21)
e
n =
r
----­
n!
r!( n - r) !
' . . .. ·· .·.•
' ...;
.. ..
90
PART 2: THE CALCULUS OF VARIATIONS
For (4.18), we have four variables taken one at a time; thus the number of
combinations is
ct =
For
4!
-4
1 ! 3!
--
2 X 2 principal minors, w e have
.. -�
; '�
-
4
c2
.
=
4!
2, 2 ,
= s
.Ud for 3 X 3 principal minors, the formula yields
4!
c� = 3, 1 !
=4
4 principal minors, a unique V!Uue �. becaue
Finally, for 4 X
C4
=
4
4!
4! 0!
--
=
·
·-:;: .;;"
1
Once all the principal minors have been calculated, the test for sign
semidefiniteness is similar to (4.12). Using the notation ID; I � 0 to mean
that each member of ID; I is � 0, we have the following:
Negative semidefiniteness of q
�
( 4.22)
ID1 1 ::; o, ID2 1
� o,
ID31 ::; o, ID4 1
�o
� o,
� o,
ID31
� o,
�o
Positive semidefiniteness of q
ID1 1
ID21
ID41
( everywhere in the domain)
EXERCISE 4.2
1
I
J
For Prob.
1 of Exercise 2.2:
F function is strictly concave/convex in (y,y') by
(a)
Check whether the
(b)
the determinantal test
(c)
(4.9).
If that test fails, check for concavity1convexity by the determinantal
test
(4.12) or the characteristic-root test.
Is the sufficient condition for a maximum/minimum satisfied?
Perform the tests mentioned in Prob.
Perform the tests mentioned in Prob.
1 above on Prob. 3 of Exercise 2.2.
1 above on Prob. 5 of Exercise 2.2.
91
CHAPTER 4: SECOND-ORDER CONDITIONS
4
(a) Show that, i n the inflation-unemployment tradeoff model of Sec. 2.5,
the integrand function is strictly convex.
( b ) What specifically can you conclude from that fact?
4.3
THE LEGENDRE NECESSARY
CONDITION
The concavity feature described in (4.4) is a global concept. When that
feature is present in the integrand function F, the extremal is guaranteed
to maximize V[y]. But when F is not globally concave, as may often happen
(such as in the dynamic monopoly problem), we would have to settle for
some weaker conditions. The same is true for convexity. In this section, we
introduce a second-order necessary condition known as the Legendre condi­
tion, which is based on local concavity;convexity. Though not as powerful
as a sufficient condition, it is very useful and indeed is used frequently.
The Legendre Condition
The great merit of the Legendre condition lies in its extreme simplicity, for
it involves nothing but the sign of Fy'y'· Legendre, the mathematician,
thought at one time that he had discovered a remarkably neat sufficient
condition: Fy'y' < 0 for a maximum of V, and Fy'y' > 0 for a minimum of Y.
But, unfortunately, he was mistaken. Nonetheless, the weak-inequality
version of this condition is indeed correct as a necessary condition:
( 4.23 )
Maximization of V[y]
�
Fy'y' � 0
for all t
e
[0, T ]
Minimization of V[y]
�
Fy'y' ;;:: 0
for all t
e
[0, T ]
[Legendre necessary condliOD]
The Fy'y' derivative is to be evaluated along the extremal.
The Rationale
To understand the rationale behind (4.23), let us first transform the middle
term in the integrand of (4.2') into a squared term by integration by parts.
Let v = Fyy' and u = p2(t), so that
dv
=
dFyy'
dt
dt
--
and
du
=
2p( t)p'(t) dt
92
PART 2 : THE CALCULUS OF VARIATIONS
Then the middle term in (4.2') can be rewritten as
( 4 .24)
0
=
-!
0
T
dFyy
' dt
p 2 ( t)
dt
[ p( O )
=
p(T)
where all the limits of integration refer to values of t (not
this into (4.2') yields
( 4.25)
d2V
=
de2
! [(
T
0
FYY -
dFyy'
-;{t
)
p 2 ( t)
u
+ Fh'p' 2 ( t)
]
=
0 assumed]
or v ) . Plugging
dt
where the integrand now consists of two squared terms p2(t) and p'2(t).
It turns out the p'2(t) term will dominate the other one over the
interval [0, T ], and thus the honpositivity (nonnegativity) of its coefficient
Fy'y' is necessary for the nonpositivity (nonnegativity) of d2Vjd e 2 for
maximization (minimization) of V, as indicated in (4.23). But because of the
presence of the other term in (4.25), the condition is not sufficient. That the
p'2(t) term will dominate can be given an intuitive explanation with the help
of the illustrative perturbing curves in Fig. 4.1. Figure 4.1a shows that
p'(t), the slope of p(t), can take large absolute values even if p(t) maintain
small values throughout; Fig. 4.1b shows that in order to attain large p(t)
values, p'(t) would in general have to take large absolute values, too. These
considerations suggest that the p' 2(t) term tends to dominate.
Note that, to reach the result in (4.25), we have made the assumption
that p(O) = p(T ) = 0 on the perturbing curve. What will happen if the
terminal point is variable and p(T ) is not required to be zero? The answer
p(t)
p(t)
'
. �a)
FIGVU 4.1
,
,
.·
(b)
93
CHAPTER 4: SECOND-ORDER CONDITIONS
[Fyy'p 2(t)lt�T·
is that the zero term on the last line of (4.24) will then be replaced by the
new expression
But even then, it is possible for some admissi­
ble neighboring path to have its terminal point located exactly where
is, thereby duplicating the result of
= 0, and reducing that new
expression to zero. This means that the eventuality of (4.25) is still possible,
and the Legendre condition is still needed. Hence, the Legendre necessary
condition is valid whether the terminal point is fixed or variable.
y*(T)
p(T)
EXAMPLE 1
In the shortest-distance problem (Example 1, preceding sec­
tion), we have already found that
Fy'y'
(1 + y'
=
2 ) - 3/ 2 > 0
Thus, by (4.23), the Legendre necessary condition for a minimum is satis­
fied. This is only to be expected, since that problem has earlier been shown
to satisfy the second-order s ufficient condition for minimization.
EXAMPLE 2
In Example 3 of Sec. 2.2, we seek to find a curve between two
fixed points that will generate the smallest surface of revolution. Does the
extremal (pictured in Fig. 2.4) minimize the surface of revolution? Since
that problem has
FY. = yy'( l 2 ) - 1/2
+y
'
we can, after differentiating and simplifying, get
Fy'y' y( 1 + ,2) -3/2
=
(1 + y'2) -312
y
expression is positive, and y-represented by the height
The
of the AZ curve in Fig. 2.4-is positive along the entire path. Therefore,
is unconditionally positive for all t in the interval
and the
Legendre condition for a minimum is satisfied.
To make certain that
is indeed positive for all t, we see from the
general solution of the problem
Fy'y'
[a, z],
y(t)
y* ( t)
=
c
[ e<t+k)/c + e-(t+k)/c]
2
y*(t)
that
takes the sign of the constant c, because the two terms in
parentheses are always positive. This means that
can only have a
single sign throughout. Given that points A and Z show positive y values,
must be positive for all t in the interval
Unlike the shortest-distance problem, however, the minimum-surface­
of-revolution problem does not satisfy the sufficient condition.
y*(t)
y*(t)
[a, z].
: :;; .
94
PART
2, THE CALCULUS OF VARIATIONS
EXAMPLE 3
For the dynamic monopoly problem, we have found in the
preceding section that the integrand function rr(P, P') is not concave, so it
does not satisfy the sufficient condition (4.3). However, since
rrp'p'
=
-
2ah2
<
0
the Legendre necessary condition for a maximum is indeed satisfied.
The n-Variable Case
The Legendre necessary condition can, with proper modification, also be
applied to the problem with n state variables (y v . . . , yn ). But instead of
checking whether Fi:l � 0 or F11 � 0, as in (4.23), we must now check
whether the n X n matrix [ Fy; 'y ']-or, alternatively, the quadratic form
whose coefficients are FY/Y _.-is negative semidefinite (for maximization of
1
V) or positive semidefinite (for minimization of V). For this purpose, first
define
( 4 .26)
where the second-order derivatives are to be evaluated along the extremals.
Then write all the 1 X 1 principal minors I � 1 1 , all the 2 X 2 principal
minors ) A 2 ) , and so forth, as explained in the text following (4.17). The
Legendre second-order necessary condition is
( 4.27)
for all t
E
[0, T ]
for all t
E
[0, T ]
[Legendre necessary condition]
It is clear that (4.27) includes (4.23) as a special case.
Note that the Legendre condition in (4.27), though apparently similar
to the determinantal test for sign semidefiniteness in (4.22), differs from the
latter in two essential respects. First, the sign-semidefiniteness test in (4.22)
involves derivatives of the FYY and Fyy' types as well as Fy'y'• but the
Legendre condition is based exclusively on the Fii type. This is why we
employ the symbol ! Il l in (4.26) and (4.27), so as to keep it distinct from the
IDI symbol used earlier in (4.22). Second, unlike the global concavity;con­
vexity characterization, the Legendre condition is local in nature; so the
second-order derivatives in (4.26) are to be evaluated along the extremals
only.
95
CHAPTER 4: SECOND-ORDER CONDITIONS
EXAMPLE 4
In Exercise 2.3, Prob.
integrand function
F = y'2
Since
Fy' = 2y' + z'
2, two
'
state
+ z 2 + y'z'
F2•
and
=
(4.26) results in
IAI
2z' + y'
Fz'z'
we find that
Substitution of these into
variables appear in the
=2
= li ��
In this case, we find that
and
Thus, by
1 �I =
2
I A2 1 = 1
3
(4.27), the Legendre condition for a minimum is satisfied.
EXERCISE 4.3
4.4
1
Check whether the extremals obtained in Probs. 6, 8, and 9 of Exercise 2.1
satisfy the Legendre necessary condition for a maximum/minimum.
2
Check whether Prob. 3 of Exercise 2.3, a two-variable problem, satisfies the
Legendre condition for a maximum/minimum.
3
Does the inflation-unemployment problem of Sec. 2.5 satisfy the Legendre
condition for minimization of total social loss?
FffiST AND SECOND VARIATIONS
Our discussion of first-order and second-order conditions has hitherto been
based on the concept of the first derivative dVIdE and the second derivative
d2VjdE2• An alternative way of viewing the problems centers around the
concepts of first variation and second variation, which are directly tied to
the name calculus of variations itself.
The calculus of variations involves a comparison of path values V[y]
and V[y* ]. The deviation of V[y] from V[y* ] is
(4.28) AV = V [y] - V ( y* ] = fTF(t , y , y') dt - f TF(t,y* , y*' ) dt
0
\. i
0
96
PART 2: THE CALCULUS OF VARIATIONS
When the integrand of the first integral is expanded into a Taylor series
around the point
it would contain a term
which
would allow us to . cancel out the second integral in (4.28). That Taylor
series is
(t, y*, y*'),
F(t, y*, y*'),
F(t,y,y') F(t,y* ,y*')
+ [ �(t - t) + Fy( y
y*) + Fy• (Y - y*')]
'
1
+ -d Fu(t - t)2 + Fyy(Y - y*)2 + Fy•y• (Y' - y*')2
2.
+2Fty(t - t)(y - y* ) + 2Fty•( t - t)(y' - y*')
+ 2 F y ( Y - y*)(y' - y*') ] +
+ Rn
where all the partial derivatives of F are to be evaluated at (t,y*,y*'). We
have, for the sake of completeness, included the terms that involve (t - t),
( 4.29)
=
-
'
y •
·
· ·
but, of course, these would all drop out. Recalling (2.3), we can substitute
y - y* = ep, and y' - y*' ep' , to get
(4 .29') F(t,y, y') F(t,y*,y*') + Fyep + FY.ep'
1
+ -d
Fyy( ep )2 + Fy'y'( ep')2 + 2Fyy• ( ep )( ep') ]
2.
+ . . . + Rn
=
=
.•·i
Using thi.41 result in
(4.28), we can then transform the latter into
( 4 .30)
€ 2 fT( FyyP2 + Fy'y'P,2
+-
2
0
+
2F:trPJI
) dt
+ h .o.t. (higher-order-terms)
In the calculus-of-variations literature, the first integral in
referred to as the
denoted by
(4.30)
is
first variation,
8V:
( 4 .3 1 )
BV = �T( F:tp + Frp') dt dV
de [by (2. 13) ]
Similarly, the second integral in (4.30) is known
the second .variation,
denoted by 8 2 V:
=
as
[by ( 4 .2')]
- .
-·· <"
. . . ... '
CHAPTER 4: SECOND-ORDER CONDITIONS
97
In a maximization problem, where .1V :,; 0, it is clear from (4.30) that
it is necessary to have 8V = 0, since e can take either sign at every point of
time. This is equivalent to the first-order condition dV1d e = 0, which in
our earlier discussion has led to the Euler equation. Once the 8V = 0
condition is met, it is further necessary to have 8 2V :,; 0, since the coeffi­
cient e 2 /2 is nonnegative at every point of time. This condition is equiva­
lent to d 2 Vjd e 2 :,; 0, the second-order necessary condition of Legendre.
Analogous reasoning would show that, for a minimization problem, it
is necessary to have 8V = 0 and 8 2 V � 0. In sum, the first-order and
second-order necessary conditions can be derived from either the derivative
route or the variation route. We have chosen to follow the derivative route
because it provides more of an intuitive grasp of the underlying reasoning
process.
CHAPTER
5
INFINITE
PLANNING
HORIZON
'
'
For an individual, it is generally adequate to plan for a finite time interval
[0, T ], for even the most far-sighted person probably would not plan too far
beyond his or her expected lifetime. But for society as a whole, or even for a
corporation, there may be good reasons to expect or assume its existence to
be permanent. It may therefore be desirable to extend its planning horizon
indefinitely into the future, and change the interval of integration in the
objective functional from [0, T ] to [0, oo]. Such an extension of horizon has
the advantage of rendering the optimization framework more comprehen­
sive. Unfortunately, it also has the offsetting drawback of compromising the
plausibility of the often-made assumption that all parameter values in a
model will remain constant throughout the planning period. More impor­
tantly, the infinite planning horizon entails some methodological complica­
tions.
5.1
METHODOLOGICAL ISSUES
OF INFINITE HORIZON
With an infinite horizon, at least two major methodological issues must be
addressed. One has to do with the convergence of the objective functional,
and the other concerns the matter of transversality conditions.
98
99
CHAPTER 5: INFINITE PLANNING HORIZON
The Convergence of the Objective
Functional
The convergence problem arises because the objective functional, now in the
form of J;F(t, y, y') dt, is an improper integral which may or may not have
a finite value. In the case where the integral diverges, there may exist more
than one y(t) path that yields an infinite value for the objective functional
and it would be difficult to determine which among these paths is optimal. 1
It is true that even in the divergent situation, various ways have been
proposed to make an appropriate selection of a time path from among all the
paths with an infinite integral value. But that topic is complicated and,
besides, it does not pertain to the calculus of variations as such, so we shall
not delve into it here.2 Instead, as a preliminary to the discussion of
economic applications in the two ensuing sections, we shall make some
comments on certain conditions that are (or allegedly are) sufficient for
convergence.
Condition I Given the improper integral J;F<t, y, y ' ) dt, if the integrand
F is finite throughout the interval of integration, and if F attains a zero
value at some finite point of time, say, t0, and remains at zero for all t > t0,
then the integral will converge.
This is in the nature of a sufficient condition. Although the integral
nominally has an infinite horizon, the effective upper limit of integration is
a finite value, t0• Thus, the given improper integral reduces in effect to a
proper one, with assurance that it will integrate to a finite value.
Condition II (False)
Given the improper integral J;F(t, y, y' ) dt, if F �
0 as t � oo, then the integral will converge.
This condition is often taken to be a sufficient condition, but it is not.
To see this, consider the two integrals
=f
00
(5.1)
I1
0 (t
1
+ 1)2
dt
and
[
2
= f0 t 1 1 dt
00
+
--
Each of these has an integrand that tends to zero as t --> oo. But while
11
1For a detailed discussion of the problem of convergence, see S. Chakravarty, "The Existence
of an Optimum Savings Program," Ecorwmetrica, January 1962, pp. 178-187.
2
The various optimality criteria in the divergent case are concisely summarized in Atle
Seierstad and Knut Sydsreter, Optimal Control Theory with Economic Applications, Elsevier,
New York, 1987, pp. 231-237.
100
PART 2: THE CALCULUS OF VARIATIONS
converges,
I2
does not:
(5.2)
/1
= lim
b - oo
[ -1 ] b 1
f+1
-­
12
O
=
lim ln ( t
b-+oo [
1)]� = oo
+
What accounts for this sharp difference in results? The answer lies i n the
speed at which the integrand falls toward zero. In the case of
/1,
where the
denominator of the integrand is a squared term, the fraction falls with a
sufficiently high speed as
convergence. In the case of
t
takes increasingly larger values, resulting in
/2 , on the other hand,
the speed of decline is not
great enough, and the integral diverges. 3 The upshot is that, in and of itself,
the condition
"F
-> 0 t -> oo"
as
does not guarantee convergence.
It is of interest also to ask the converse question: Is the condition
" F -> 0 as t -> oo" a necessary condition for J';F(t, y, y') dt to converge?
Such a necessary condition seems intuitively plausible, and it is in fact
4
commonly accepted as such. But counterexamples can be produced to show
that an improper integral can converge even though F does not tend to
zero, so the condition is not necessary. For instance, if
F( t)
�
{: if n :<;; t :<;; n + 2n1
(n = 1,2, ... )
otherwise
� F(t)
th n
has no limit as t becomes infinite; yet the integral
converges to the magnitude
J; sin t2 dt
I:J: _ 1(1jk 2). 5 As
J;F(t) dt
another example, the integral
can be shown to converge to the value
/2
4 ../rr
sine-function integrand does not have zero as its limit.
6
even though the
Note, however, that these counterexamples involve either an unusual
type of discontinuous integrand or an integrand that periodically changes
sign, thereby allowing the contributions to the integral from neighboring
time intervals to cancel out one another. Such functions are not usually
3
For an integral
J;F(t) dt
whose integrand maintains a single sign (say, positive), the
following criterion can be used: The integral
converges
if
F(t) vanishes
at infinity to a higher
order than the first, that is, if there exists a number a > 1 such that, for all values of
matter how large), 0 <
F(t)
F(t)
:S
Mjta
holds, where
M is
a constant. The integral
t (no
diverges if
vanishes at infinity to an order not higher than the first, that is, if there is a positive
constant
N
such that
tF(t) " N
> 0. See
R.
Courant,
Differential and Integral Calculus,
translated from German by E. J. McShane, 2d ed., lnterscience, New York, 1937, Vol. 1,
pp. 249-250.
4
See, for example, S. Chakravarty, " The Existence of an Optimum Savings Program,"
Econo­
metrica, January 1962, p. 182, and G. Hadley and M. C. Kemp, Variational Methods in
Economics, North-Holland, Amsterdam, 1971, pp. 52, 63.
5
See Watson Fulks, Mvanced Calculus: An Introduction to Analysis, 3d ed., Wiley, New York,
1978, pp. 570-571 .
6
See R. Courant, Differential
and Integral Calculus,
lnterscience, New York, Vol. 1 , p . 253.
101
CHAPTER 5: INFINITE PLANNING HORIZON
used in the objective functionals in economic models. Typically, the inte­
grand in an economic model-representing a profit function, utility func­
tion, and the like-is assumed to be continuous and nonnegative. For such
functions, the counterexamples would be irrelevant.
Condition III
In the integral J�F(t , y, y') dt, if the integrand takes the
form of G(t, y, y')e -P1, where p is a positive rate of discount, and the G
function is bounded, then the integral will converge.
A distinguishing feature of this integral is the presence of the discount
factor e - pt which, ceteris paribus, provides a dynamic force to drive the
integrand down toward zero over time at a good speed. When the G(t, y, y')
component of the integrand is positive (as in most economic applications)
and has an upper hound, say, G, then the downward force of e -pt is
sufficient to make the integral converge. More formally, since the value of
the G function can never exceed the value of the constant G, we can write
( 5 .3)
f G ( t, y , y' ) e -pt dt f Ge - 1 dt
00
0
;s;
00
0
"
=
(;
p
-
The last equality, based on the formula for the present value of a perpetual
constant flow, shows that the second integral in (5.3), with the upper bound
(; in the integrand, is convergent. It follows that the first integral, whose
integrand is G ;s; G, must also be convergent. We have here another suffi­
cient condition for convergence.
There also exist other sufficient conditions, but they are not simple to
apply, and will not be discussed here. In the following, if the F function is
given in the general form, we shall always assume that the integral is
convergent. This would imply, for normal economic applications, that the
integrand function F tends to zero as t tends to infinity. When specific
functions are used, on the other hand, convergence is something to be
explicitly checked.
The Matter of Transversality Conditions
Transversality conditions enter into the picture when either the terminal
time or the terminal state, or both, are variable. When the planning horizon
is infinite, there is no longer a specific terminal T value for us to adhere to.
And the terminal state may also be left open. Thus transversality conditions
are needed.
To develop the transversality conditions, we can use the same proce­
dure as in the finite-horizon problem of Chap. 3. Essentially, these condi­
tions would emerge naturally as a by-product of the process of deriving the
Euler equation, as in (3.9), which emanates from (3.7). In the present
""!!
102
·- · � � M V-=�
context, (3.9) must be modified to
where each of the two terms must individually vanish.
Since there is no fixed T in the present context, !::. T is perforce
nonzero, and this necessitates the condition
( 5 .5)
lim ( F
t � O>
- y'Fy') = 0
[transversality condition for infinite horizon]
This condition has the same economic interpretation as transversality
condition (3.11) in the finite-horizon framework. If the problem pertains to
a profit-maximizing firm, for instance, the condition requires the firm to
take full advantage of all profit opportunities.
As to the second term in (5.4), if an asymptotic terminal state is
specified in the problem:
( 5 .6)
lim y(t) = y.. = a given constant
t � ac
[which is the infinite-horizon counterpart of y(T) = Z in the finite-horizon
problem], then the second term in (5.4) will vanish on its own (!::. yr = 0) and
no transversality condition is needed. But if the terminal state is free, then
we should impose the additional condition
( 5 . 7)
J'
:j
i
( 5.4)
( transversality condition for free terminal state]
This condition can be given the same economic interpretation as transver­
sality condition (3.10) for the finite-horizon problem. If the problem is that
of a profit-maximizing firm with the state variable y representing capital
stock, the message of (5. 7) is for the firm to use up all its capital as t --> oo.
It may be added that if the free terminal state is subject to a restric­
tion such as y.. � Y min• then the Kuhn-Tucker conditions should be used
[cf. (3. 17) and (3.17')]. In practical applications, however, we can always
apply (5. 7) first. If the restriction y.. � y min is satisfied by the solution, then
we are done with the problem. Otherwise, we have to use Y min as a given
terminal state.
Although these transversality conditions are intuitively reasonable,
their validity is sometimes called into question. This is because in the field
of optimal control theory, to be discussed in Part 3, alleged counterexamples
have been presented to show that standard transversality conditions are not
necessarily applicable in infinite-horizon problems. And the doubts have
spread to the calculus of variations. When we come to the topic of optimal
control, however, we shall argue that these counterexamples are not really
valid counterexamples. Nevertheless, it is only fair to warn the reader at
this point that there exists a controversy surrounding this aspect of
infinite-horizon problems.
CHAPTER 5: INFINITE PLANNING HORIWN
103
One way out of the dilemma is to avoid the transversality conditions
and use plain economic reasoning to determine what the terminal state
should be, as t --+ oo. Such an approach is especially feasible in problems
where the integrand function F either does not contain an explicit t
argument (an " autonomous" problem in the mathematical sense or where
the t argument appears explicitly only in the discount factor e-1>t (an
"autonomous" problem in economists' usage). In a problem of this type,
there is usually an implicit terminal state Yoo-a steady state-toward which
the system would gravitate in order to fulfill the objective of the problem. If
we can determine Yoo• then we can simply use the terminal condition (5.6) in
place of a transversality condition.
5.2 THE OPTIMAL INVESTMENT PATH
OF A FIRM
The gross investment of a firm, Ic, has two components: net investment,
dKjdt, and replacement investment, assumed equal to oK, where o is
the depreciation rate of capital K. Since both components of investment are
intimately tied to capital, the determination of an optimal investment path,
1/ (t), is understandably contingent upon the determination of an optimal
capital path, K*(t). Assuming that investment plans are always realizable,
then, once K*(t) has been found, the optimal investment path is simply
I =
( 5 .8)
I/ ( t) = K *'( t ) + oK*(t )
But if, for some reason, there exist obstacles to the execution of investment
plans, then some other criterion must be used to determine I/ (t) from
K*(t). In the present section, we present two models of investment that
illustrate both of these eventualities.
The Jorgenson Model7
In the Jorgenson neoclassical theory of investment, the firm is assumed to
produce its output with capital K and labor L with the " neoclassical
production function" Q = Q(K, L ) that allows substitution between the
two inputs. This feature distinguishes it from the acceleration theory of
investment in which capital is tied to output in a fixed ratio. The neoclassi­
cal production function is usually accompanied by the assumptions of
positive but diminishing marginal products (QK, QL > 0; QKK• QLL < 0) and
constant returns to scale (linear homogeneity).
7
Dale W. Jorgenson, " Capital Theory and Investment Behavior," American Economic Review,
May 1963, pp. 247-259. The veraion given here omits certain less essential features of the
original model.
PART 2, THE CALCULUS OF VARIATIONS
104
The firm's cash revenue at any point of time is PQ, where P is the
given product price. Its cash outlay at any time consists of its wage bill, WL
( W denotes the money wage rate), and its expenditure on new capital, mig
( m denotes the price of " machine"). Thus the net revenue at any point of
time is
PQ( K, L) -
WL -
m ( K' + oK)
Applying the discount factor e -pt to this expression and summing over time,
we can express the present-value net worth N of the firm as
( 5 .9)
N[ K, L ]
=
{0 [ PQ( K, L ) -
WL -
m ( K' + oK)) e-p' dt
The firm's objective is to maximize its net worth N by choosing an optimal
K path and an optimal L path.
The objective functional in (5.9) is an improper integral. In view of the
presence of the discount factor e-pt, however, the integral will converge,
according to Condition III of the preceding section, if the bracketed net-rev­
enue expression has an upper bound. Such would be the case if we can rule
out an infinite value for K', that is, if K is not allowed to make a vertical
jump.
There are two state variables, K and L, in the objective functional; the
other symbols denote parameters. There will be two Euler equations yield­
ing two optimal paths, K*(t) and L*(t), and the K*(t) path can then lead
us to the 1/(t) path. We may expect the firm to have a given initial capital,
K0, but the terminal capital is left open.
Optimal Capital Stock
In applying the Euler equations
d
FK - - FK' = 0
dt
.".-··.�·. < . .' ' .
•
to the present model, we first observe that the integrand in (5.9) is linear in
both K' and L' . (The L' term is altogether absent; i.e., it has a zero
coefficient.) Thus, in line with our discussion in Sec. 2.1, the Euler equa­
tions will not be differential equations, and the problem is degenerate. The
partial derivatives of the integrand are
FK = ( PQK - m o)e-pt
FL = ( PQL - W ) e -p1
and the Euler equations indeed emerge as the twin conditions
(5. 10 )
QK =
m(o +
p --
p)
and
for all t
,,. ; ,
�
0
105
CHAPTER 5: INFINITE PLANNING HORIZON
with no derivatives of t in them. In fact, they are nothing but the standard
" marginal physical product
real marginal resource cost" conditions 8 that
arise in static contexts, but are now required to hold at every t.
The important point about (5. 10) is that it implies constant K* and L*
solutions. Taking a generalized Cobb-Douglas production function Q =
K"L/3, (a + f3 * 1), for instance, we have Q K = aK" - 1L 13 and QL =
{3K"L f3 - l. When these are substituted into (5. 10), we obtain (after eliminat­
ing L)
=
( 5 . 1 1 ) K* =
[
m(8
+
aP
p)
](f3 -l}j(l -n-f3)( w ) -f3/(l -a-f3)
-
{3 P
= constant
And substitution o f this K* back into (5.10) can give u s a similar constant
expression for L* . The constant K* value represents a first-order necessary
condition to be satisfied at all points of time, including t = 0. Therefore,
unless the firm's initial capital, K0 , happens to be equal to K*, or the firm
is a newly instituted enterprise free to pick its initial capital (variable initial
state), the condition cannot be satisfied.
The Flexible Accelerator
If it is not feasible to force a jump in the state variable K from K0 to K*,
the alternative is a gradual move t o K*. Jorgenson adopts what amounts to
the SQ-called flexible-accelerator mechanism to effect a gradual adjustment
in net investment. The essence of the flexible accelerator is to eliminate in a
systematic way the existing discrepancy between a target capital, K, and the
actual capital at time t, K(t):9
( 5 . 1 2)
I( t ) = i [ K - K(t)]
( 0 < j < 1)
The gradual nature of the adjustment in the capital level would presumably
enable the firm to alleviate the difficulties that might be associated with
abrupt and wholesale changes. While the particular construct in (5.12) was
originally introduced by economists merely as a convenient postulate, a
theoretical justification has been provided for it in a classic paper by Eisner
and Strotz.
8
This fact is immediately obvious from the second equation in (5.10). As for the first equation,
mlJ means the marginal depreciation cost, and mp means the interest·earning opportunity cost
of holding capital, and their sum constitutes the marginal user cost of capital, which is the
exact counterpart of W on the labor side.
The correct application of the flexible accelerator calls for setting the target K at the level in
(5. 1 1 ), but the Jorgenson paper actually uses a substitute target that varies with time, which is
suboptimal. For a discussion of this, see John P. Gould, " The Use of Endogenous Variables in
Dynamic Models of Investment," Quarterly Journal of Economics, November 1969, pp.
580-599.
9
PART 2o THE CALCULUS OF VARIATIONS
106
The Eisner-Strotz Model10
The Eisner-Strotz model focuses on net investment as a process that
expands a firm's plant size. Thus replacement investment is ignored. As­
suming that the firm has knowledge of the profit rate '1T associated with
each plant size, as measur.ed by the capital stock K, we have a profit
function 7T( K)-which, incidentally, provides an interesting contrast to the
profit function 7T(L) in the labor-adjustment model of Sec. 3.4. To expand
the plant, an adjustment cost C is incurred whose magnitude varies posi­
tively with the speed of expansion K'(t). So we have an increasing function
C = C(K'), through which the firm's internal difficulties of plant adjust­
ment as well as the external obstacles to investment (such as pressure on
the capital-goods industry supply) can be explicitly taken into account in the
optimization problem of the firm. If the C function adequately reflects these
adjustment difficulties, then once the K*(t) path is found, we may just take
its derivative K*'(t) as the optimal net investment path without having to
use some ad hoc expedient like the flexible-accelerator mechanism.
The objective of the firm is to choose a K*(t) path that maximizes the
total present value of its net profit over time:
( 5 . 13)
Maximize
TI[ K )
subject to
K(O)
{0 [ 7r ( K ) - C ( K')]e-P' dt
=
=
K0
( K0 given)
This functional is again an improper integral, but inasmuch as the net
return can be expected to be bounded from above, convergence should not
be a problem by Condition III of the preceding section. Note that although
we have specified a fixed initial capital stock, the terminal capital stock is
left open. Note also that this problem is autonomous.
The Quadratic Case
It is possible to apply the Euler equation to the general-function formula­
tion in (5.13). For more specific results, however, let us assume that both
the 7T and C function are quadratic, as graphed in Fig. 5. 1 :
(5.14)
'1T =
a. K - {3K2
( 5 . 15)
C
aK' 2
10
=
+
bK'
( a , f3 > 0)
( a , b > 0)
Robert Eisner and Robert H. Strotz, "Determinants of Business Investment," in
Monetary Policy,
Impacts of
A Series of Research Studies Prepared for the Commission on Money and
Credit, Prentice-Hall, Englewood Cliffs, NJ,
here is contained in
Sec.
1963,
pp.
60-233.
The theoretical model discussed
1 of their paper; see also their App. A.
107
CHAPTER 5: INFINITE PLANNING HORIZON
c
0
(b)
(a)
FIGuaE 5.1
The profit rate has a maximum, attained at K = a/2{3. The C curve has
been drawn only in the first quadrant, where K' > 0, because we are
confining our attention to the case of plant expansion. Considered together,
these two curves should provide an upper bound for the net return 7T C,
to guarantee convergence of the improper integral in (5.13).
In this quadratic model, we have
-
F = ( a K - {3K2 - aK'2 - bK' )e-pt
with derivatives:
Fx = (a - 2{3K ) e -pt
Fx• = - (2aK' + b ) e -P1
(5 . 16)
�K' = p( 2aK' + b ) e -p1
Thus, by (2.19), we have the Euler equation
f3
K" - p K ' - - K =
a
( 5 .17)
bp
- a
, . . ·� .
�. 1
--
2a
with general solution11
( 5.18 )
11
The Euler equation given by Eisner and Strotz contains a minor error-the coefficient of the
K term in (5.17) is shown
as
-f3j2a instead of -f3ja. As a result, their characteristic roots
and particular integral do not coincide with (5.18). This fact, however, does not affect their
qualitative conclusions.
_\,· ·
PART 2, THE CALCULUS OF VARIATIONS
108
where
[characteristic roots)
and
[particular integral]
The characteristic roots r1 and r2 are both real, because the expression
under the square-root sign is positive. Moreover, since the square root itself
exceeds p, it follows that r1 > p > 0, but r2 < 0. AB to the particular
integral K, it is uncertain in sign. Economic considerations would dictate,
however, that it be positive, because it represents the intertemporal equilib­
rium level of K. For the problem to be sensible, therefore, we must stipulate
that
a > bp
( 5 . 19)
The Definite Solution
To get the definite solution, we first make use of the initial condition
K(O) = K0, which enables us-after setting t = 0 in (5. 18)-to write
( 5 .20)
We would like to see whether we can apply the terminal condition (5.6).
Even though no explicit terminal state is given in the model, the au­
tonomous nature of the problem suggests that it has an implicit terminal
state-the ultimate plant size (capital stock) to which the firm is to expand.
AB Fig. 5.1a indicates, the highest attainable profit rate occurs at plant size
K = aj2{3. Accordingly, we would not expect the firm to want to expand
beyond that size. Mter the cost function C is taken into account, moreover,
the firm may very well wish to select an ultimate size even smaller. In fact,
from our knowledge of differential equations, the particular integral K
would be the logical candidate for the ultimate plant size K�. Referring to
(5. 18), however, we see that while the second exponential term (where
r2 < 0) tends to zero as t --> oo, the first exponential term (where r1 > 0)
tends to ± oo if A1 � 0. Since neither + co nor - co is acceptable as the
terminal value for K on economic grounds, the only way out is to set
A1 = 0. Coupling this information with (5.20), we then have A 2 = K0 - K,
so that the optimal K path is
( 5.21)
K * ( t ) = ( K0 - K)er•t + K
A few things may be noted about this result. First, since r2 < 0, K*(t)
converges to the particular integral K; thus K indeed emerges as the
optimal terminal plant size Kx. Moreover, given that b and p are positive,
K = (a - bp)/ 2{3 is less than a/2{3. Thus, after explicitly taking into
account the adjustment cost, the firm indeed ends up with a plant size
��::.: '
-,( ;.. ,.
109
CHAPTER 5: INFINITE PLANNING HORIZON
±
.•.
------- t
.,·
FIGURE 5.2
smaller than the 7T-mruamizing one shown in Fig. 5.1a . Finally, the discrep­
ancy between the initial and terminal sizes, K0 - K, is subject to expo­
nential decay. The K*(t) path should therefore have the general shape
illustrated in Fig. 5.2.
The Transversality Conditions
Although we have been able to definitize the arbitrary constants without
using any transversality conditions, it would be interesting to see where the
transversality condition (5.5) would lead us. Using the information in (5.16),
we know that
( 5.22)
For the K and K' terms, we should use the K* in (5.18) and its derivative,
namely,
Plugging these into (5.22), we get the complicated result
(5.22')
F* - K*'FK*'
A1(a - 2 f3 K)e<r, -p)t + A1 2 ( ar/ - f3 ) e <2 r, -p ll
=
+ A 1 A 2 (2ar1 r2 - 2{3)e1'1 +r2-p)t + A 2(a - 2 f3 K )e <r. -p)t
+ A2 2 ( ar/ -
f3 ) e 12'•-plt + ( a K - f3 K2 )e - P1
The transversality condition is that (5.22') vanish as t -> oo. Since r2 is
negative, the last three terms cause no difficulty. But the exponential in the
third term reduces to e0 = 1, because r1 + r = p [see (5.18)], and that term
2
will not tend to zero. Worse, the first two terms are explosive since r1 > p .
In order to force the first three terms to go to zero, the only way is to set
·�
"'� " m• =�om v.-unom
1 10
.:
A 1 = 0. Thus, the transversality condition leads us to precisely what we
have concluded via economic reasoning.
As to transversality condition (5. 7), it is not really needed in this
problem. But if we try to apply it, it will also tell us to set A 1 = 0 in order to
tame an explosive exponential expression.
-
The Optimal Investment Path
and the Flexible Accelerator
If the adjustment-cost function C = C( K') fully takes into account the
various internal and external difficulties of adjustment, then the optimal
investment path is simply the derivative of the definite solution (5.21):
( 5 .23)
But since (5.21) implies that
( K0 - K)e'•1 = K*(t) - K
the investment path (5.23) can be simplified to
'
:·
(5.23')
The remarkable thing about (5.23') is that it exactly depicts the
flexible-accelerator mechanism in (5.12). The discrepancy between K, the
target plant size, and K*(t), the actual plant size on the extremal at any
point of time, is systematically eliminated through the positive coefficient
- r2 • The only unsettled matter is that, in the flexible accelerator, - r2 must
be less than one. This requirement is met if we impose the following
additional restriction on the model:
f3
- <
( 5 .24)
a
.
1 +p
What makes the flexible accelerator in (5.23') fundamentally different
from (5.12) is that the former is no longer a postulated behavior pattern,
but an optimization rule that emerges from the solution procedure. The
Eisner-Strotz model, in its quadratic version, has thus contributed a much
needed theoretical justification for the popularly used flexible-accelerator
mechanism.
EXERCISE 5.2
1
Let the profit function and the adjustment-cost function (5.14) and (5.15)
take the specific forms
1r =
50K - K 2
C
=
K'2
+
2K'
. . ·;
'. .
Ill
CHAPTER 5: INFINITE PLANNING HORIZON
but leave the discount rate p in the parametric form. Find the optimal path
of K (definite solution) by directly applying the Euler equation (2.19).
2
Use the solution formula (5.21) to check your answer to the preceding
problem.
3
Find the Euler equation for the general formulation of the Eisner-Strotz
model as given in (5.13).
7T
50K - K2 as in Prob. 1, hut change the
2
adjustment-cost function to C = K' • Use the general Euler equation you
derived in the preceding problem for (5.13) to get the optimal K path.
4 Let the profit function be
5
5.3
=
Demonstrate that the parameter restriction (5.24) is necessary to make
- r2 a positive fraction.
THE OPTIMAL SOCIAL SAVING
BEHAVIOR
Among the very first applications of the calculus of variations to economics
is the classic paper by Frank Ramsey on the optimal social saving behavior.12
This paper has exerted an enormous if delayed influence on the current
literature on optimal economic growth, and is well worth a careful review.
The Ramsey Model
The central question addressed by Ramsey is that of intertemporal resource
allocation: How much of the national output at any point of time should be
for current consumption to yield current utility, and how much should be
saved (and invested) so as to enhance future production and consumption,
and hence yield future utility?
The output is assumed to be produced with two inputs, capital K and
labor L. The production function, Q = Q(K,L), is time invariant, since no
technological progress is assumed. Other simplifying assumptions include
the absence of depreciation for capital and a stationary population. How­
ever, the amount of labor services rendered can still vary. The output can
either be consumed or saved, but what is saved always results in investment
and capital accumulation. Then, using the standard symbols, we have
Q = C + S = C + K', or
·
(5.25)
C = Q( K, L ) - K'
Consumption contributes to social welfare via a social utility (index)
function U(C ), with nonincreasing marginal utility, U"(C) :5 0. This speci12Frank P. Ramsey, "A Mathematical Theory of Saving," Economic Journal, December 1928,
pp. 543-559.
1 12
PART 2, THE CALCULUS OF VARIATIONS
u
D�------- 1 U<Cl
0
u
0
c
"
c
"
c
(a)
c
(b)
u
'·'
(c)
FIGURE 5.3
fication is consistent with all three utility functions illustrated in Fig. 5.3.
In every case, there is an upper bound for utility, 0. That level of utility is
attainable at a finite consumption level 6 in Figs. 5.3a and b, but can only
be approached asymptotically in Fig. 5.3c.
In order to produce its consumption goods, society incurs disutility of
labor D(L), with nondecreasing marginal disutility, D''(L) � 0. The net
social utility is therefore
D(L), where and L-as K and Q-are
functions of time. The economic planner's problem is to maximize the social
utility for the current generation as well as for all generations to come:
U{C) -
( 5 .26)
Maximize
U
C
{! U(C) - D( L)] dt
0
C,
C
In the integrand, depends solely on
and, by (5.25), depends on K, L,
and K', whereas D depends only on L. Thus, this problem has two state
variables, K and L. Because there is no L' term in the integrand, however,
the problem is degenerate on the L side, and in general it would be
inappropriate to stipulate an initial condition on L.
CHAPTER s, INFINITE PLANNING HORIZON
113
The Question of Convergence
Unlike the functional of the Eisner-Strotz model, the improper integral in
(5.26) does not contain a discount factor. This omission is not the result of
neglect; it stems from Ramsey's view that it is " ethically indefensible" for
the (current-generation) planner to discount the utility of future genera­
tions. While this may be commendable on moral grounds, the absence of a
discount factor unfortunately forfeits the opportunity to take advantage of
Condition III of Sec. 5.1 to establish convergence, even if the integrand has
an upper bound. In fact, since the net utility is expected to be positive as t
becomes infinite, the integral is likely to diverge.
To handle this difficulty, Ramsey replaces (5.26) with the following
substitute problem:
( 5 .26' )
·
Minimize
(0 [ B - U( C ) + D( L ) ] dt
subject to
K(O)
=
K0
( K0 given)
where B (for Bliss) is a postulated maximum attainable level of net utility.
Since the new functional measures the amount by which the net utility
U(C) - D(L ) falls short of Bliss, it is to be minimized rather than maxi­
mized. Intuitively, an optimal allocation plan should either take society to
Bliss, or lead it to approach Bliss asymptotically. If so, the integrand in
(5.26') will fall steadily to the zero level, or approach zero as t -+ co. As far as
Ramsey is concerned, the convergence problem is thereby resolved. The
substitution of (5.26) by (5.26'), referred to as " the Ramsey device," is
widely accepted as sufficient for convergence. Our earlier discussion of
Condition I in Sec. 5.1 would confirm that if the integrand attains zero at
some finite time and remains at zero thenceforth, then convergence indeed
is ensured. But, as underscored in Condition II, the mere fact that the
integrand tends to zero as t becomes infinite does not in and of itself
guarantee convergence. The integrand must also fall sufficiently fast over
time.
Even though the convergence problem is not clearly resolved by
Ramsey, we shall now proceed on the assumption that the general-function
integrand in (5.26') does converge.
The Solution of the Model
The problem as stated in (5.26') is an autonomous problem in the state
variables K and L . From the integrand
F = B - U( C ) + D( L )
where
C
=
Q( K , L ) - K'
PART 2: THE CALCULUS OF VARIATIONS
114
we can get, on the L side, the derivatives
FL = - U'(C)
ac
+ D' ( L ) = -J.LQL + D' ( L )
aL
[J.L
=
U'(C)]
Fu = 0
For notational simplicity, we are using J.L to denote marginal utility. (The
abbreviation of marginal utility is " mu.") Like U(C), J.L is a function of C
and hence indirectly a function of K, L, and K'. On the K side, we can
obtain the derivatives
ac
FK = - U'(C)
=
Q
aK - J.L K
ac
= - U' ( C ) ( - 1) = U' ( C ) = J.L
FK. = - U'(C)
aK'
These derivatives will enable us to apply the Euler equations (2.27).
We note first that since Fu = 0, the Euler equation for the L variable,
FL - dFv!dt 0, reduces simply to FL = 0, or
=
for all t
(5.27)
�
0
The marginal disutility of labor must, at each point of time, be equated to
the product of the marginal utility of consumption and the marginal prod­
uct of labor. On the K side, the Euler equation FK - dFK.jdt = 0 gives us
the condition - J.LQK - d}Ljdt = 0, or
(5.28)
d}Ljdt
-- = - QK
J.L
for all t
�
0
This result prescribes a rule about consumption: J.L, the marginal utility of
consumption, must at every point of time have a growth rate· equal to the
negative of the marginal product of capital. Following this rule, we can map
out the optimal path for J.L . Once the optimal J.L path is found, (5.27) can be
used to determine an optimal L path. As mentioned earlier, however, the
problem is degenerate in L, so it is not appropriate to preset an initial
level L(O).
The Optimal Investment and Capital
Paths
Of greater interest to us is the optimal K path and the related investment
(and savings) path K ' . Instead of deducing these from the previous results,
let us find this information by taking advantage of the fact that the present
problem falls under Special Case II of Sec. 2.2. Without t as an explicit
argument in the integrand function, the Euler equation for K is, according
. . � ·.- '
;.. ! ;
�•.
115
CHAPTER 5: INFINITE PLANNING HORIZON
to
(2.21),
(5.29)
F - K'FK' = constant, or
B - U( C ) + D( L ) - K'J.L = constant
for all t
�
0
This equation can be solved for K' as soon as the (arbitrary) constant on the
right-hand side can be assigned a specific value. To find this value, we note
that this constant is to hold for all t, including t --+ oo. We can thus make
use of the fact that, as t -> oo , the economic objective of the model is to have
U(C) - D( L ) tend to Bliss. This means that U must tend to 0 and J.L must
tend to zero as t becomes infinite. It follows that the arbitrary constant in
(5.29) has to be zero. If so, the optimal path of K' is
B - U( C ) + D( L )
K*' = ------J.L
(5.30)
or, with the time argument explicitly written out,
K*' ( t)
(5.30' )
B - U ( C( t ) ] + D ( L ( t) ]
J.L( t)
= _____:_:.____:_:._.c..__
_
'---'----
This result i s known as the Ramsey rule. I t stipulates that, optimally, the
rate of capital accumulation must at any point of time be equal to the ratio
of the shortfall of net utility from Bliss to the marginal utility of consump­
tion. On the surface, this rule is curiously independent of the production
function. And this led Ramsey to conclude (page 548) that the production
function would matter only insofar as it may affect the determination of
Bliss. But this conclusion is incorrect. According to (5.28), J.L is to be
optimally chosen by reference to QK. This means that the denominator of
(5.30') does depend crucially on the production function.
A Look at the Transversality Conditions
We shall now show that the use of the infinite-horizon transversality
condition would also dictate that the constant in (5.29) be set equal to zero.
When condition (5.5) is applied to the two state variables in the present
problem, it requires that
lim ( F - L'Fv ) = 0
t � oo
and
lim ( F - K'Fx, ) =
t - «>
0
I n view of the fact that Fv 0, the first of these conditions reduces t o the
condition that F -> 0 as t -> oo. This would mean the net utility U(C) ­
D(L) must tend to Bliss. Note that, by itself, this condition still leaves the
constant in (5.29) uncertain. However, the other condition will fix the
constant at zero because F - K'Fx' is nothing but the left-hand-side ex­
pression in (5.29).
=
116
PART 2: THE CALCULUS OF VARIATIONS
The present problem implicitly specifies the terminal state at Bliss.
Consequently, the transversality condition (5. 7) is not needed.
From the Ramsey rule, it is possible to go one step further to find the
K * (t) path by integrating (5.30'). For that, however, we need specific forms
of U(C) and D(L ) functions. The general solution of (5.30') will contain one
arbitrary constant, which can be definitized by the initial condition K(O) =
K0• And that would complete the solution of the model. We shall not
attempt to illustrate the model with specific functions; rather, we shall show
the application of phase-diagram analysis to it in the next section.
EXERCISE 5.3
1
Let the production function and the utility function in the Ramsey model
take the specific forms:
Q( K )
=
rK
1
U = u - - c ­b
b
•
( r > 0)
( b > 0)
with initial capital K0.
(a) Apply the condition (5.28) to derive a C*(t) path. Discuss the
characteristics of this path.
(b) Use the Ramsey rule (5.30') to find K*'(t). Discuss the characteristics
of this path, and compare it with C*(t). How specifically are the two
paths related to each other? [Hint: Since the D(L ) term is absent in
the problem, we have B = 0.]
(c) Bearing in mind that Q0 rK0 (production of Q0), and Q0 = C* 0 +
K*'(O) (allocation of Q0), and using the relationship between K*'(O)
and C*0 implied by the result in (b), find an expression for C*0 in
terms of K0• Write out the definite solution for K*'(t).
(d) Integrate K*'(t) to obtain the K*(t) path. What is the rate of growth
of K * ? How does that rate vary over time?
=
2
Find the K * (t) path for the preceding problem by the alternative procedure
outlined below:
(a) Express the integrand of the functional ( B U(C)] in terms of K and
K', that is, as F(K, K').
(b) Apply the Euler equation (2.18) or (2.19) to get a second-order
differential equation. Find the general solution K*(t), with arbitrary
constants A1 and A2. Then derive the optimal savings path K*'(t);
retain the arbitrary constants.
(c) Find the savings ratio K*'(t)jQ*(t) = K*'(t)jrK*(t). Show that this
ratio approaches one as t -> oo, unless one of the arbitrary constants is
zero.
-
117
CHAPTER 5: INFINITE PLANNING HORIZON
(d) I s a unitary limit for the savings ratio economically acceptable? Use
your answer to this question to definitize one of the constants. Then
definitize the other constant by the initial condition K(O) == K 0 .
Compare the resulting definite solution with the one obtained in the
preceding problem.
5.4
PHASE-DIAGRAM ANALYSIS
In models of dynamic optimization, two-variable phase diagrams are preva­
lently employed to obtain qualitative analytical resultsY This is particu­
larly true when general functions are used in the model, and the time
horizon is infinite. For a calculus-of-variations problem with a single state
variable, the Euler equation comes as a single second-order differential
equation. But it is usually a simple matter to convert that equation into a
system of two simultaneous first-order equations in two variables. The
two-variable phase diagram can then be applied to the problem in a straight­
forward manner. We shall illustrate the technique in this section with the
Eisner-Strotz model and the Ramsey model. We shall then adapt the phase
diagram to a finite-horizon context.
The Eisner-Strotz Model Once Again
In their model of a firm's optimal investment in its plant, discussed in
Sec. 5.2, Eisner and Strotz are able to offer a quantitative solution to the
problem when specific quadratic profit and cost-of-adjustment functions are
assumed. Writing these functions as
1r =
C
=
a K - {3K 2
aK' 2 + bK'
( a, {3 > 0)
( a , b > O)
[from (5.14)]
[from (5.15)]
they derive the Euler equation
(5.31)
where
(5.32)
{3
bp - a
K" - p K' - - K =
2a
a
---
p
[from ( 5 .17)]
is a positive rate of discount. From the quantitative solution,
K*(t) = ( K o - K)er•t + K
a - bp
where K =
2{3
[from (5.21)]
_
--
13The method of two-variable phase diagrams i s explained i n Alpha C . Chiang. Fundamental
Methods of Mathematical Economics, 3d ed., McGraw-Hill, New York, 1984, Sec. 18.5.
PART 2o THE CALCULUS OF VARIATIONS
118
we see that the capital stock
K
its specific target value
K
(the plant size) should optimally approach
by following a steady time path that exponen­
tially closes the gap between the initial value and the target value of K.
Now we shall show how the Euler equation can be analyzed by means
I
of a phase diagram. We begin by introducing a variable
(net investment)
(5.33)
I( t)
=
K'( t )
I' ( t )
[implying
=
K"( t) ]
(5.31) as a system
which would enable u s to rewrite the Euler equation
of
two first-order differential equations:
f3
bp - a
I' = pi + - K +
a
2a
(5.34)
[i.e.,
--
I' = f( I, K ) ]
[ i.e., K '
K' = I
=
g( I, K ) ] '
This is an especially simple system, since both the
linear in the two variables
function.
g
I
and
K,
::::::: : = 0
f
and
g
functions are
and in fact K is even absent in the
·
demarcation curves,
dO.w=. �' fi..,t oni"' of bu,;noM ;, to d,.w two
and K' =
Individually, each of these curves
0.
I' =
serves to delineate the subset of points in the
able in question can be stationary
(dljdt = 0
IK space where the vari­
dKjdt = 0). Jointly, the
or
two curves determine at their intersection the intertemporal equilib­
rium-steady state-of the entire system (dijdt = 0 and
Setting I' = {) in (5.34) and solving for K, we get
(5.35)
K =
a - bp
2 f3
--
Similarly, by setting K'
(5.36)
ap
- -I
f3
=
[equation for
I' =
dKjdt
=
0).
0 curve)
0, we get
I=0
[equation for K' =
0 curve)
These two equations both plot as straight lines in the phase space, as shown
in Fig. 5.4, with the I' =
curve sloping downward, and the K' =
curve
0
=
0
coinciding with the vertical axis. The unique intertemporal equilibrium
occurs at the vertical intercept of the I'
0 curve, which, by (5.35), is
a positive magnitude by (5.19). Note that this equilib­
rium coincides exactly with the particular integral
(target plant size)
K = (a
-
found in
bp)/2/3,
(5.18). This is of course only to be expected.
K
By definition, all points on the I' = 0 curve are characterized by
stationarity in the variable I. We have therefore drawn a couple of vertical
" sketching bars" on the curve, to indicate that there should not be any
east-west movement for points on the curve. Similarly, horizontal " sketch. ... ! l_: '
;
1 19
CHAPTER 5: INFINITE PLANNING HORIZON
K
-
K' = 0
+
L
. .
� _,.
·.\.
:.·;.;!
,;·.·
FIGURE 6.4
ing bars" have been attached to the K' = 0 curve to indicate that the
stationarity of K forbids any north-south movement on the curve.
Points off the demarcation curves are, on the other hand, very much
involved in dynamic motion. The direction of movement depends on the
signs of the derivatives I' and K' at a particular point in the IK space; the
speed of movement depends on the magnitudes of those derivatives. From
(5.34), we find by differentiation that
(5.37)
ar
= >O
aI p
and
aK'
iJ]
-=1>0
The positive sign of ol'jiJl implies that, moving from west to east (with I
increasing), I' should be going from a negative region, through zero, to a
positive region (with I' increasing). To record this ( - , 0, + ) sign sequence,
a minus sign has been added to the left of the I' = 0 label, and a plus sign
has been added to the right of the I' = 0 label. Accordingly, we have
also drawn leftward /-arrowheads ( I' < 0) to the left, and rightward
/-arrowheads (I' > 0) to the right, of the I' = 0 curve. For K', the positive
sign of oK'!oi similarly implies that, going from west to east, K' should be
moving from a negative region, through zero, to a positive region. Hence,
the sign sequence is again ( - , 0, + ), and this explains the minus sign to the
left, and the plus sign to the right, of the K' = 0 label. The K-arrowheads
are downward (K' < 0) to the left, and upward (K' > 0) to the right, of the
K' = 0 curve.
120
PART 2o THE CALCULUS OF VARIATIONS
Following the directional restrictions imposed by the IK-arrowheads
and sketching bars, we can draw a family of streamlines, or trajectories, to
portray the dynamics of the system from any conceivable initial point. Every
point should be on some streamline, and there should be an infinite number
of streamlines. But usually we only draw a few. The reader should verify in
Fig. 5.4 that the streamlint'fs conform to the IK-arrowheads, and, in particu­
lar, that they are consistent with the sketching bars which require them to
cross the I' = 0 curve with infinite slope and cross the K' = 0 curve with
zero slope.
The Saddle-Point Equilibrium
The intertemporal equilibrium of the system occurs at point E, the intersec­
tion of the two demarcation curves. The way the streamlines are structured,
we see two stable branches leading toward E (one from the southeast
direction and the other from the northwest), two unstable branches leading
away from E (one toward the northeast and the other toward the south­
west), and the other streamlines heading toward E at first but then turning
away from it. As a result, we have a so-called saddle-point equilibrium .
It is clear that the only way to reach the target level of capital at E is
to get onto the stable branches. Given the initial capital K0, for instance, it
is mandatory that we select I* 0 as the initial rate of investment, for only
that choice will place us on the " yellow brick road" to the equilibrium. A
different initial capital will, of course, call for a different I* 0• The restriction
on the choice of I* 0 is thus the surrogate for a transversality condition. And
this serves to underscore the fact that the transversality condition consti­
tutes an integral part of the optimization rule. While all the streamlines,
being based on (5.34), satisfy the Euler equation, only the one that also
satisfies the transversality condition or its equivalent-one that lies along a
stable branch-qualifies as an optimal path.
Traveling toward the northwest on the stable branch in Fig. 5.4, we
find a steady climb in the level of K, implying a steady n·arrowing of the
discrepancy between K0 and the target capital level. This is precisely what
(5.32) tells us. Accompanying the increase in K is a steady decline in the
rate of investment I. Again, this is consistent with the earlier quantitative
solution, for by differentiating (5.32) twice with respect to t, we do find
I*'(t) = K*"( t ) = ri ( Ko - K ) erzt < 0
since, in the present example, K0
<
K.
A Simplified Ramsey Model
To simplify the analysis of the Ramsey model, we shall assume that the
labor input is constant, thereby reducing the production function to Q(K).
CHAPTER 5: INFINITE PLANNING HORIZON
121
Then we will have only one state variable and one Euler equation with
which to contend. Another consequence of omitting L and D(L) is to make
Bliss identical with the upper bound of the utility function, 0.
The Euler equation for the original Ramsey model can be written as
[ from (5 .28) ]
(5.38)
or, making explicit the dependence of the marginal product on
K,
(5 .38' )
This is a first-order differential equation in the variable J.l.. However, since
the variable K also appears in the equation, we need a differential equation
in the variable K to close the dynamic system. This need is satisfied by
(5.25), C = Q(K, L) - K', which, when adapted to the present simplified
context, becomes
K' = Q( K ) - C(tL)
(5 .39)
The replacement o f Q(K, L ) by Q(K ) i s due to the elimination o f L. The
substitution of C by the function C(J.l.) reflects the fact that in the usual
marginal-utility function, 1-L is a monotonically decreasing function of C, so
that C can be taken as an inverse function of marginal utility 1-L ·
Combining (5.38') and (5.39), we get a two-equation system
K'
( 5 . 40 )
Ji
=
=
Q( K ) - C ( tL )
[ i.e., K ' = {( K, tL)]
- tL Q' ( K )
[ i.e., jl
=
g ( K, tL) ]
containing two general functions, Q(K) and C(J.l.).
Before we proceed, certain qualitative restrictions should be placed on
the shapes of these functions. For the time being, let us assume that the
U(C) function is shaped as in the top diagram of Fig. 5.5, with B as the
Bliss level of utility and CB the Bliss level of consumption. There exists
cons u mption saturation at CB, and further consumption beyond that level
entails a reduction in U. From this function, we can derive the marginal­
utility function J.l.(C) in the diagram directly beneath, with 1-L = 0 at C
CB.
Since J.l.(C) is monotonic, the inverse function C(J.l.) exists, with C'(J.l.) < 0.
The diagram in the lower-left corner, on the other hand, sets forth the
assumed shape of Q(K ), the other general function, here plotted-for a
good reason-with Q on the horizontal rather than the vertical axis. The
underlying assumption for this curve is that the marginal product of capital
Q'(K ) is positive throughout-there is no capital saturation -but the law
of diminishing returns prevails throughout.
=
122
PART 2: THE CALCULUS OF VARIATIONS
u
B
Utility
function
0 •_
_
_
_
_
_
_
_
_.___
Ca
I
'II " U'(C)
Marginalutility
function
I
1
\
c
,. ·
p = U'(C)
Phase
1
apace-
I
I �------- - - - - - -r
1
I
I
I
I
I
I
K
I
I Production
I
function
I ------�-T
45° line
1
I
I
I
FIGURE 5.5
The Phase Diagram
To construct the phase diagram, we need the K' = 0 and ,_,:
From (5.40), the following two equations readily emerge:
( 5 .41 )
( 5 .42)
Q( K ) = C ( �J- )
11- = 0
-
0 curves.
[equation for K' = 0 curve]
[equation for It'
=
0 curve]
The first of these is self-explanatory. The second results from our assump­
tion of no capital saturation: Since Q'(K) * 0, we can have It' = 0 if and
only if 11- = 0.
:�-l,
123
CHAPTER 5: INFINITE PLANNING HORIZON
To find the proper shape of the K'
0 curve, we equate Q(K) and
C(J-L) in accordance with (5.41). It is in order to facilitate this equating
process that we have chosen to plot the Q(K) function with the Q variable
on the horizontal axis in Fig. 5.5, for then we can equate Q and C
=
graphically by making a vertical movement across the two diagrams. Con­
sider, for instance, the case of J-L = 0, that is, the point C8 on the J-L(C)
curve. To satisfy (5.41), we simply select point G on the Q(K) curve, the
point that lies directly below point C8. The vertical height of point G may
be interpreted as the Bliss level of K; thus we denote it by K8. The ordered
pair (K = K8, J-L = 0) then clearly qualifies as a point on the K'
0 curve
in the phase space. Other ordered pairs that qualify as points on the K' = 0
curve can be derived similarly.
The easiest way to trace out the entire K'
0 curve is by means of the
rectangularly aligned set of our diagrams in the lower part of Fig. 5.5. Aside
from the J-L(C) and Q(K) diagrams, we include in this diagram set a 45° line
diagram (with K on both axes) and a phase-space diagram which is to serve
as the repository of the K'
0 curve being derived. These four diagrams
are aligned in a way such that each pair of adjacent diagrams in the same
" row" has a common vertical axis, and each pair of adjacent diagrams in
the same " column" has a common horizontal axis-except that, in the left
column, the C axis and the Q axis are not inherently the same, but rather
to be made the same via a deliberate equating process. Now the point
( K = K8, J-L = 0) can be traced out via the dashed rectangle, C8GHK8,
spanned by the J-L(C) curve, the Q(K) curve, and the 45° line. Indeed,
starting from any selected point on the J-L(C) curve, if we go straight down
to meet the Q(K ) curve [thereby satisfying the condition (5.41)], turn right
to meet the 45° line, and then go straight up into the phase space (thereby
translating the vertical height of point G, or K8, into an equal horizontal
distance in the phase space) to complete the formation of a rectangle, then
the fourth corner of this rectangle must be a point on the K' = 0 curve.
Such a process of rectangle construction yields a K'
0 curve that slopes
downward and cuts the horizontal axis of the phase space at K K 8•
=
=
=
=
=
The Saddle-Point Equilibrium
We now reproduce the K' = 0 curve in Fig. 5.6, and add thereto the Ji = 0
curve, which, according to (5.42), should coincide with the K axis. At the
intersection of these two curves is the unique equilibrium point E, which is
characterized by K = K8 and J-L = 0, and which therefore corresponds to
Bliss.
0 curve are vertical. Those for the
The sketching bars for the K'
Ji = 0 curve are, however, horizontal, and they lie entirely along that curve.
Since the sketching bars coincide with the Ji = 0 curve, the latter will serve
not only as a demarcation curve, but also as the locus of some streamlines.
=
124
PART 2: THE CALCULUS OF VARIATIONS
K' = O
SU!.ble branch
._
r
:--:>
··.,:; ··
!
-� '•
, ::- .
!J' = 0
0
K
Stable branch
FIGURE 5.6
The sign sequences for K'
derivatives
(5.43)
aK'
aK
- =
Q'( K ) > 0
and
and
Ji can be uceriained from the partial
ap:
aJ.L
- =
- Q'( K ) < 0
[ from (5.40)]
These yield the ( , 0, + ) sequence rightward for K', and the ( + , 0, )
sequence upward for Jl. Consequently, the K-arrowheads point eastward in
the area to the right of the K' = 0 curve, and they point westward in the
area to its left, whereas the J.L-arrowheads point southward in the area
above the Jl = 0 curve, and they point northward in the area below it.
The resulting streamlines again produce a saddle point. Given any
initial capital K0, it is thus necessary for us to select an initial marginal
utility that will place us on one of the stable branches of the saddle point. A
specific illustration, with K0 < KB , is shown in Fig. 5.6, where there is a
unique J.Lo value, J.L* 0, that can land us on the " yellow brick road" leading to
E. All the other streamlines will result ultimately in either (1) excessive
capital accumulation and failure to attain Bliss, or (2) continual decumula­
tion ("eating up") of capital and, again, failure to reach Bliss. As in the
Eisner-Strotz model, we may interpret the strict requirement on the proper
selection of an initial J.L value as tantamount to the imposition of a transver­
sality condition.
The fact that both the Eisner-Strotz model and the Ramsey model
present us with the saddle-point type of equilibrium is not a mere coinci­
dence. The requirement to place ourselves onto a stable branch merely
-
-
125
CHAPTER 5: INFINITE PLANNING HORIZON
implies that there exists an optimization rule to be followed, much as a
profit-maximizing firm must, in the simple static optimization framework,
select its output level according to the rule MC MR. If the equilibrium
were a stable node or a stable focus-the "all roads lead to Rome" situation
-there would be no specific rule imposed, which is hardly characteristic of
an optimization problem. On the other hand, if the equilibrium were an
unstable node or focus, there would be no way to arrive at any target level
of the state variable at all. This, again, would hardly be a likely case in a
meaningful optimization context. In contrast, the saddle-point equilibrium,
with a target that is attainable, but attainable only under a specific rule, fits
comfortably into the general framework of an optimization problem.
=
Capital Saturation Versus Consumption
Saturation
The preceding discussion is based on the assumption of consumption satu­
ration at C8, with U(C8) B and J.L(C8) = 0. Even though capital satura­
tion does not exist and Q'(K ) remains positive throughout, the economy
would nevertheless refrain from accumulating capital beyond the level K8.
As a variant of that model, we may also analyze the case in which capital,
instead of consumption, is saturated.
Specifically, let Q(K) be a strictly concave curve, with Q(O) 0, and
slope Q'(K ) z 0 as K 'S K. Then we have capital saturation at K. At the
same time, let U(C) be an increasing function throughout. Under these new
assumptions, it becomes necessary to redraw all the curves in Fig. 5.5 except
the 45° line. Consequently, the K' 0 curve in the phase space may be
expected to acquire an altogether different shape. However, it is actually
also necessary to replace the horizontal j.L = 0 curve, because with con­
sumption saturation assumed away and with J.L > 0 throughout, the hori­
zontal axis in the diagram is now strictly off limits. In short, there will be an
entirely new phase diagram to analyze.
Instead of presenting the detailed analysis here, we shall leave to the
reader the task of deriving the new K' 0 and j.L 0 curves and pursuing
the subsequent steps of determining the KJ.L-arrowheads and the stream­
lines. However, some of the major features of the new phase diagram may
be mentioned here without spoiling the reader's fun: (1) In the new phase
diagram, the K' 0 curve will appear as a U-shaped curve, and the j.L = 0
curve will appear as a vertical line. (2) The new equilibrium point will occur
at K K, and the marginal utility corresponding to K (call it p.) will now
be positive rather than zero. (3) The equilibrium value of the marginal
utility, P, , implies a corresponding equilibrium consumption C which, in
turn, is associated with a corresponding utility level 0. Even though the
latter is not the highest conceivable level of utility, it may nevertheless be
regarded as " Bliss" in the present context, for, in view of capital saturation,
we will never wish to venture beyond 0. The magnitude of this adapted
=
=
=
=
=
=
=
126
PART 2, THE CALCULUS OF VARIATIONS
Bliss is not inherently fixed, but is dependent on the location of the
capital-saturation point on the production function.
rium is again a saddle point.
(4) Finally, the equilib­
Yet another variant of the model arises when the U( C) curve is
replaced by a concave curve asymptotic to B rather than with a peak at B ,
as i n Fig. 5.3c. In this new case, the J.L curve will become a convex curve
asymptotic to the C axis. What will happen to the K' = 0 curve now
depends on whether the Q(K) curve is characterized by capital saturation.
If it is, the present variant will simply reduce to the preceding variant. If
not, and if the Q(K) curve is positively sloped throughout, the K' =
0 curve
will turn out to be a downward-sloping convex curve asymptotic to the K
axis in the phase space. The other curve,
because, according to
(5.40), p!
p!
=
0,
is now impossible to draw
cannot attain a zero value unless either
J.L = 0 or Q' ( K ) = 0, but neither of these two alternatives is available to us.
Consequently, we cannot define an intertemporal equilibrium in this case.
Finite Planning Horizon and Turnpike
Behavior
The previous variants of the Ramsey model all envisage an infinite planning
horizon. But what if the planning period is finite, say,
100
years or
250
years? Can the phase-diagram technique still be used? The answer is yes. In
particular, phase-diagram analysis is very useful in explaining the so-called
" turnpike" behavior of optimal time paths when
long, but finite, planning
horizons are considered.14
In the finite-horizon context, we need to have not only a given initial
capital K 0 as in the infinite-horizon case, but also a prespecified terminal
capital, Kr, to make the solution definite. The magnitude of Kr represents
the amount of capital we choose to bequeath to the generations that outlive
our planning period.
As
such, the choice of Kr is purely arbitrary. If we
avoid making such a choice and take the problem to be one with a truncated
vertical terminal line subject to Kr
� 0,
for K will inevitably dictate that Kr
however, the optimal solution path
= 0,
because the maximization of
utility over our finite planning period, but not beyond, requires that we use
up all the capital by time
but positive Kr. In Fig.
T. More reasonably, we should pick an arbitrary
5.7,
we assume that Kr > K0•
The specification of K0 and Kr is not sufficient to pinpoint a unique
optimal solution. In Fig.
5.7a, the streamlines labeled
81, . • • , 84 (as well as
others not drawn), all start with K0, and are all capable of taking us
14See Paul A. Samuelson, "A Catenary Turnpike Theorem Involving Consumption and the
Golden Rule," American Economic Review, June 1965, pp. 486-496.
;�
127
CHAPTER 5: INFINITE PLANNING HORIZON
/1
-
.
_ __ ,.,
·-··: >
• . -: .
·; - ··
FIGURE 5.7
eventually to the capital level KT. The missing consideration that can
pinpoint the solution is the length of the planning period, T.
Let the T1 curve be the locus of points that represent positions
attained on the various streamlines after exactly T1 years have elapsed
from t 0. Such a curve, referred to as an isochrone (equal-time curve),
displays a positive slope here for the following reason. Streamline 81, being
the farthest away from the K' = 0 curve and hence with the largest K'
value among the four, should result in the largest capital accumulation at
the end of T1 years (here, Kr), whereas streamline 84, at the opposite
extreme, should end up at time T1 with the least amount of capital (here,
=
:.-
.
PART 2, THE CALCULUS OF VARIATIONS
128
not much above K0). If the planning period happens to be T1 years,
streamline S1 will clearly constitute the best choice, because while all
streamlines satisfy the Euler equation, only S 1 satisfies additionally the
boundary conditions K(O) K0 and K(T1) = Kr, with the indicated jour­
ney completed in exactly T1 years.
Streamlines are phase paths, not time paths. Nevertheless, each
streamline does imply a specific time path for each of the variables K and
IL· The time paths for K corresponding to the four streamlines are illus­
trated in Fig. 5.7b, where, in order to align the K axis with that of Fig. 5.7a,
we plot the time axis vertically down rather than horizontally to the right.
In deriving these time paths, we have also made use of other isochrones,
such as T2 and T3 (with a larger subscript indicating a longer time). Moving
down each streamline in Fig. 5.7a, we take note of the K levels attained at
various points of time (i.e., at the intersections with various isochrones).
When such information is plotted in Fig. 5.7b, the time paths P11
, P4
emerge, with Pi being the counterpart of streamline Si .
For the problem with finite planning horizon T1, the optimal time path
for K consists only of the segment K 0G on P1 . Time path P1 is relevant to
this problem, because S 1 is already known to be the relevant streamline; we
stop at point G on P1, because that is where we reach capital level Kr at
time T1. However, if the planning horizon is pushed outward to T2 , then we
should forsake streamline S1 in favor of S3. The corresponding optimal
time path for K will then be P3, or, more correctly, the K0J segment of it.
Other values of T can also be analyzed in an analogous manner. In each
case, a different value of T will yield a different optimal time path for K.
And, collectively, all the finite-horizon optimal time paths for K are differ­
ent from the infinite-horizon optimal K path P2-the Ramsey path-asso­
ciated with streamline s2, the stable branch of the saddle point.
With this background, we can discuss the turnpike behavior of
finite-horizon optimal paths. First, imagine a family traveling by car be­
tween two cities. If the distance involved is not too long, it may be the
simplest to take some small, but direct, highway for the trip. But if the
distance is sufficiently long, then it may be advantageous to take a more
out-of-the-way route to get onto an expressway, or turnpike, and stay on it
for as long as possible, till an appropriate exit is reached close to the
destination city. It turns out that similar behavior characterizes the
finite-horizon optimal time paths in Fig. 5.7b.
Having specified K0 as the point of departure and Kr as the destina­
tion level of capital, if we successively extend out the planning horizon
sufficiently far into the future, then the optimal time path can be made to
arch, to any arbitrary extent, toward the Ramsey path P2 or toward the KB
line-the "turnpike" in that diagram. To see this more clearly, let us
compare points M and N. If the planning horizon is T4, the optimal path is
segment K0M on P4, which takes us to capital level Kr (point M) at time
T4• If the horizon is extended to T5, however, we ought to select segment
=
• • •
129
CHAPTER 5: INFINITE PLANNING HORIZON
K0 N on P3 instead, which takes us to capital level Kr (point N) at time
T5• The fact that this latter path, with a longer planning period, arches
more toward the Ramsey path and the K8 line serves to demonstrate the
turnpike behavior in the finite-horizon model.
The reader will note that the turnpike analogy is by no means a perfect
one. In the case of auto travel, the person actually drives on the turnpike.
In the finite-horizon problem of capital accumulation, on the other .hand,
the turnpike is only a standard for comparison, but is not meant to be
physically reached. Nevertheless, the graphic nature of the analogy makes it
intuitively very appealing.
EXERCISE 5.4
1
·
2
3
In the Eisner-Strotz model, describe the economic consequences of not
adhering to the stable branch of the saddle point:
(a) With K0 as given in Fig. 5.4, what will happen if the firm chooses .an
initial rate of investment greater than I * 0?
(b) What if it chooses an initial investment less than I* 0?
In the Ramsey model with an infinite planning horizon, let there be capital
saturation at K, but no consumption saturation.
(a) Would this change of assumption require any modification of the
system (5.40)? Why?
( b ) Should (5.41) and (5.42) be modified? Why? How?
(c) Redraw Fig. 5.5 and derive an appropriate new K' = 0 curve.
(d) In the phase space, also add an appropriate new Ji = 0 curve.
On the basis of the new K' = 0 and Ji = 0 curves derived in the preceding
problem, analyze the phase diagram of the capital-saturation case. In what
way(s) is the new equilibrium similar to, and different from, the
consumption-saturation equilibrium in Fig. 5.6? [ Hint: The partial
derivatives in (5.43) are not the only ones that can be taken.]
4 Show that when depreciation of capital is considered, the net product can
have capital saturation even if the gross product Q( K ) does not.
5
In the finite-horizon problem of Fig. 5.7a, retain the original K0, but let
capital Kr be located to the right of K8.
(a) Would the streamlines lying below the stable branch remain relevant
to the problem? Those lying above? Why?
(b) Draw a new phase diagram similar to Fig. 5.7a , with four streamlines
as follows: S1 (the stable branch), S2, S3, and S4 (three successive
relevant streamlines, with S2 being the closest to S1). Using
isochrones, derive time paths P1,
, P4 for the K variable,
corresponding to the four streamlines.
(c) Is turnpike behavior again discernible?
. • •
130
PART 2: THE CALCULUS OF VARIATIONS
5.5 THE CONCAVITY / CONVEXITY
SUFFICIENT CONDITION AGAIN
F(t ,
It was earlier shown in Sec. 4.2 that if the integrand function
y, y') in a
fixed-endpoint problem is concave (convex) in the variables (y, y'), then the
Euler equation is sufficient for an absolute maximum (minimum) of V[y ].
Moreover, this sufficiency condition remains applicable when the terminal
time is fixed but the terminal state is variable, provided that the supplemen­
tary condition
is satisfied. For the infinite-horizon case, this supplementary condition
becomes15
(5.44)
.
Fy'
!
is to be evaluated along the optimal path, and (jt y!'_)
In this condition,
represents the deviation of any admissible neighboring path y(t) from the
optimal path y*(t).
-
Application to the Eisner-Strotz Model
In the Eisner-Strotz model, the integrand function F(t, K, K') yields the
following second derivatives, as shown in (5.16):
and
where all the parameters are positive. Applying the sign-definiteness test in
(4.9), we thus have
FK'K
FKK
i
=
4'Cil.e -:,IP', > ·o.
.
'
..
" · '·
.•
I t follows that the quadratic form q associated \lrith these second derivatives
is negative definite, and the integrand functi(m F js strictly concave in the
variables (K, K').
·
15A more formal statement of this condition
can
·
be found in G. Hadley and M. C. Kemp,
Variational Methods in Economics, North-Holland, Amsterdam, 1971, p. 102, Theorem 2. 17. 5.
CHAPTER 5: INFINITE PLANNING HORIZON
form
131
For the present model, the supplementary condition (5.44) takes the
lim [ FK' ( K
(5 .45)
where the
(5 .46)
t -+ oo
-
;,
K* ) ] :S:: O
FK' term should specifically be
FK' - ( 2aK' + b ) e - P1
=
[ from (5.16) ]
Substituting the K*'(t) expression in (5.23) for the K' term in (5.46),
have
we
( 5 .46')
It is clear that FK' tends to zero as t becomes infinite. As for the ( K - K * )
component of (5.45), the assumed form of the quadratic profit function
depicted in Fig. 5.1a suggests that as t becomes infinite, the difference
between the K value on any admissible neighboring path and the K* value
is bounded. Thus, the vanishing of FK' assures us that the supplementary
condition (5.45) can be satisfied as an equality. Consequently, the strict
concavity of the integrand function would make the Euler equation suffi­
cient for a unique absolute maximum in the total profit II(K).
Application to the Ramsey Model
The integrand function of the (simplified) Ramsey model is
F( t , K, K' )
=
B - U( C )
=
B - U [ Q( K ) - K ']
We assume that U'(C) � 0, U"(C ) < 0, Q'(K) > 0, and Q"( K )
the following first derivatives of F:
FK =
- U'( C) Q'( K )
FK. =
and
the second derivatives are found to be
- U' ( C ) ( - 1 )
=
<
0. From
U'(C )
-
FK' K' U"( C ) ( - 1) = U"( C )
FKK' = FK'K U" ( C ) Q' ( K )
FKK - U"( C ) [ Q' ( K ) ] 2 - U'( C ) Q''( K )
=
=
=
Again applying the sign-definiteness test in (4.9), we find that,_, ,
ID t l
( 5.47)
But since
( 5 . 48 )
ID:al
!
F
= FFKKK'K' FK'K
' KK
=
I
=
FK'K' > 0
U"( C ) U'(C)Q"( K)
.
. . ..
� o r!_ :;.�·
132
PART 2' THE CALCULUS OF VARIATIONS
is not strictly positive, we cannot claim that the associated quadratic form q
is positive definite and that the F function is strictly convex. Nevertheless,
it is possible to establish the (nonstrict) convexity of the F function by the
sign-semidefiniteness test in (4.12). With reference to the principal minors
defined in (4.11), we find that
and
These, together with the information in
(5.47) and (5.48), imply that
and
Thus, by (4. 12), we can conclude that the integrand function F is convex.
Turning to the supplementary condition (5.44), we need to confirm
that
( 5.49)
lim
t .... oo
[ Fd K - K*)]
�
0
Since Fx· = U'(C), it is easy to see that as t becomes infinite (as we move
toward Bliss and as the marginal utility steadily declines), Fx· tends to zero.
As for the ( K - K* ) term, it is unfortunately not possible to state generally
that the deviation of K(t) from K*(t) tends to zero, or is bounded, as
t ..... oo. Since the production function is assumed to have positive marginal
product throughout, the Q(K) curve extends upward indefinitely, and there
is no bound to the economically meaningful values of K.
In this connection, we see that the assumption of capital saturation
would help in the application of the sufficiency condition. If Q(K) contains
a saturation point, then (K - K*) will be bounded as t --> oo, and (5.49) can
be satisfied as an equality. In fact, when capital saturation is assumed,
condition (5.49) can be satisfied even if there is no consumption saturation.
With the capital-saturation point K now serving to define " Bliss," all
admissible paths must end at K. Thus the ( K - K*) component of (5.49)
must tend to zero as t ..... oo. Since Fx· U'(C) is bounded as we approach
Bliss, (5.49) is satisfied as an equality. Consequently, given the convexity of
the F function, the Euler equation is in this case sufficient for an absolute
minimum of the integral J;[ B - U(C)] dt.
=
CHAPTER
6
CONSTRAINED
PROBLEMS
In the previous discussion, constraints have already been encountered
several times, even though not mentioned by name. Whenever a problem
has a specified terminal curve or a truncated vertical or horizontal terminal
line, it imposes a constraint that the solution must satisfy. Such con­
straints, however, pertain to the endpoint. In the present chapter, we
concern ourselves with constraints that more generally prescribe the behav­
ior of the state variables.
As a simple example, the input variables K
and
L
may appear in a model as the state variables, but obviously their time paths
cannot be chosen independently of each other because
K
and
L
are linked
to each other by technological considerations via a production function.
Thus the model should make room for a production-function constraint. It
is also possible to have a constraint that ties one state variable to the time
derivative of another. For instance, a model may involve the expected rate of
inflation
7T
and the actual rate of inflation
expected rate,
7T,
p
as state variables. The
is subject to revision over time when, in light of experi­
ence, it proves to be off the mark either on the up side or on the down side.
The revision of the expected rate of inflation may
straint such as
d7T
- =j (p - 7T )
dt
as previously encountered i n
(2.41).
(0 < j
:$;
be
captured by a con­
1)
While such a constraint
substitution, be expunged from a model, it can also
be
can ,
through
retained and explic­
itly treated as a constraint. The present chapter explains how this is done.
133
134
6.1
PART 2: THE CALCULUS OF VAR!ATIONS
FOUR BASIC TYPES OF
CONSTRAINTS
Four basic types of constraints will be introduced in this section. As in static
optimization problems with constraints, the Lagrange-multiplier method
plays an important role in -the treatment of constrained dynamic optimization problems.
Equality Constraints
Let the problem be that of maximizing
(6.1)
subject to a set o f
m
independent but consistent constraints (m < n)
(6.2)
and appropriate boundary conditions. By the " independence" of the m
constraints, it is meant that there should exist a nonvanis,hing Jacobian
determinant of order m , such as
{6.3)
But, of course, any m of the Yj variables can be used in IJI , not necessarily
the first m of them. Note that, in this problem, the number of constraints,
m , is required to be strictly less than the number of state variables, n.
Otherwise, with (say) m = n, the equation system (6.2) would already
uniquely determine the y/t) paths, and there would remain no degree of
freedom for any optimizing choice. In view of this, this type of constrained
dynamic optimization problem ought to contain at least two state variables,
before a single constraint can meaningfully be accommodated.
In line with our knowledge of Lagrange multipliers in static optimiza­
tion, we now form a Lagrangian integrand function, 9", by augmenting the
original integrand F in (6.1) as follows:
=
m
F + L, A;(t) ( C; - g i )
i- 1
,. . ...
_·> .:·
CHAPTER s, CONSTRAINED PROBLEMS
135
Although the structure of this Lagrangian function appears to be identical
with that used in static optimization, there are two fundamental differences.
First, whereas in the static problem the Lagrange-multiplier term is added
to the original objective function, here the terms with Lagrange multipliers
A i are added to the integrand function F, not to the objective functional
J0TFdt. Second, in the present framework, the Lagrange multipliers A;
appear not as constants, but as functions of t. This is because each
constraint gi in (6.2) is supposed to be satisfied at every point of time in the
interval [0, T ), and to each value of t there may correspond a different value
of the Lagrange multiplier A; to be attached to the (c; g i ) expression. To
emphasize the fact that A i can vary with t, we write A;(t). The Lagrangian
integrand !F has as its arguments not only the usual t, y1 , and yj,
(j
1, . . , n), but also the multipliers A ; , (i = 1, . . . , m).
The replacement of F by !F in the objective functional gives us the
new functional
-
=
.
( 6 .5)
which we can maximize as if it is an un constrained problem. As long as all
of the constraints in (6.2) are satisfied, so that c; gi = 0 for all i , then the
value of !F will be identical with that of F, and the free extremum of the
functional "Y in (6.5) will accordingly be identical with the constrained
extremum of the original functional V.
It is a relatively simple matter to ensure the requisite satisfaction of all
the constraints. Just treat the Lagrange multipliers as additional state
variables, each to be subjected to an Euler equation, or the Eu ler-Lagrange
equation , as it is sometimes referred to in the present constrained frame­
work. To demonstrate this, let us first note that the Euler-Lagrange equa­
tions relating to the y1 variables are simply
-
(6.6)
d
!FYj - - :FY . = O
dt j
for all
t
E
[0, T ]
( j = l , . . , n ) (cf. ( 2 .27)]
.
As applied to the Lagrange multipliers, they are, similarly,
( 6 . 7}
.9;.,
-
d
0
dt 5";.,· =
for all
t
E
[0, T ]
( i = 1, . . . , m)
However, since !F is independent of any A.[, we have .9*;. '.- = 0 for every i,
that the m equations in (6. 7) reduce to
( 6.7')
5";.. ,
=
0
or
for all t
E
[ 0, T ]
so
(by ( 6 .64))
which coincide with the given constraints. Thus, by subjecting all the Ai
. · ·'··.
PART 2: THE CALCULUS OF VARIATIONS
136
variables to the Euler-Lagrange equation, we can guarantee the satisfaction
of all the constraints.
As
a matter of practical procedure for solving the present problem,
(1)
therefore, we can
apply the Euler-Lagrange equation to the
n
state
yi only, as in (6.6), (2) take the m constraints exactly as originally
given in (6 . 2 ) , and (3) use these n + m equations to determine the y/t) and
the A ;(t) paths. In step (1), the resulting n differential equations will
variables
contain
2n
arbitrary constants in their general solutions. These may be
definitized by the boundary conditions on the state variables.
EXAMPLE 1 Find the curve with the shortest distance between two given
=
points A = (0,
z0) and
= 0.
Y r, zr) lying on a surface
The distance between the given pair of points is measured bf. the
integral f[(1 + y'2 + z'2) 1 /2
which resembles the integral f (1 +
y'2) 112
for the distance between two points lying in a plane. Thus the
problem is to
y0,
B
dt
( T,
dt,
cf>(t, y, z )
0
Minimize
4>( t,y, z) = 0
y(O) = y0 , y(T)
subject to
and
=
Yr• z ( O )
=
z0 , z ( T )
This is a problem with two state variables (n
2) and
(m = 1).
As the first step, we form the Lagrangian integrand
=
.9'= F + A( t) (O
- 4> ) = F - >.( t )4>
= { 1 + y' 2 + z'2 )112 - A( t) cf>( t , y , z )
Partially differentiating .9', we
.9;"
=
see
that
- A( t)lf>z
These derivatives lead to the two Euler-Lagrange equations
- A( t) 4>y - :t [ y' ( 1 + y'z + z' 2) - I/2] = 0
- A{t ) 4>. - :t [ z' (l + y'2 + z'zfi/2]
=0
= Zr
one constraint
CHAPTER 6' CONSTRAINED PROBLEMS
137
which, together with the constraint
4>( t'
y'
z
) =0
give us three equations to determine the three optimal paths for y,
z,
and A .
Differential-Equation Constraints
Now suppose that the problem is to maximize (6. 1), subject to a consistent
set of m independent constraints ( m < n) that are differential equations:1
( 6 .8)
and appropriate boundary conditions. Even though the nature of the con­
straint equations has been changed, we can still follow essentially the same
procedure as before.
The Lagrangian integrand function is still
and the Euler-Lagrange equations with respect to the state variables
still in the form of
!F.
Yj
d
-
- !F_ ,
dt
Yj = 0
for all t
E
[0, T ]
(j =
l,
yJ
are
. . . , n)
Moreover, similar equations with respect to the Lagrange multipliers A; are
again nothing but a restatement of the given constraints. So we have a total
of n Euler-Lagrange equations for the state variables, plus the m con­
straints to determine the n + m paths, y; ( t ) and A;(t), with the arbitrary
constants to be definitized by the boundary conditions.
Inequality Constraints
When the constraints are characterized by inequalities, the problem can
1The independence among the m differential equations in (6.8) means that there should exist a
nonvanishing Jacobian determinant of order m, such as
Again, any m
of the Y/ can be used in the Jacobian.
138
PART 2: THE CALCULUS OF VARIATIONS
generally be stated as follows:
Maximize
f0TF(t, yl, . . . , yn , YI'•
•
. . , yn' ) dt
subject to
g1 ( t, y t• · · · • Yn , Yt', . . . , y:)
and
appropriate boundary conditions
(6.9)
:5
cl
Since inequality constraints are much less stringent then equality con­
straints, there is no need to stipulate that m < n . Even if the number of
constraints exceeds the number of state variables, the inequality constraints
as a group will not uniquely determine the Yj paths, and hence will not
eliminate all degrees of freedom from our choice problem. However, the
inequality constraints do have to be consistent with one another, as well as
with the other aspects of the problem.
To solve this problem, we may again write the Lagrangian integrand as
While the Euler-Lagrange equations for the Yj variables,
for all
t
E
[0, T ]
{j = l , . . . , n )
are no different from before, the corresponding equations with respect to
the Lagrange multipliers must be duly modified to reflect the inequality
nature of the constraints. To ensure that all the Ai(tX c i - gi) terms vanish
in the solution (so that the optimized values of .r and F are equal), we need
a complementary-slackness relationship between the i th multiplier and the
i th constraint, for every i (a set of m equations):
for all
t E [0, T ]
(i
=
l,
. . . , m)
This complementary-slackness relationship guarantees that (1) whenever
the i th Lagrange multiplier is nonzero, the i th constraint will be satisfied
as a strict equality, and (2) whenever the ith constraint is a strict inequal­
ity, the i th Lagrange multiplier will be zero. It is this relationship that
serves to maintain the identity between the optimal value of the original
integrand F and that of the modified integrand .r in (6.4).
"; ! '
. •
CHAPTER 6, CONSTRAINED PROBLEMS
139
Isoperimetric Problem
The final type of constraint to be
as exemplified by the equation
(6.11)
considered bends thtHn:tegral constraint,
·
fa ( t , y , y' ) dt = k
·
(k
0
=
constant)
One of the earliest problems involving such a constraint is that of finding
the geometric figure with the largest area that can be enclosed by a curve of
some specified length. Since all the figures admissible in the problem must
have the same perimeter, the problem is referred to as the isoperimetric
problem . In later usage, however, this name has been extended to all
problems that involve integral constraints, that is, to any problem of the
general form
( TF( t, yt,
Maximize
),
subject to
),
0
, Yn• Yt • · · · , yn ) dt
1
· · ·
1
( TG ( t, Yt> · · · • Yn, Yt • · · · • Yn ) dt - kl
1
0
(6. 12)
f
I
f0TGm( t, yl> . . . , yn , yl'• . . . , y: ) dt = k m
appropriate boundary conditions
and
In the isoperimetric problem, there is again no need to require m < n ,
because even with m � n, freedom of optimizing choice is not ruled out.
Two characteristics distinguish isoperimetric problems from other con­
strained problems. First, the constraint in (6. 11) is intended, not to restrict
the y value at every point of time, but to force the integral of some function
to attain a specific value. In some sense, therefore, the. constraint is more
indirect. The other characteristic is that the solution values of the Lagrange
multipliers A i (t) are all constants, so that they can simply be written as A ; .
G
To verify that the Lagrange multipliers indeed are constants in the
solution, let us consider the case of a single state variable
and a single
integral constraint
( 6 .13)
(m
=
n
=
1).
r ( t)
y
First, let us define a function
= f0 a c t , y, y') dt
T,
Note that the upper limit of integration here is not a specific value
but
the variable t itself. In other words, this integral is an indefinite integral
140
PART
and hence a function of t. At
(6.14)
f( O ) =
f0 o dt = o
t
=
2' THE CALCULUS OF VARIATIONS
0 and t = T, respectively, we have
and
r( T )
=
f0 o dt
=
k
(by (6.11}1
It is clear that f(t) measures the accumulation o f G from time 0 to time t.
For our immediate purpose, we note from (6.13) that the derivative of the
f(t) function is simply the G function. So we can write
G ( t , y , y' ) - f'( t )
(6.15)
= 0
which is seen to conform to the general structure of the differential con­
straint g(t, y, y') = c, with g = G - f' and c = 0. Thus we have in effect
transformed the given integral constraint into a differential-equation con­
straint. And we can then use the previously discussed approach of solution.
Accordingly, we may write the Lagrangian integrand
(6.16)
F = F( t, y , y' ) - + A ( t ) (O - G ( t , y, y' ) + f'( t ) ]
=
F( t , y , y') - A ( t ) G ( t , y , y' )
+
A ( t ) f' ( t )
(We denote this particular Lagrangian integrand by F rather than sr,
because it is merely an intermediate expression which will later be replaced
by a final Lagrangian integrand.) In contrast to the previously encountered
sr expressions, we note that F involves an extra variable r -although the
latter enters into F only in its derivative form f'(t). This extra variable,
whose status is the same as that of a state variable, must also be subjected
to an Euler-Lagrange equation. Thus, there arise two parallel conditions in
this problem:
( 6.17)
d Fy - t FY, = O
d
( 6.18)
Fr - t Fr' = O
d
-
d -
for all
t
for all
t E [0, T ]
E
[0, T ]
However, inasmuch as F is independent of r, and since Fr'
that (6.18) reduces to the condition
(6.18')
d
- - A( t )
dt
=
0
-
=
A(t),
we see
A(t) = constant
This then validates our earlier claim about the constancy of A. In view of
this, we may write the Lagrange multiplier of the isoperimetric problem
simply as A.
CHAPTER 6: CONSTRAINED PROBLEMS
Turning next to
141
(6. 17) and using (6.16), we can
obtain a more specific
version of that Euler-Lagrange equation:
( 6.19)
But a moment's reflection will tell us that this same condition could have
been obtained from a modified (abridged) version of F without the
term in it, namely,
f'(t)
!F= F( t,y , y' )
( 6 .20)
- AG ( t, y,y' )
( A = constant)
Thus, in the present one-state-variable problem with a single integral
constraint, we can as a practical procedure use the modified Lagrange
integrand .r in (6.20) instead of F, and then apply the Euler-Lagrange
equation to
alone, knowing that the Euler-Lagrange equation for A will
merely tell us that A is a constant, the value of which can be determined
from the isoperimetric constraint.
The curious reader may be wondering why the Lagrange-multiplier
y
term
AG enters in .r in (6.20) with a minus sign. The answer is that this
AC - Ag
follows directly from the standard way we write the Lagrangian integrand,
g) =
has a minus sign attached to A g. When
where the term A(c
written in this way, the Lagrange multiplier A would come out with a
positive sign in the solution, and can be assigned the economic interpreta­
tion of a shadow price.
The foregoing procedure can easily be generalized to the n-state­
variable, m-integral-constraint case. For the latter, the modified Lagrangian
function is
( A; are all constants)
Find a curve AB, passing though two given points A = (0,
and B = (T, Y r ) in the
plane, and having a given length k, that maxi­
mizes the area under the curve.
The problem, as illustrated in Fig. 6.1, is to
EXAMPLE 2
Maximize
subject to
and
For the problem to
y0)
ty
V=
f0Ty dt
1/2
fT (1 + y'2 ) dt = k
0
y( O) = Yo y( T ) = Yr
be the meaningful, the constant k shall be assumed
to
be larger then the length of line segment AB, say, L . (If k < L , the problem
has no solution, for the curve AB in question is impossible to draw. Also, if
PART 2: THE CALCULUS OF VARIATIONS
142
y
t
nG\JRE 6.1
L, then the curve AB can only be drawn as a straight line, and no
optimizing choice exists.)
To find the necessary condition for the extremal, we first write the
Lagrangian integrand a Ia (6.20):
k
=
(6.22)
(A
=
constant)
Since this does not explicitly contain t as an argument, we mq · take
advantage of formula (2.21), and write the Euler-Lagrange equatiort iD a
form that has already been partly solved, namely,
·
S"- y'�·
(6.23)
Since �·
( 6.23' )
=
= c1
·
( c1 = arbitrary constant)
- Ay'(1 + y'2 )-112, (6.23) can be expressed more s� M
- 1/2
1/2
y - A(1 + y' 2 ) + Ay'2 (1 + y'2 )
= c1
Via a series of elementary operations to be described, this equation
solved for y' . Transposing terms and simplifying, we can write
or
can
be
·
·'
-
�
$quaring both sides and subtracting 1 from both sides, we get
Az - (y - cl)2
y' 2 = --'-�i"­
(y - cl)2
;·
, '·
The squani root of this gives us the following expression of y' in tenns of A,
y, and
c1:
·
.·
·
:.
':
CHAPTER 6o
143
CONSTRAINED PROBLEMS
Since y' = dyjdt, however, the last result may alternatively be written as
y - ci
dy
--;====
=7
z
(6.23")
V>..2 - ( Y - c i )
dt
=
which the reader will recognize as a nonlinear differential equation with
separable variables, solvable by integrating each side in turn.
The integral of dt, the right-hand-side expression in (6.23"), is simply
t + constant. And the integral of the left-hand-side expression is
J.? - (y - c1)2 + constant.2 Thus, by equating the two integrals, con­
solidating the two constants of integration into a single symbol, -c2 ,
squaring both sides, and rearranging, we finally obtain the general solution
-V
( 6 .24)
Since this is the equation for a family of circles, with center (c2, c 1) and
radius >.. , the desired curve AB must be an arc of a circle. The specific
values of the constants c 1 , c2 , and >.. can be determined from the boundary
conditions and the constraint equation.
EXERCISE 6.1
I
Prepare a summary table for this section, with a column for each of the
following: (a) type of constraint, (b) mathematical form of the constraint,
(c) m < n or m � n? (d) A; varying or constant over time?
2
Work out all the steps leading from (6.23) to (6.23").
3
Find the extremal (general solution) of V =
4
5
6
j[y'2 dt, subject to J0Ty dt
=
k.
Find the y(t) and z(t) paths (general solutions) that give an extremum of
the functional
<y'2 + z'2 ) dt, subject to y - z' = 0.
With reference to Fig. 6.1, find a curve AB , with a given length and passing
through the given points A and B, that produces the largest area between
the curve AB and the line segment AB.
Jt
Find a curve AB with the shortest possible length that passes thro1,1gh two
given points A and B, and has the area under the curve equal to a given
constant K. [Note: This problem serves to illustrate the principle of
reciprocity in isoperimetric problems, which states that the problem of
maximum area enclosed by a curve of a given perimeter and the problem
of minimum perimeter for a curve enclosing that area are reciprocal to
2Let x ., y - c1, so that dx - dy. The integral of the left-hand-side expression of(6.23") is then
Jxf .;J.2. - x2 dx.
to standard tables of integrals (e. ., Formula 204 in CRC
Raton, FL, 1987), this integral is
equal to - .;A2 - x2 - - VA2 - (y - c1) 2 , plus constant of integration.
-
According
g
Standard Mathematical Tables, 28th ed., CRC Press,
a
Boca
144
PART 2, THE CALCULUS OF VARIATIONS
each other and share the same extremal.) [ Hint: When working out this
problem, it may prove convenient to denote the Lagrange multiplier by p.,
and later define A
1 /p., to facilitate comparison with Example 2 of this
section.)
=
6.2
SOME ECONOMIC APPLICATIONS
REFORMULATED
While the Lagrange-multiplier method can play a central role in constrained
problems, especially in problems with general functions, it is also possible,
and sometimes even simpler, to handle constrained problems by substitu­
tion and elimination of variables. Actually, most of the economic models
discussed previously contain constraints, and in the solution process we
have substituted out certain variables. In the present section, we shall
reconsider the Ramsey model and the inflation-unemployment tradeoff
model, to illustrate the method of Lagrange multipliers. The results would,
of course, be identical regardless of the method used.
The Ramsey Model
In the Ramsey model (Sec. 5.3), the integrand function in the objective
functional is
F = B - U( C ) + D( L )
where C = Q( K, L) - K'
Previously, the F function was considered to contain only two variables, K
and L, because we substituted out the variable C. When we took the
derivatives FK and FK'• in particular, the chain rule was used to yield the
results
The use of this rule reduces C to an intermediate variable that disappears
from the final scene.
To show how the Lagrange-multiplier method can be applied, let us
instead treat C as another variable on the same level as K and L. We then
recognize the constraint
g ( C , L , K, K' ) = Q( K, L ) - K' - C = 0
and reformulate the problem as one with an explicit constraint:
Minimize
( 6.25)
('lB -
U( C ) + D( L ) ] dt
subject to
Q( K, L ) - K' - C = O
and
boundary conditions
145
CHAPTER 6: CONSTRAINED PROBLEMS
By (6.4), we write the Lagrangian integrand function as
( 6 .26)
fF= B - U( C ) + D( L ) + A[ - Q( K , L) + K ' + C ]
Since we now have three variables (C, L, K ) and one Lagrange multiplier A
(not a constant), there should be altogether four Euler-Lagrange equations:
( 6.2 7)
( 6.28)
( 6.29)
( 6.30)
d
!J'C, = - U'(C) + A = - J.L + A = 0
=>
9(; - dt
d
.9'£ .9'£· = D'(L) - AQL = D'(L) - J.LQL = 0
dt
d
d
dJ.L
fFK - dt fFK' = - AQK - dt A = - J.L QK - dt = 0
d
fFx = - Q( K, L) + K' + C = 0
� - dt
A = J.L
[by ( 6.2 7 ) ]
[by (6.2 7 ) ]
Condition (6.2 7) tells u s that the Lagrange multiplier A i s equal to
J.L-the symbol we have adopted for the marginal utility of consumption.
Condition (6.28) is identical with (5.2 7). Similarly, (6.29) conveys the same
information as (5.28). Lastly, (6.30) merely restates the constraint. Thus the
use of the Lagrange-multiplier method produces precisely the same conclu­
sions as before.
The Inflation-Unemployment Tradeoff
Model
The inflation-unemployment tradeoff model of Sec. 2.5 will now be reformu­
lated as a problem with two constraints. As might be expected, the presence
of two constraints makes the solution process somewhat more involved.
For notational simplicity, we shall use the symbol y to denote Yr - Y
(the deviation of the current national income Y from its full-employment
level Y,). Then the integrand function F-the loss function (2.39)-can be
written as
(a >
0)
The expectations-augmented Phillips relation (2.40) and the adaptive-expec­
tations equation (2.41), which were previously used to substitute out y and
p in favor of the variable 1r, will now be accepted as two constraints:
g1( t, y, p , 7r, 7r1 ) = f3y + p - 7r =
g2( t, y, p , 7r , 7r1 ) = j( p -
7T ) - 7T'
0
=
[ from ( 2 .40)]
0
[from ( 2 .4 1 ) ]
The problem thus becomes one where one of the constraints involves
a
146
PART 2, THE CALCULUS OF VARIATIONS
simple equality, and the other constraint involves a differential equation:
jT(y 2 + ap 2 )e-pt dt
f3y + p - 'TT' = 0
j(p - Tr) - Tr' = O
Minimize
(6.3 1 )
"
·:..
0
subject to
;,:,· ·
boundary conditions
and
The Lagrangian integrand function is
(6.32 )
!F= (y2 + ap 2 ) e -pt + A l( -f3y - p + 'TT' ) + .\2( -jp + j'TT' + 7r' )
with three variables y, p, and and two (nonconstant' Lagrange multipli­
ers .\1 and A 2 . The Euler-Lagrange equations are
'TT'
d
( 6.33)
!F.
y
(6.34)
!F_ - - !F_, = 2ape -P1 - A - J·A = 0
dt
(6.35)
p
( 6 . 3 7)
.9:A t
:1
dt 5'.
d
d
�
· , '·
' ·:
( 6.36)
-
-
dt
d
= 2ye -pt - f3 A = 0
-�
1
1
p
�· = A t
+ j.\ 2 -
d
dt
sr,;, -
j
2
A2 = 0
-dt .9: . = - f3y - p + Tr = 0
d
·!
At
0
= p
dt sr.;, -j + j'TT' + 'TT'' =
'
. ·�·
·;
,..- ...
To solve these equations simultaneously requires quite a few steps.
One way of doing it is the following. First, solve (6.36) for y, substitute the
result into (6. 33), and solve for .\1 to get
2
A I = f3 2 (7r - p)e-pt
Next, solve (6.37) for the variable p:
(6.38)
� .. '· '·
.
1
p = 'TT' + -;'TT'
J '
( 6.39)
Substitution of (6.39) into (6. 38) results in th, ·'i�i!)aHan of
the AI expression:
··
(6.40 )
.
;
. .. �
'TT'
and
p in
CHAPTER 6: CONSTRAINED PROBLEMS
147
Then substituting both (6.39) and (6.40) into (6.34) and solving for A 2 , we
obtain (after simplification)
( 6.4 1)
This result implies the total derivative
(6.42)
dA2
dt
--
=
Finally, we can substitute the A1 , A2 , and dA2jdt expressions of the last
three equations into (6.35) to get a single summary statement of the five
simultaneous Euler-Lagrange equations. After combining and canceling
terms, the complicated expressions reduce to the surprisingly simple result
( 6.43)
This result qualifies as the condensed statement of conditions (6.33) through
(6.37) because all five equations have been used in the process and incorpo­
rated into (6.43).
Since (6.43) is identical with the earlier result (2.46), we have again
verified that the Lagrange-multiplier method leads to the same conclusion.
Note, however, that this time the Lagrange-multiplier method entails more
involved calculations. This shows that sometimes the more elementary
elimination-of-variable approach may work better. It is in situations where
the elimination of variables is unfeasible (e.g., with general functions) that
the Lagrange-multiplier method shows its powerfulness in the best light.
EXERCISE 6.2
1
The constraint in the Ramsey model (6.25) could have been equivalently
written as C - Q(K, L ) + K' = 0. Write the new Lagrangian integrand
function and the new Euler-Lagrange equations. How do the analytical
results differ from those in (6.27) through (6.30)?
... ..,_
� ·. :
148
PART
2
2: THE CALCULUS OF VARIATIONS
The Eisner-Strotz model of Sec. 5.2 can be reformulated as a constrained
problem:
Maximize
subject to
j�(
0
-rr -
C
and
-
-rr -
C)e-P' dt
2
o:K + {3K
2
aK' - bK'
=
=
0
0
boundary conditions
(a) How many variables are there in this problem? How many constraints?
(b) Write the Lagrangian integrand function.
(c) Write the Euler-Lagrange equations and solve them. Compare the
result with (5.17).
6.3 THE ECONOMICS OF
EXHAUSTIBLE RESOURCES
In the discussion of production functions, there is usually a presumption
that all the inputs are inexhaustible, so that as they are being used up, more
of them can be obtained for future use. In reality, however, certain re­
sources-such as oil and minerals-are subject to ultimate exhaustion. In
the absence of guaranteed success in the discovery of new deposits or
substitute resources, the prospect of eventual exhaustion must be taken
into account when such a resource is put into use. The optimal production
(extraction) of an exhaustible resource over time offers a good illustration of
the isoperimetric problem.
The Hotelling Model of Socially Optimal
Extraction
In a classic article by Harold Hotelling,3 the notion of " the social value" of
an exhaustible resource is used for judging the desirability of any extraction
pattern of the resource. The gross value to society of a marginal unit of
output or extraction of the resource is measured by the price society is
willing to pay to call forth that particular unit of output, and the net value
to society is the gross value less the cost of extracting that unit. If the price
of the resource, P, is negatively related to the quantity demanded, as
illustrated in Fig. 6.2, then the gross social value of an output Q0 is
3
Harold Hotelling, " The Economics of Exhaustible Resources," Journal of Political Economy,
April 1931, pp. 137-175.
-
�·
- '·
·•.
�:
149
CHAPTER 6: CONSTRAINED PROBLEMS
p
P(Q)
0
----- Q
FIGURE 6.2
measured by the shaded area under the curve, or the integral f(/0P(Q) dQ.
To find the net social value, we subtract from the gross social value the total
cost of extraction C (Q). Generalizing from a specific output level Q0 to any
output level Q, we may write the net social value of the resource, N, as:4
( 6.44)
N( Q ) = j QP( Q ) dQ - C( Q )
0
To avoid any confusion caused by using the same symbol Q as the variable
of integration as well as the upper limit of integration, we can alternatively
write ( 6 .44) as
(6.44')
N( Q ) = jQP(x)
dx - C( Q )
0
Assuming that the total stock of the exhaustible resource is given at S0, we
have the problem of finding an extraction path Q(t) so as to
Maximize
(6.45)
subject to
V=
{N
( Q ) e-pt dt
0
{'Qdt = S0
0
This, of course, is an isoperimetric problem. Since it is reasonable to expect
the net-social-value function to have an upper bound, the improper integral
in the objective functional should be convergent.
By (6.20), the Lagrangian integrand is
( 6.46)
!F= N( Q ) e-pt
- >.Q
(A = constant)
4
ln Hotelling's paper, the extraction cost is subsumed under the symbol P, which is inter­
preted as the " net price" of the exhaustible resource. Accordingly, the C(Q) term does not
appear in his treatment.
150
PART 2, THE CALCULUS OF VARIATIONS
Note that the integrand sr does not contain the derivative of Q; hence the
extremal to be obtained from the Euler-Lagrange equation may not fit given
fixed endpoints. If no rigid endpoints are imposed, however, we can still
apply the Euler-Lagrange equation, which, because SQ· = 0, reduces in the
present case to the condition .9Q = 0, or
�
N'(Q)e-pt - A = 0
(6 .47)
Upon applying the differentiation formula (2.8) to
N(Q) in (6.44'), we have
[ P( Q) - C'(Q)] e-P1 - A = 0
(6. 47')
The economic interpretation of this condition becomes easier when we
recall that, for an isoperimetric problem, the Lagrange multiplier A is a
constant. Along the optimal extraction path, the value of P(Q) - C ' (Q)
associated with any point of time must have a uniform present value, A. By
slightly rearranging the terms, we can reinterpret the condition as requiring
that P(Q) - C'(Q) grow at the rate p:
(6 .4 7" )
P(Q) - C ' (Q) = AeP1
[social optimum]
From the last equation, it is also clear that A has the connotation of " the
initial value of P(Q) - C'(Q)." If the P(Q) and the C(Q) functions are
specific, we can solve (6.47'') for Q in terms of A and t, say Q(A, t). The
latter, when substituted into the constraint in (6.45), then enables us to
solve for A.
Pure Competition Versus Monopoly
One major conclusion of the Hotelling paper is that pure competition can
yield an extraction path identical with the socially optimal one, whereas a
monopolistic firm will adopt an extraction path that is more conservationis­
tic, but socially suboptimal. We shall now look into this aspect of the
problem.
Assume for simplicity that there are n firms under pure competition.
The th firm extracts at a rate Qi out of a known total stock of Si under its
control. The firm's problem is to maximize the total discounted profits over
time, taking the product price to be exogenously set at P0:
i
( 6.48)
.
Maximize
subject to
'
{0 [ P0Qi - Ci(Qi ) ] e -pt dt
(0 Qi dt = si
The Lagrangian integrand now becomes
, . ,_.
1,;;
'
CHAPTER 6: CONSTRAINED PROBLEMS
151
and the application of the Euler-Lagrange equation yields the condition
( Po - C/(Q;)]e-pt - A = 0
or
[pure competition]
(6.49}
This condition is perfectly consistent with that for social optimum, (6.47"),
in that it, too, requires the difference between the price of the exhaustible
resource and its marginal cost of extraction to grow exponentially at the
rate p.
In contrast, the problem of monopolistic profit maximization is
Maximize
( 6.50)
( [ R(Q) - C(Q)]e -pt dt
0
subject to
This time, �tb the Lagrangian integrand
:T= [ R(Q) - C(Q)) e - pt - >.Q
the Euler-Lagrange equation leads to the condition
R'(Q) - C'( Q) = AeP1
(6.51)
[monopoly)
which differs from the rule for socially optimal extraction. Here, it is the
difference between the marginal revenue (rather than price) and the
marginal cost that is to grow at the rate p, with A now representing
the initial value of the said difference.
The Monopolist and Conservation
·
·
.
:
· ·
The conclusion of the Hotelling paper is, however, not only that monopolis­
tic production of an exhaustible resource is suboptimal, but also that it is
specifically biased toward excessive conservationism. How valid is the latter
claim? The answer is: It is under certain circumstances, but not always.
This issue has been examined by various people using different specific
assumptions. As one might expect, differences in assumptions result in
different conclusions. In a paper by Stiglitz,5 for example, it is shown that if
the elasticity of demand increases with time (with the discovery of substi­
tutes), or if the cost of extraction is constant per unit of extraction but
5
Joseph E. Stiglitz, "Monopoly and the Rate of Extraction of Exhaustible Resources," Ameri.,
•
can Economic Reuiew, September 1976, pp. 655-661 .
.f
i'
...
152
PART
2o THE CALCULUS OF VARIATIONS
decreases with time (with better technology), then the monopolist tends to
be more conservationistic than under the social optimum. But the opposite
conclusion is shown to prevail in a paper by Lewis, Matthews, and Burness,6
under the assumption that the extraction cost does not vary with the rate of
extraction (capital costs, leasing fees, etc., are essentially fixed costs), and
that the elasticity of demand increases with consumption (sufficiently low
prices can attract bulk users at the margin to switch from substitutes).
In one of Stiglitz's submodels, the monopolist faces the (inverse)
demand function
( 6 .52 )
( 0 < a < 1)
-
with elasticity of demand 1/(1
a). The per-unit cost of extraction is
constant at any given time, but it can decline over time:
c = tj>(t)Q
(6 .53)
( 4>' < 0 )
Thus the profit is
PQ -
C
<=
1/J(t)Qa - tj>(t)Q
And the monopolist's dynamic optimization problem i s to
Maximize
{ f l/l ( t)Q" - tf>(t)Q] e-pt dt
subject to
{Q dt
(6.54)
0
0
=
80
The Lewis-Matthews-Burness model, on the other hand, uses a general
(inverse) demand function P(Q) that is stationary, although its elastidty of
demand is assumed to increase with consumption. Since the cost of extrac­
tion is assumed to be a fixed cost ( = <I>), the monopolist's problem is to
Maximize
(6.55)
( [ P( Q ) Q - <l>]e-P1 dt
subject to
The Euler-Lagrange equation is applicable to each of these two prob­
lems in a manner similar to the Hotelling model. From that process, we can
deduce an optimal rate of growth of Q for the monopolist. Comparing that
Tracy R. Lewis, Steven A Matthews, and H. Stuart Burness, "Monopoly and the Rate of
Extraction of Exhaustible Resources: Note," American Economic Review, March 1979, pp.
6
227-230.
1 53
CHAPTER 6: CONSTRAINED PROBLEMS
rate to the rate of growth dictated by the social optimum will then reveal
whether the monopolist is excessively or insufficiently conservationistic.
Instead of analyzing problems (6.54) and (6.55) separately, however, we
shall work with a single, more general formulation that consolidates the
considerations present in
(6.54) and (6.55).
Let the demand function and the cost function be expressed
e81D ( P )
( 6.56)
Q
( 6.57)
C = C ( Q , t)
=
as
[ D' ( P ) < 0]
( CQ � o, ctQ
�
o)
These functions include those used by Stiglitz and Lewis, Matthews, and
Burness as special cases. In the following, we shall use the standard
abbreviations for marginal revenue and marginal cost, MR = R'(Q) and
MC = C'(Q), as well as the notation rx = (dxjdt)jx for the rate of growth
of any variable x.
The condition for social optimum that arises from the Euler-Lagrange
equation, (6.47"), requires that r< P - MC) = p. Using a familiar formula for
the rate of growth of a difference, 7 we can rewrite this as
p
And this implies that
-
rP
(6 .58)
P
r
MC p
=
p(l
- -
_
p
MC
MC
p
)
r Mc
MC
+
MC
We can translate this rp into a corresponding
linked to rp via the elasticity of demand
follows:8
p
=P
rMC
rQ by using the fact that rQ is
E < 0, or its negative, E = \ E \ , as
(6.59)
7
See Alpha C. Chiang, Fundamental Methods ofMathematical Economics, 3d ed., McGraw-Hill,
New York, 1984, Sec. 10.7.
8
First take the natural log of both sides of (6.56) to get
ln Q = gt + In D ( P )
Then derive
rQ b y
differentiating In Q with respect t o t :
d
d
1
d
1
dP
rQ = - ln Q = g + - ln D ( P ) = g + -- - D ( P ) = g + -- D' ( P ) dt
dt
D ( P ) dt
D(P)
dt
=g
+
D' ( P )P dPjdt
-p- = g + trp = g - Erp
D(P)
(where E
= lEI)
154
PART 2: THE CALCULUS OF VARIATIONS
Substituting (6.58) into (6.59), we then find the socially optimal rate of
growth of Q, denoted by rQ. (s for social optimum), to be
( 6 .60 )
[ ( �) � ]
C
rQ• = g - E p 1 -
C
+
[social optimum]
rMc
The counterpart expression for rQm (m for monopoly) can be derived
by a similar procedure. Condition (6.51) requires that r(MR - MC> = p , which
may be rewritten as
MR
MR -
MC
rMR -
MC
rM
MR - MC c
= P
And this implies that
( 6.61 )
However, since MR and P are related to each other by the equation
( 6 .62)
we may derive another expression for
( 6 ·63)
rMR ""
1 dMR
MR ----;it =
dPjdt
1
P ( 1 - 1 /E )
1
[(
rMR
from (6.62), namely,
) + E2 dt J
1 dP
1E dt
dEjdt
P dE
1
rE
= p
- + (1
- = rp + E
/E ) E E
- 1
- 1
EquatiJW (6.61) and (6.63), solving for rp, and then substituting into (S.t)W,
we get the rate of growth of Q under monopoly:
·
(6.64)
[ ( �) :�
rQm = g - E p 1 -
+
rMc E
�
r
1 E
]
·
[monopoly]
A comparison of (6.64) with (6.60) reveals that we can have
ifMC
( 6.65)
= rMc = rE =
0
The situation of MC 0 occurs when extraction is costless. For rMc = 0,
the marginal cost of extraction has to be time invariant [ C1Q = 0 in (6.57)].
And the condition rE = 0 is met when the demand elasticity is constant over
time. This set of conditions-also considered by Stiglitz-is quite stringent,
but if the conditions are met, then the monopolist will follow exactly the
same extraction path as called for under the social optimum, which is in no
=
\ :�-
CHAPTER 6: CON�NED PROBLEMS
155
Q
Path with larger rQ
0
'----- t
FIGURE 6.3
-
way overconservationistic. Note that the Ep term is negative and implies
a declining Q over time, unless offset by a sufficiently large positive value of
g. If the demand for the exhaustible resource is stationary (g 0), or shifts
downward over time (g < 0) as a result of, say, the discovery of substitutes,
then rq must be negative.
A different outcome emerges, however, when there is a positive MC
that is output-invariant (MC k > 0) as well as time-invariant (rMc = 0).
Retaining the assumption that rE = 0, we now find
=
=
( 6.66)
rq .
=
g -
[ ( �}]
E p 1-
and
Since P > MR, it follows that kjP is less than kjMR, so that rQ• < rQ m at
any level of Q. The social optimizer and the monopolist now follow different
Q(t) paths. Recall, however, that both paths must have the same total area
under the curve,. namely, 80. To satisfy this constraint, the monopolist's
Q(t) path, with the larger rq value (e.g., - 0.03 as against - 0.05), must be
characterized by a more gentle negative slope, as illustrated by the solid
curve in Fig. 6.3. Since such a curve entails lower initial extraction and
higher later extraction than the broken curve, the monopolist indeed emerges
in the present case as a conservationist, relative to the social optimizer. We
have reached the same conclusion here as Stiglitz, although the assump­
tions are different.
Nevertheless, it is also possible to envisage the opposite case where the
monopolist tends to undertake excessive early extraction. Assume again that
MC = rMc 0. But let the elasticity of demand E vary with the rate of
extraction Q. More specifically, let E '(Q) > 0-a case considered by Lewis,
Matthews, and Burness. Then, as Q varies over time, E must vary accord­
ingly, even though E is not in itself a function of t. With these assump­
tions, we now have a nonzero rate of growth of E:
=
dEjdt
E
rE = -- =
( 6.67 )
' .
·�-
· ;;
E' ( Q ) ( dQjdt)
E
E' ( Q)Q
rQ
E
156
PART 2o THE CALCULUS OF VARIATIONS
Taking the
we get
rQ
in this equation to be
rQm
= g - Ep +
Collecting terms and solving for
rQ m
( 6.68)
rQ m •
=
rQ m •
and substituting (6.67) into (6.64),
E' ( Q ) Q
r
E - 1 Qm
we finally obtain
g - Ep
1 - E'( Q ) Q/( E - 1)
which should be contrasted against
rQ s
=
g -
-
(from ( 6.60) with MC =
Ep
rMc =
0]
Suppose that g Ep < 0. Then, if the denominator in (6.68) is a positive
fraction, we will end up with rQ m < rQ s (e.g., dividing - 0.04 by � results in
- 0.08, a smaller number). And, in view of Fig. 6.3, the monopolist will in
such a case turn out to be an anticonservationist. It can be shown9 that the
denominator in (6.68) is a positive fraction when R"(Q) < 0, that is, when
dMR/dQ < 0. But a declining marginal revenue is a commonly acknowl­
edged characteristic of a monopoly situation. Therefore, we can see from the
present analysis that monopoly is by no means synonymous with conserva­
tionism.
EXERCISE 6.3
1
2
In (6.47"), we interpret A to mean the initial value of P(Q)
an alternative economic interpretation of A from (6.47').
- C'(Q). Find
If the maximization problem (6.45) is subject instead to the weak inequality
J;Q dt � 80, what modification(s) must be made to the result of the
analysis?
9By differentiating (6.62) with respect to Q, and
(
dMR
-
dQ
=
)
(
P'(Q)
letting the derivative be negative, we have
1
1 - -
E
) + P-E'(Q)
1
E2
<
0
Multiplying both sides of this inequality by E2QjP, and making use of the fact that
=
- 1/E, we can transform the inequality to the form -(E
E'(Q)Q
<
E-
1
-
1)
+ E'(Q)Q
<
0, or
P'(Q)QjP
In the latter inequality, the left-hand side is positive by virtue of our assumption that
E'(Q)
>
0; the right-hand side is positive because (6.51) requires MR to be positive, and, in
view of (6.62), the positivity of MR in turn dictates that
positive fraction, and so
is
the denominator in (6.68).
E
> 1. Hence,
E'(Q)Qj(E -
1) is a
CHAPTER 6: CONSTRAINED PROBLEMS
157
3
If the transversality conditions (5.5) and (5.7) are applied to the Lagrangian
integrands .r for social optimum and monopoly, respectively, what terminal
conditions will emerge? Are these conditions economically reasonable?
4 Compare rQ• and rQm in (6.60) and (6.64), assuming that (1) MC = rMc 0,
(2) the elasticity of demand is output-invariant, and (3) the demand
becomes more elastic over time because of the emergence of substitutes.
5 Assume that (1) the demand elasticity is output-invariant and time­
invariant, and (2) the MC is positive and output-invariant at any given
time, but increases over time. Write out the appropriate expressions for rq,
and rQ m • find the difference rQ m - rq , , study its algebraic sign, and
indicate under what conditions the monopolist will be more
conservationistic and less conservationistic than the social optimizer.
6 Let q = the cumulative extraction of an exhaustible resource: q(t) =
J� Q(t) dt. Verify that the current extraction rate Q is related to q by the
equation Q(t) = q'(t).
(a) Assume that the cost of extraction C is a function of the cumulative
extraction q as well as the current rate of extraction q'. How must the
N(Q) expression in (6.44) be modified?
(b) Reformulate the social optimizer's problem (6.45), expressing it in
terms of the variables q and q'(t).
(c) Write the Lagrangian integrand, and find the Euler-Lagrange equation.
7 Using the definition of q in the preceding problem, and retaining the
assumption that C = C(q, q'), reformulate the competitive firm's problem
(6.48) in terms of the variables q and q'(t). Show that the Euler-Lagrange
equation for the competitive firm is consistent with that of the social
optimizer.
""
�
.
· ··,
.
-
·::: '
OPTIMAL
CONTROL
THEORY
CHAPTER
7
OPTIMAL
CONTROL:
THE
MAXIMUM
PRINCIPLE
The calculus of variations, the classical method for tackling problems of
dynamic optimization, like the ordinary calculus, requires for its applicabil­
ity the differentiability of the functions that enter in the problem. More
importantly, only interior solutions can be handled. A more modern devel­
opment that can deal with nonclassical features such as corner solutions, is
found in optimal control theory. As its name implies, the optimal-control
formulation of a dynamic optimization problem focuses upon one or more
control variables that serve as the instrument of optimization. Unlike the
calculus of variations, therefore, where our goal is to find the optimal time
path for a state variable y, optimal control theory has as its foremost aim
the determination of the optimal time path for a control variable, u . Of
course, once the optimal control path, u*(t), has been found, we can also
find the optimal state path, y*(t), that corresponds to it. In fact, the optimal
u*(t) and y*(t) paths are usually found in the same process. But the
presence of a control variable at center stage does alter the basic orientation
of the dynamic optimization problem.
A couple of questions immediately suggest themselves. What makes a
variable a "control" variable? And how does it fit into a dynamic optimiza­
tion problem? To answer these questions, let us consider a simple economic
illustration. Suppose there is in an economy a finite stock of an exhaustible
· 161
PART 3: OPTIMAL CONTROL THEORY
162
S (such as coal or oil), as discussed in the Hotelling model, with
S(O) = S0. As this resource is being extracted (and used up), the resource
resource
stock will be reduced according to the relation
-
dS(t)
dt
-- =
- E( t)
where E(t) denotes the rate of extraction of the resource at time t. The E( t )
variable qualifies as a control variable because it possesses the following two
properties. First, it is something that is subject to our discretionary choice.
Second, our choice of E(t) impinges upon the variable
which indicates
the state of the resource at every point of time. Consequently, the E(t)
variable is like a steering mechanism which we can maneuver so as to
" drive" the state variable S(t) to various positions at any time t via the
differential equation dSjdt = -E(t). By the judicious steering of such a
control variable, we can therefore aim to optimize some performance crite­
rion as expressed by the objective functional. For the present example, we
may postulate that society wants to maximize the total utility derived from
using the exhaustible resource over a given time period [0, T]. If the
terminal stock is not restricted, the dynamic optimization problem may take
the following form:
S(t)
Maximize
subject to
and
JTU( E)e-P1 dt
0
dS
dt
S(O )
-
=
=
-E(t)
S0
S( T ) free
( S0 , T given)
In this formulation, it happens that only the control variable E enters
into the objective functional. More generally, the objective functional can be
expected to depend on the state variable(s) as well as the control variable(s).
Similarly, it is fortuitous that in this example the movement of the state
variable S depends only on the control variable E. In general, the coprse of
movement of a state variable over time may be affected by the state
variable(s) as well as the control variable(s), and indeed even by the
variable itself.
With this background, we now proceed to the discussion of the method
of optimal control.
t
7.1 THE SIMPLEST PROBLEM
OF OPTDKAL CONTROL
To keep the introductory framework simple, we first consider a problem
with a single state variable y and a single control variable u. As suggested
earlier, the control variable is a policy instrument that enables us to
163
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
u
�
c
a
0
I
I
I
Control
path
v--1
I
I
t,
.
.
A
y
c
I
I
0
(a)
t,
T
t2
(b)
FIGURE 7.1
influence the state variable. Thus any chosen control path u(t) will imply an
associated state path y(t). Our task is to choose the optimal admissible
control path u*(t) which, along with the associated optimal admissible state
path y*(t), will optimize the objective functional over a given time interval
[0, T).
Special Features of Optimal Control
Problems
It is a noteworthy feature of optimal control theory that a control path does
not have to be continuous in order to become admissible; it only needs to be
piecewise continuous. This means that it is allowed to contain jump discon­
tinuities, as illustrated in Fig. 7.1a, although we cannot permit any discon­
tinuities that involve an infinite value of u . A good illustration of
piecewise-continuous control in daily life is the on-off switch on the com­
puter or lighting fixture. Whenever we turn the switch on ( u 1) and off
(u = 0), the control path experiences a jump.
The state path y(t), on the other hand, does have to be continuous
throughout the time period [0, T ]. But, as illustrated in Fig. 7.1b, it is
permissible to have a finite number of sharp points, or corners. That is to
say, to be admissible, a state path only needs to be piecewise differentiable. 1
Note that each sharp point on the state path occurs at the time the control
path makes a jump. The reason for this timing coincidence lies in the
procedure by which the solution to the problem is obtained. Once we have
=
1
Sharp points on a state path can also be accommodated in the calculus of variations via the
Weierstrass-Erdmann conditions. We have not discussed the subject in this book, because of
the relative infrequency of its application in economics. The interested reader can consult any
book on the calculus of variations.
164
PART 3: OPTIMAL CONTROL THEORY
found that the segment of the optimal control path for the time interval
[0,
is, say, the curve
in Fig. 7 . 1 a , we then try to determine the
t1)
ab
corresponding segment of the optimal state path. This may turn out to be,
say, the curve AB in Fig.
whose initial point satisfies the given initial
condition. For the next time interval,
we again determine the
7.1b,
[t1, t2),
optimal state-path segment on the basis of the pre-found optimal control,
curve cd, but this time we must take point B as the " initial point" of the
optimal state-path segment. Therefore, point B serves at once as the
terminal point of the first segment, and the initial point of the second
segment, of the optimal state path. For this . reason, there can be no
discontinuity at point B, although it may very well emerge as a sharp point.
Like admissible control paths, admissible state paths must have a finite y
value for every
t in the time interval [0, T].
Another feature o f importance i s that optimal control theory is capable
u , such as the
'1& for all t E [0 , T ], where '1& denotes some bounded
control set. The control set can in fact be a closed, convex set, such as
u(t) E [0 , 1]. The fact that '1& can be a closed set means that corner
solutions (boundary solutions) can be admitted, which injects an important
nonclassical feature into the framework. When this feature is combined
with the possibility of jump discontinuities on the control path, an interest­
ing phenomenon called a bang-bang solution may result. Assuming the
control set to be '1& = [0, 1], if for instance the optimal control path turns out
to jump as follows:
of directly handling a constraint on the control variable
restriction
u(t) E
u*(t)
u*(t)
u* (t)
=
=
=
1
for
0
for
t E ( 0 , t1)
t E (t1, t2)
for t E [ t2, T ]
1
then we are " banging" against one boundary of the set 9&, and then against
the other, in succession; hence, the name " bang-bang."
Finally, we point out that the simplest problem in optimal control
theory, unlike in the calculus of variations, has a free terminal state
(vertical terminal line) rather than a fixed terminal point. The primary
reason for this is as follows: In the development of the fundamental
first-order condition known as
notion of an
arbitrary .iu.
the maximum principle, we shall invoke the
tl.u must, however, imply an
Any arbitrary
associated .iy. If the problem has a fixed terminal state, we need to pay
heed to whether the associated tl.y will lead ultimately to the designated
terminal state. Hence, the choice of
.iu
may not be fully and truly arbitrary.
If the problem has a free terminal state (vertical terminal line), on the other
hand, then we can let the arbitrary
.iu
lead to wherever it may without
having to worry about the final destination of y. And that simplifies the
problem.
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
165
The Simplest Problem
Based on the preceding discussion, we may state the simplest problem of
optimal control as
(7.1)
{0 F( t, y, u ) dt
Maximize
V=
subject to
y = f( t, y, u )
y(O) = A y( T ) free
( A, T given)
u(t) E � for all t E [0, T ]
and
Here, as in the subsequent discussion, we shall deal exclusively with the
problem. This way, the necessary conditions for optimization
can be stated with more specificity and less confusion. When a minimization
problem is encountered, we can always reformulate it as a maximization
problem by simply attaching a minus sign to the objective functional. For
example, minimizing J F(t,y, u ) dt is equivalent to maximizing JJ ­
maximization
F(t, y, u) dt.
J
In (7. 1), the objective functional still takes the form of a definite
integral, but the integrand function F no longer contains a y' argument as
in the calculus of variations. Instead, there is a new argument u . The
presence of the control variable u necessitates a linkage between u and y,
to tell us how u will specifically affect the course taken by the state variable
y. This information is provided by the equation y = f(t, y, u), where the
dotted symbol y, denoting the time derivative dy /dt, is an alternative
notation to the y' symbol used heretofore. 2 At the initial time, the first two
arguments in the f function must take the given value t = 0 and y(O) = A,
so only the third argument is up to us to choose. For some chosen policy at
t = 0, say, u 1(0), this equation will yield a specific value for y,. say y 1(0),
which entails a specific direction the y variable is to move. A different
policy, u (0), will in general give us a different value, yiO), via the f
2
function . And a similar argument should apply to other points of time. What
this equation does, therefore is to provide the mechanism whereby our
choice of the control u can be translated into a specific pattern of movement
of the state variable y. For this reason, this equation is referred to as the
equation of motion for the state variable (or the state equation for short).
Normally, the linkage between u and y can be adequately described by a
first-order differential equation y = f(t, y, u ). However, if it happens that
the pattern of change of the state variable y cannot be captured by the first
a
2 Even though the y and y' symbols are interchange ble, we shall exclusively use y in the
context of optimal control theory, to make a visual distinction from the calculus·of-variations
context.
PART 3: OPTIMAL CONTROL THEORY
166
derivative y but requires the use of the second derivative y """ d 2y1dt 2, then
the state equation will take the form of a second-.order differential equation,
which we must transform into a pair of first-order differential equations.
The complication is that, in the process, an additional state variable must be
introduced into the problem. An example of such a situation can be found in
Sec. 8 . 4.
We shall consistently use the lowercase letter f as the function symbol
in the equation of motion, and reserve the capital-letter F for the integrand
function in the objective functional. Both the F and f functions are
assumed to be continuous in all their arguments, and possess continuous
first-order partial derivatives with respect to t and y, but not necessarily
with respect to u .
The rest o f problem (7.1) consists o f the specifications regarding the
boundaries and control set. While the vertical-terminal-line case is the
simplest, other terminal-point specifications can be accommodated, too. As
to the control set, the simplest case is for � to be the open set � =
( - oo, + oo). If so, the choice of u will in effect be unconstrained, in which
case we can omit the statement u ( t ) E � from the problem altogether.
A Special Case
As
a special case, consider the problem where the choice of u is uncon­
strained, and where the equation of motion takes the particularly simple
form
y=u
Then the optimal control problem becomes
(7.2)
fTF(t, y, u) dt
Maximize
V=
subject to
y=u
y(O) = A
and
0
y( T ) free
F
.
.•
( A, T given)
By substituting the equation of motion into the integrand function,
ever, we can eliminate y , and rewrite the problem as
( 7.2' )
fTF(t, y, y) dt
Maximize
V
subject to
y(O) = A
=
0
y( T ) free
how­
" l.'
( A, T given)
This is precisely the problem of the calculus of variations with a vertical
terminal line. The fundamental link between the calculus of variations and
CHAPTER 7: OPTIMUM GONTROL: THE MAXIMUM PRINCIPLE
167
optimal control theory is thus apparent. But the equations of motion
encountered in optimal control problems are generally much more compli­
cated than in (7.2).
7.2 THE MAXIMUM PRINCIPLE
The most important result in optimal control theory-a first-order neces­
condition-is known as the maximum principle. This term was coined
by the Russian mathematician L. S. Pontryagin and his associates. 3 As
mentioned earlier in Sec. 1.4, however, the same technique was indepen­
dently discovered by Magnus Hestenes, a mathematician at the University
of California, Los Angeles, who later also extended Pontryagin's results.
The statement of the maximum principle involves the concepts of the
Hamiltonian function and costate variable. We must therefore first explain
these concepts.
sary
The Costate Variable and the Hamiltonian
Function
Three types of variables were already presented in the problem statement
(7.1): t (time), y (state), and u (control). It turns out that in the solution
process, yet another type of variable will emerge. It is called the costate
variable (or auxiliary variable), to be denoted by A . As we shall see, a
costate variable is akin to a Lagrange multiplier and, as such, it is in the
nature of a valuation variable, measuring the shadow price of an associated
state variable. Like y and u , the variable A can take different values at
different points of time. Thus the symbol A is really a short version of A(t).
The vehicle through which the costate variable gains entry into the
optimal control problem is the Hamiltonian function , or simply the Ham il­
tonian , which figures very prominently in the solution process. Denoted by
H, the Hamiltonian is defined as
( 7 .3)
H( t , y , u , A) = F( t , y , u) + A( t ) f( t , y, u )
Since H consists of the integrand function F plus the product of the costate
variable and the function f, it itself should naturally be a function with
four arguments: t, y, u as well as A. Note that, in (7.3), we have assigned a
unitary coefficient to F, which is in contrast to the yet undetermined A(t)
3L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko, The Mathemal·
ical Theory of Optimal Processes, translated from the Russian by K. N. Trirogoff,
Interscience,
New York, 1962. This book won the 1962 Lenin Prize for Science and Technology.
.
..,
168
PART 3, OPTIMAL CONTROL THEORY
coefficient for f. Strictly speaking, the Hamiltonian should have been
written as
( 7 .4)
H "' >. 0F( t , y , u )
+
A( t) f( t , y ,
u)
where >. 0 is a nonnegative constant, also yet undetermined. For the
vertical-terminal-line problem (7.1), it turns out that the constant A0 is
always nonzero (strictly positive); thus, it can be normalized to a unit value,
thereby reducing (7.4) to (7.3). The fact that A0 * 0 in the simplest problem
is due to two stipulations of the maximum principle. First, the multipliers
A 0 and >.( t ) cannot vanish simultaneously at any point of time. Second, the
solution to the vertical-terminal-line problem must satisfy the transversality
condition A(T ) = 0, to be explained in the ensuing discussion. The condition
A( T ) = 0 would require a nonzero value for >.0 at t = T. But since A0 is a
nonnegative constant, we can conclude that · A 0 is a positive constant, which
can then be normalized to unity.
For formulations of the optimal control problem other than (7. 1), on
the other hand, A 0 may turn out to be zero, thereby invalidating the
Hamiltonian in (7.3). The purist would therefore insist on checking in every
problem that A 0 is indeed positive, before using the Hamiltonian (7.3). The
checking process would involve a demonstration that >.0 = 0 would lead to a
contradiction and violate the aforementioned stipulation that A 0 and A(t)
cannot vanish simultaneously.4 In reality, however, the eventuality of a zero
A 0 occurs only in certain unusual (some say " pathological") situations
where the solution of the problem is actually independent of the integrand
function F, that is, where the F function does not matter in the solution
process.5 This is, of course, why the coefficient A0 should be set equal to
zero, so as to drive out the F function from the Hamiltonian. Since most of
the problems encountered in economics are those where the F function
does matter, the prevalent practice among economists is simply to assume
A 0 > 0, theri normalize it to unity and use the Hamiltonian (7.3), even when
the problem is not one with a vertical terminal line. This is the practice we
shall follow.
The Maximum Principle
In contrast to the Euler equation which is a single second-order differential
equation in the state variable y, the maximum principle involves two
4
For specific examples of the checking process, see Akira Takayama,
Mathematical Economics,
2d ed., Cambridge University Press, Cambridge, 1985, pp. 617-618, 674-675, and 679-680.
5An example of such a problem can be found in Morton I. Kamien and Nancy L. Schwartz,
Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and
Management, 2d ed. Elsevier, New York, 1991, p. 149.
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
169
first-order differential equations in the state variable y and the costate
variable A. Besides, there is a requirement that the Hamiltonian be maxi­
mized with respect to the control variable u at every point of time. For
pedagogical effectiveness, we shall first state and discuss the conditions
involved, before providing the rationale for the maximum principle.
For the problem in (7.1), and with the Hamiltonian defined in (7.3),
the maximum principle conditions are
Max H( t,y, u , A )
u
(7. 5)
A(T)
aH
ay
=
[O, T ]
[ equation of motion for y ]
y= A =
E
•. " · .
aH
()).
for all t
,
; ·' ·
[ equation of motion for A]
[ transversality condition ]
0
The symbol Max H means that the Hamiltonian is to be maximized
u
with respect to u alone as the choice variable. An equivalent way of
expressing this condition is
( 7 .6)
H( t, y , u* , A)
'
·
'2::.
for all t
H( t , y , u , A )
E
[0, T)
where u* is the optimal control, and u i s any other control value. In the
following discussion, we shall, for simplicity, sometimes use the shorter
notation " Max H " to indicate this requirement
without explicitly mention­
.
ing u . The reader will note that it is this requirement of maximizing H
with respect to u that gives rise to the name " the maximum principle."
It might appear on first thought that the requirement in (7.6) could
have been more succinctly embodied in the first-order condition iJHjiJu = 0
(properly supported by an appropriate second-order condition). The truth,
however, is that the requirement Max H is a much broader statement of
u
the requirement. In Fig. 7.2, we have drawn three curves, each indicating a
possible plot of the Hamiltonian H against the control variable u at a
specific point of time, for specific values of y and A. The control region is
assumed to be the closed interval [a, c). For curve 1, which is differentiable
with respect to u , the maximum of H occurs at u = b, an interior point of
the control region �; in this case, the equation iJHjiJu 0 could indeed
serve to identify the optimal control at that point of time. But if curve 2 is
the relevant curve, then the control in � that maximizes H is u = c, a
boundary point of �- Thus the condition iiHjiJu = 0 does not apply, even
though the curve is differentiable. And in the case of curve 3, with the
Hamiltonian linear in u , the maximum of H occurs at u = a, another
boundary point, and the condition aH;au 0 is again inapplicable because
=
=
PART 3: OP'I'IMAL OONTIIOL THBORY
170
H
�..;:;
I
I
I
I
:
--�----------- -----4·a
0
b
'
I
--L- a
-
c
FIGURE
7.2
nowhere is that derivative equal to zero. In short, while the condition
may serve the purpose when the Hamiltonian is differentiable
with respect to u and yields an interior solution, the fact that the control
region may be a closed set, with possible boundary solutions, necessitates
the broader statement Max H. In fact, under the maximum principle the
u
Hamiltonian is not even required to be differentiable with respect to u .
The case where the Hamiltonian is linear in u is of special interest.
For one thing, it is an especially simple situation to handle when H plots
against u as either a positively sloped or a negatively sloped straight line,
since the optimal control is then always to be found at a boundary of u . The
only task is to determine which boundary. (If H plots against u as a
horizontal straight line, then there is no unique optimal control.) More
importantly, this case serves to highlight how a thorny situation in the
calculus of variations has now become easily manageable in optimal co.ntrol
theory. In the calculus of variations, whenever the integrand function is
linear in y', resulting in Fy'y' = 0, the Euler equation may not yield a
solution satisfying the given boundary conditions. In optimal control theory,
in contrast, this case poses no problem at all.
Moving on to the other parts of (7 .5), we note that the condition
y = oHjiJA is nothing but a restatement of the equation of motion for the
state variable originally specified in (7 .1). To reexpress y as a partial
derivative of H with respect to the costate variable A is solely for the sake
of showing the symmetry between this equation of motion and that for the
costate variable. Note, however, that in the latter equation of motion, A is
the negatiue of the partial derivative of H with respect to the state variable
y. Together, the two equations of motion are referred to collectively as the
Hamiltonian system, or the canonical system (meaning the "standard"
oHjo u = 0
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
171
system of differential equations) for the given problem. Although we have
more than one differential equation to deal with in optimal control
theory-one for every state variable and every costate variable-each dif­
ferential equation is of the first order only. Since the control variable never
appears in the derivative form, there is no differential equation for u in the
Hamiltonian system. But from the basic solution of (7.5) one can, if desired,
derive a differential equation for the control variable. And, in some models,
it may turn out to be more convenient to deal with a dynamic system in the
variables (y, u ) in place of the canonical system in the variables (y, A ).
The last condition in (7.5) is the transversality condition for the
free-terminal-state problem-one with a vertical terminal line. As we would
expect, such a condition only concerns what should happen at the terminal
time T.
Find the curve with·the shortest distance from a given point
a given straight line L. We have encountered this problem before in
the calculus of variations. To reformulate it as an optimal control problem,
let point P be (0, A), and assume, without loss of generality, that the line L
is a vertical line. (If the given position of line L is not vertical, it can always
be made so by an appropriate rotation of the axes.) The previously used F
function, (1 + y' 2 ) I /Z can be rewritten as (1 + u 2 ) I12, provided we let y' = u ,
or j = u . Also, to convert the distance-minimization problem to one of
maximization, we must attach a minus sign to the old integrand. Then our
problem is to
EXAMPLE 1
P
to
( 7 . 7)
{T
o
J
Maximize
V=
subject to
j=u
and
y(O) = A
-
1 /2
( 1 + u 2 ) dt
y( T ) free
( A , T given)
Note that the control variable is not constrained, so the optimal control will
be an interior solution.
Step i We begin by writing the Hamiltonian function
1/
H = - ( 1 + u 2 ) 2 + Au
( 7.8)
Obsening that H is differentiable and nonlinear, we can apply the first-order
condition iJHjiJu = 0 to get
aH
au
,,
;.
.
172
PART 3o OPTIMAL CONTROL THEORY
This yields the solution6
(7.9)
=
u ( t)
Further differentiation of
/
A ( l - A2} - l 2
aH;au using the product rule yields
a 2H
au
-2 = - (1 + u2)
- 3;2
<0
Thus the result in (7.9) does maximize H. Since (7.9) expresses
of A , however, we must now look for a solution for A.
u
in terms
Step ii T o do that, we resort t o the equation o f motion for the costate
variable A = -aH;ay in (7.5). But since (7.8) shows that H is independent
of y, we have
( 7 .10 }
aH
ay
-
A = -- =0
A ( t)
= constant
Conveniently, the transversality condition A(T) = 0 in (7.5) is sufficient for
definitizing the constant. For if A is a constant, then its value at t
T is
also its value for all t. Thus,
=
(7.10')
A* ( t ) = 0
t
for all
E
[0, T ]
Looking back to (7.9), we can now also conclude that
( 7.11}
u*(t) = 0
t
for all
E
[0, T ]
Step iii From the equation of motion y = u , we are now able to write
( 7.12)
y( t) = constant
-
y=O
Moreover, the initial condition
stant and write
y(O) = A enables us to definitize this con­
(7.12')
y* ( t ) = A
6
·�··
The aH;au = 0 equation can be written as
u(l + u2)- 112 = A
Squaring both sides, multiplying through by
u2 ( 1
(1
-
+
.\2 )
u2), and collecting terms, we get
=
.\2
This result implies that A.2 * 1, for otherwise the equation would become 0 = 1, which is
impossible. Dividing both sides by the nonzero quantity (1 - A2), and taking the square root,
we finally arrive at (7.9).
:."'� . '
1 73
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
y
y'(t) = A
A t-----------... B
0
t
T
FIGURE 7.3
This y* path, illustrated in Fig. 7.3, is a horizontal straight line. Alterna­
tively, it may be viewed as a path orthogonal to the vertical terminal line.
EXAMPLE 2
( 7. 13)
'·
Find the optimal control that will
= fo2(2y - 3u) dt
Maximize
V
subject to
j=y + u
y(O) = 4
y(2) free
u(t) E � = [0, 2]
and
Since this problem is characterized by linearity in
set, we can expect boundary solutions to occur.
Step
i
u
and a closed control
The Hamiltonian of (7.13), namely,
H = 2y - 3 u + A(Y + u ) = (2 + A)y + ( A - 3)u
is linear in u , with slope aH;au = A - 3. If at a given point o f time, we find
A > 3, then an upward-sloping curve like curve 1 in Fig. 7.4 will prevail; to
maximize H, we have to choose u* = 2. If A < 3, on the other hand, then
curve 2 will prevail, and we must choose u* = 0 instead. In short,
( 7 . 14)
u* (t) =
{�}
Both u* = 2 and u* = 0 are, of course, boundary solutions. Note that,
because H is linear in u , the usual first-order condition aH;au = 0 is
inapplicable in our search for u*.
Step i i It is our next task to determine A(t), as needed in (7. 14). From the
equation of motion for A, we have the differential equation
aH
A = - -. = - 2 - A
ay
or
A + A = -2
PART 3: O PTIMAL CONTROL THEORY
174
H
Max H
Fl� 'f.4
Its general solution is7
A(t)
=
( k arbitrary)
ke-1 - 2
Since the arbitrary constant k can be definitized to k 2e 2 by using the
transversality condition A(T) = A(2) = 0, we can write the definite solution
as
=
( 7 .15)
Note that A*(t) is a decreasing function, falling steadily from an initial
value A*(O) = 2 e 2 - 2 "' 12.778 to a terminal value A*(2) 2 - 2 0.
Thus, A* exceeds 3 at first, but eventually falls below 3. The critical point of
time, when A* = 3 and when the optimal control should be switched from
u* 2 to u* = 0, can be found by setting A*(t) 3 in (7.15) and solving for
t. Denoting that particular t by the Greek letter T, we have
=
=
( 7 .1 6)
=
=
T
=
2 - ln 2.5 "' 1 .084
Consequently, the optimal control in (7.14) can be restated more specifically
in two phases:
( 7 . 17)
=
Phase I :
u*1
Phase II:
u*11 = u*[T, 2]
u*[O, T)
=
=
2
0
7
First-order linear differential equations of this type are explained in Sec. 14.1 of Alpha C.
Chiang, Fundamental Methods of Mathematical Economics, 3d ed., McGraw-Hill, New York,
1984.
.
�
CHAJ'TER 7:
175
OPTDWJI coNTRoL: THE MAXIIfUM PIUNCIPLE
u•
Phase I
(•J :o
2
Phase II
1-----___,j
0 �--------_.._________��2
1 ' = 1 .083
y•
39.339
(b)
15.739
I
I
I
- - - - - - - - - -�- - - - - - - - I
i
I
I
4.000
0 =-----"--..-.t�--o ---�2.__ t
1 ' = 1.083
As graphically depicted in Fig.
simple variety of bang-bang.
FIGURE 7.5
7.5a, this optimal control exemplifies a
Step iii Even though the problem only asks for the optimal control path,
we can also find the optimal state path, in two phases. In phase I, the
equation of motion for y is j = y + u = y + 2, or
with initial value y(O) = 4
j -y = 2
Its solution is
y*1
( 7.18)
=
y*[O, r)
=
2(3e1 - 1)
In phase II, the equation of motion for y is y = y
+
0; �
y -y = O
with general solution
( 7 . 19)
y *11 = y*[r, 2] = ce1
( c arbitrary)
Note that the constant c cannot be definitized by the initial condition
4 given in (7.13) because we are already in phase II, beyond t = 0.
Nor can it be definitized by any terminal condition because the terminal
state is free. However, the reader will recall that the optimal y path is
required to be continuous, as illustrated in Fig. 7.1b. Consequently, the
y(O)
=
PART 3, OPTIMAL CONTROL THEORY
176
initial value of y *11 must be set equal to the value of y'"r evaluated at
Inasmuch as
r.
(by ( 7.18)]
and
y *11( r)
=
ce•
[by (7 .19) ]
we find, by equating these two expressions and solving for c, that c
e-•), so that the optimal y path in phase II is
=
2(3 -
( 7.19' )
The value of y* at the switching time T is approximately 2(3eL096 1)
15.739.
By joining the two paths (7.18) and (7.19'), we obtain the complete y*
path for the time interval [0, 2], as shown in Fig. 7.5b. In this particular
example, the joined path happens to look like one single exponential curve,
but the two segments are in fact parts of two separate exponential func­
tions.
-
=
EXERCISE 7.2
1
2
2, A(t)
3 at only
A(t) = 3 for
In Example
is a decreasing function, and attains the value
one point of time, T. What would happen if it turned out that
all t?
Find the optimal paths o f the control, state, and costate variables to
Maximize
subject to
and
f0' 3ydt
y =y + u
y(O) = 5
u(t) E (0, 2]
Be sure to check that the Hamiltonian is maximized rather than minimized.
3
Find the optimal paths of the control, state, and costate variables to
Maximize
subject to
and
,
t(Y
0
-
u2) dt
y=u
y(O) = 0
y(2) free
u ( t) unconstrained
Make sure that the Hamiltonian is maximized rather than minimized.
177
CHAPTER 7 , OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
4
Find the optimal paths of the control, state, and costate variables to
Maximize
il �
subject to
y = u -y
and
y(O)
-
=
+ u2) dt
(y2
1
y( l ) free
u
( t) unconstrained
Make sure that the Hamiltonian is maximized rather than minimized.
[ Hint: The two equations of motion should be solved simultaneously.
Review the material on simultaneous differential equations in Alpha C.
Chiang, Fundamental Methods of Mathematical Economics, 3d ed.,
McGraw-Hill, New York,
Sec.
1984,
18.2.]
7.3 THE RATIONALE OF THE
MAXIMUM PRINCIPLE
We shall now explain the rationale underlying the maximum principle.
What we plan to do is not to give a detailed proof-the complete proof given
by Pontryagin and his associates (Chap. 2 of their book) runs for as many as
40 pages-but to present a variational view of the problem, to make the
maximum principle plausible. This will later be reinforced by a comparison
of the conditions of the maximum principle with the Euler equation and the
other conditions of the calculus of variations.
A Variational View of the Control
Problem
To make things simple, it is assumed here that the control variable u is
unconstrained, so that u* is an interior solution. Moreover, the Hamilto­
nian function is assumed to be differentiable with respect to u , and the
aH 1a u 0 condition can be invoked in place of the condition " Max H." As
u
usual, we take the initial point to be fixed, but the terminal point is allowed
to vary. This will enable us to derive certain transversality conditions in the
process of the discussion. The problem is then to
=
(7.20)
Maximize
v=
subject to
y
and
y(O)
=
f0TF( t, y, u ) dt
f( t , y, u )
= Yo
(given)
178
PART 3, OPTIMAL CONTROL THEORY
Step i As the first step in the development of the maximum principle, let
us incorporate the equation of motion into the objective functional, and
then reexpress the functional in terms of the Hamiltonian.
The reader will observe that, if the variable
always obeys the
equation of motion, then the quantity [
- j] will assuredly take a
zero value for all in the interval [0, T ]. Thus, using the notion of Lagrange
multipliers, we can form an expression
for each value of
and still get a zero value. Although there is an infinite number of values of
t in the interval [0, T ) summing
- y] over in the period
[0 , T ) would still yield a total value of zero:
y
f(t, y, u )
A(t)( f(t, y, u ) - y]
A(t)( f(t, y, u)
t
t
t,
,
jTA
(t ) [ f(t, y, u) - yj dt = 0
0
( 7.2 1)
For this reason, we can augment the old objective functional by the integral
in (7.21) without affecting the solution. That is, we can work with the new
objective functional
(7.22)
r= V + J0TA(t) [ f(t,y, u ) - y]. dt
=
t, y, u) + A(t ) [ f(t , y , u ) - y ] } d' t
({F(
0
V,
confident that r will have the same value as
as long as the eq� of
motion in (7.20) is adhered to at all times.
Previously, we have defined the Hamiltonian function as
H(t, y, u , A ) = F(t, y, u) + A(t ) f(t, y, u )
substitution of the H function into (7.22) can simplify
The
functional to the form
( 7 .22' )
i/=
=
the
new
jT
[H(t,y, u , >.) - >.(t) Y ] dt
0
t,y , u , A) dt - (>.(
t)ydt
(H(
0
0
It is important to distinguish clearly between the second term in the
Hamiltonian,
on the one hand, and the Lagrange-multiplier
expression,
)[
), on the other. The latter explicitly contains
whereas the former does not. When the last integral in (7.22') is integrated
A(t) f(t,y, u),
A(t f(t, y, u ) - y
y,
179
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
by parts,8 we find that
- f TA.( t) y dt = - A. ( T ) Yr + .A(O)y0 + f Ty(t)A. dt
0
0
·
Hence, through substitution of this result, the new objective functional can
be further rewritten as
{ 7 .22" ) 'Y=
H( t, y, u , .A ) + y(t) ,q dt - A.(T)yr
fr[
0
+
.A{ O)y0
--------�--� ------- �
The r expression comprises three additive component terms, !1 1, fl2,
and !13. Note that while the 01 term, an integral, spans the entire planning
period [0, T ], the !12 term is exclusively concerned with the terminal time
T, and !13 is concerned only with the initial time.
Step ii
The value of r depends on the time paths chosen for the three
variables y, u , and .A, as well as the values chosen for T and Y r · In the
present step, we shall focus on .A .
The .A variable, being a Lagrange multiplier, differs fundamentally
from u and y , for the choice of the .A(t) path will produce no effect on the
value of r, so long as the equation of motion y = f( t, y, u ) is strictly
adhered to, that is, so long as
( 7 .23)
aH
a.A
y= ­
for all t e
[O , T ]
So, to relieve us from further worries about the effect of A(t) on r, we
simply impose ( 7 .23) as a necessary condition for the maximization of r.
This accounts for one of the three conditions of the maximum principle.
This, of course, is hardly an earth-shaking step, since the equation of
motion is actually given as a part of the problem itself.
8The formula for integrating a definite integral by parts has been given in (2.15). �. we
replace the symbol u in (2.15) by w, because u is now used to denote � Control v�. Let
u A ( l)
w = y(t)
=
Then, since
( implying that du = A dt)
(implying that
dw = jdt)
A(t)jdt = u dw, we have
- tA(t)jdt
- - [ A (t)y(t)]� + t
y(t)A dt
0
0
wbidl leada to the J"811\ltl in the text.
180
PART 3: 0PTDMAL CONTROL T»EORY
Step iii We now turn to the u(t) path and its effect on the ( t ) path. If we
have a known u*(t) path, and if we perturb the u*(t) path with a perturb­
ing curve p(t), we can generate " neighboring" control paths
y
u ( t) = u*(t) + £p(t)
( 7 .24)
one for each value o f £ . But, i n accordance with the equation o f motion
y = f(t, y, u), there will then occur for each £ a corresponding perturbation
in the y* ( t ) path. The neighboring paths can be written as
y
y(t) = y* ( t ) + £q(t)
( 7.25)
T and Yr are variable, we also have
T = T* + 6.T and Yr = y/ + AYr
dT
dyr
implying
6.T and d'; = tlyr
d£ =
Furthermore, if
�
( 7.26)
�
(
)
In view of the u and y expressions in (7.24) and (7.25), we can express Y in
terms of £, so that we can again apply the first-order condition dYjd£ = 0.
The new version of Y is
·
( 7 .27) Y=
H [ t,y* + £q( t) , u* + £p(t) , A] + A[y* + £q(t)] } dt
r(•){
0
- A( T) Yr + A{O)y0
Step iv We now apply the condition d'Yjd£ 0. In the differentiation
process, the integral term yields, by formula (2.11), the derivative
=
( 7 .28)
�
T<• l
{[:� q(t) + :�p(t) 1 + Aq(t) } dt + [ H + A L_ r �:
y
h�
And the derivative of the second term in (7.27) with respect to £ is, from t
product rule,
( 7 .29)
dy
d A( T) dT
.
- A( T) der - Yr �
--;[; = -A( T ) Ayr - YrA( T) 6.T
[ by (7.26)]
On the other hand, the A(O)y0 term in (7.27) drops out in differentiation.
Thus dYjd� is the sum of (7.28) and (7.29). In adding these two expres­
sions, however, we note that one component of (7.28) can be rewritten as
follows:
[. ]
Ay
t-T
dT
d£
.
= A( T)yr t:. T
[ by ( 7.26)]
Thus, when the sum of (7.28) and (7.29) is set equal to zero, the first-order
CHAPTER 7 : OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
181
condition emerges (after rearrangement) as
r
�� = l [(:: + A ) q ( t) + :�p(t) ] dt
+ [ H ] t =T LlT - A ( T ) dJr = 0
(7.30)
0
.
The three components of this derivative relate to different arbitrary
things: The integral contains arbitrary perturbing curves
and q(t),
whereas the other two involve arbitrary LlT and Llyr, respectively. Conse­
quently, each of the three must individually be set equal to zero in order to
satisfy
By setting the integral component equal to zero, we can
deduce two conditions:
p(t)
(7.30).
iJH
iJy
aH
=0
au
and
The first gives us the equation of motion for the costate variable A (or the
costate equation for short). And the second represents a weaker version of
the
condition-weaker in the sense that it is predicated on the
u
assumption that
is differentiable with respect to u and there is an
interior solution. Since the simplest problem has a fixed T and free Yr, the
LlT term in
is automatically equal to zero, but
is not. In order to
make the - A(T)
expression vanish, we must impose the restriction
"MaxH"
H
(7.30)
ilyr
!:!.yr
A( T )
=
0
(7.5).
This explains the transversality condition in
Note that although the A(t) path was earlier, in Step ii, brushed aside
as having no effect on the value of the objective functional, it has now made
an impressive comeback. We see that, in order for the maximum principle to
work, the A(t) path is not to be arbitrarily chosen, but is required to follow a
prescribed equation of motion, and it must end with a terminal value of zero
if the problem has a free terminal state.
7.4 ALTERNATIVE TERMINAL
CONDITIONS
What will happen to the maximum principle when the terminal condition
specifies something other than a vertical terminal line? The general answer
is that the first three conditions in
will still hold, but the transversality
condition will assume some alternative form.
(7.5)
Fixed Terminal Point
The reason why the problem with a fixed terminal point (with both the
terminal state and the terminal time fixed) does not qualify as the " sim-
182
PART 3o OPTIMAL CONTROL THEORY
plest" problem in optimal control theory is that the specification of a fixed
terminal point entails a complication in the notion of an " arbitrary"
perturbing curve p(t) for the control variable u . If the perturbation of the
u*(t) path is supposed to generate through the equation of motion j =
f(t, y, u ) a corresponding perturbation in the y*(t) path that has to end at a
preset terminal state, then the choice of the perturbing curve p(t) is not
truly arbitrary. The question then a,rises as to whether we can still legiti­
mately deduce the condition oHjou = 0 from (7.30).
Fortunately, the validity of the maximum principle is not affected by
this compromise in the arbitrariness of p(t). For simplicity, however, we
shall not go into details to demonstrate this point. For our purposes, it
suffices to state that, with a fixed terminal point, the transversality is
replaced by the condition
y ( T ) = Yr
( T , yr given)
Horizontal Terminal Line (Fixed­
Endpoint Problem)
If the problem has a horizontal terminal line (with a free terminal time but
a fixed "endpoint," meaning a fixed terminal state), then Y r is fixed
( Llyr = 0), but T is not (LlT is arbitrary). From the second and third
component terms in (7.30), it is easy to see that the transversality condition
for this case is
( 7 .31 )
The Hamiltonian function must attain a zero value at the optimal terminal
time. But there is no restriction on the value of A. at time T.
Terminal Curve
In case a terminal curve Yr = cf>(T) governs the selection of the terminal
point, then LlT and Llyr are not both arbitrary, but are linked to each other
by the relation Llyr = cf>'(T) LlT. Using this to eliminate Llyr, we can
combine the last two terms in (7.30) into a single expression involving LlT
only:
[ H ]1= r tl T - A.( T ) cf>' ( T ) LlT = [ H - A. c/>' ]1 - r tl T
It follows that, for an arbitrary LlT, the transversality condition should be
( 7 .32)
Truncated Vertical Terminal Line
Now consider the problem in which the terminal time T is fixed, but the
terminal state is free to vary, only subject to Yr � Ymin> where Ymin denotes
a given minimum permissible level of y.
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
183
Only two types of outcome are possible in the optimal solution: y/ >
= Ymin · In the former outcome, the terminal restriction is
automatically satisfied. Thus, the transversality condition for the problem
with a regular vertical terminal line would apply:
Ymin ' or Yr*
( 7 .33)
A( T )
=
0
for Y / > Ymin
In the other outcome, y/ = Ymin• since the terminal restriction is
binding, the admissible neighboring y paths consist only of those that have
terminal states Yr � Ymin · If we evaluate (7.25) at t = T and let y/ Ymin•
we obtain
=
Yr = Ymin + Eq( T )
Assuming that q(T) > 0 on the perturbing curve q(t),9 the requirement
� 0. But, by the Kuhn-Tucker conditions,
the nonnegativity of E would alter the first-order condition d'YjdE = 0 to
d'YjdE � 0 for our maximization problem.10 It follows that (7.30) would
now yield an inequality transversality condition
Yr � Ymin would dictate that E
At the same time, we can see from (7.26) that, given E � 0, the requirement
of Yr � Y min-which is the same as Yr � y/ in the present context-im­
plies t:.yr � 0. Thus the preceding inequality transversality condition re­
duces to
( 7.34)
A(T)
�
0
for y/
=
Ymin
Combining (7.33) and (7.34) and omitting the * symbol, we can finally
write a single summary statement of the transversality condition as follows:
( 7 .35)
( Yr - Ymin )A(T)
=
0
Note that the last part of this statement represents the familiar complemen­
tary-slackness condition from the Kuhn-Tucker conditions. As in the similar
problem with a truncated vertical terminal line in the calculus of variations,
the practical application of (7.35) is not as complicated as the condition may
appear. We can always try the A(T ) = 0 condition first, and check whether
the resulting y/ value satisfies the terminal restriction y/ � Y min· If it
does, the problem is solved. If not, we then set Yr* = Ymin in order to satisfy
the complementary-slackness condition, and treat the problem as one with a
given terminal point.
9
This assumption does not bias the final result of the deductive process here.
10
The Kuhn-Tucker conditions are explained in Alpha C. Chiang, Fundamental
Mathematical Economics, 3d ed., McGraw-Hill, New York, 1984, Sec. 21.2.
Methods Of
,---
PART 3, 0PTndAL CONTROL THEORY
184
Truncated Horizontal Terminal Line
Let the terminal state be fixed, but the terminal time T be allowed to vary
subject to the restriction that T * ::s; Tmax• where Tmax is the maximum
permissible value of T-a preset deadline. Then we either have T* < Tmax
or T* = Tmax in the optimal solution.
In the former outcome, the terminal restriction turns out to be
nonbinding, and the transversality condition for the problem with a regular
horizontal terminal line would still hold:
[ H ]t-T = 0
( 7.36)
for T * < Tmax
But if T* = Tmax• then by implication all the admissible neighboring y
paths must have terminal time T ::s; Tmwc By analogous reasoning to that
leading to the result (7.34) for the truncated vertical terminal line, it is
possible to establish the transversality condition
[ H )t-T � O
( 7 .37)
for T * = Tmax
By combining (7.36) and (7.37) and omitting the * symbol, we then
obtain the following summary statement of the transversality condition:
( 7.38)
EXAMPLE 1
Maximize
subject to
y =y
and
y( O ) = 1
+
u
y(l) = 0
With fixed endpoints, we need no transversality condition in this problem.
Step i
Since the Hamiltonian function is nonlinear:
and since
H = - u 2 + A (y + u )
u
is unconstrained, we can apply the first-order
aH
- = -2u
au
This yields the solution
( 7 .39)
u = A/2
+ ,\
condition
=0
or, more accurately,
u ( t) = !A( t)
Since iPH;au2 = - 2 i s negative, this u(t) solution does maximize rather
than minimize H. But since this solution is expressed in terms of A(t), we
must find the latter path before u(t) becomes determinate.
185
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
Step ii
From the costate equation of motion
aH
ay
-A
we get the general solution
A(t)
( 7.40)
=
ke-1 ( k arbitrary)
.
, ·· , _
To definitize the arbitrary constant, we try to resort to the boundary
conditions, but, unfortunately, for the fixed-terminal-point problem these
conditions are linked to the variable y instead of A. Thus, it now becomes
necessary first to look into the solution path for
Step iii
y.
The equation of motion for y is y y
=
(7.40), however, we can rewrite this equation as y
=
+
u . Using (7.39) and
+
y tke - 1, or
This is a first-order linear differential equation with a variable coefficient
and a variable term, of the type
= w(t)-here with u(t) =
- 1 and w(t) tke - 1 • Via a standard formula, its solution can be found as
followsY
dyjdt + u(t) y
=
y ( t ) = e- f - l dt ( c + I � ke- le f-ldt dt )
( 7 . 4 1)
•,
= e1 ( c + J �ke-1e-1 dt )
= e1( c - �ke-21)
1
( c arbitrary)
-1
ce1 - -ke
4
=
u
See Alpha C. Chiang, Fundamental Methods of Mathematical Economics, 3d ed., McGraw­
Hill, New York, 1984, Sec. 14.3. In performing the integrations involved in the application of
the formula, we have omitted the constants of integration whenever they can be subsumed
under other constants. Alternatively, we can find the complementary function and the particu­
lar integral separately and then combine them. With a variable term in the differential
equation, we can obtain the particular integral by the method of undetermined coefficients
(ibid., Sec. 15.6)
186
PART 3: OPTIMAL CONTROL THEORY
Step iv Now the boundary conditions y(O) = 1 and y(1) = 0 are directly
applicable, and they give us the following definite values for c and k:
c = 1 -1 e 2
Consequently, substituting-these into (7.41), (7.40), and (7.39), we havttthe
following definite solutions for the three optimal paths:
1
y* ( t) = 1 - e 2 et -
e2
1 -
e2
e-t
and
The search for the u*(t), y*(t), and A*(t) paths in the present problem
turns out to be an intertwined process. This is because, unlike the simplest
problem of optimal control, where the transversality condition A(T) = 0
may enable us to get a definite solution of the costate path A*(t) at an early
stage of the game, the fixed-terminal-point problem does not allow the
application of the boundary conditions on y(O) and y(T) until the final stage
of the solution process.
EXAMPLE
condition
2 Let us reconsider the preceding example, with the terminal
y(1) = 0 replaced by the restriction
T= 1
y(1)
;e:
3
The problem is then one with a truncated vertical terminal line, and the
appropriate transversality condition is (7.35). We shall first try to solve this
problem as if its vertical terminal line is not truncated. If y*(1) turns out to
be ;e: 3, then the problem is solved; otherwise, we shall then redo the
problem by setting y(1) = 3.
Step i The Hamiltonian remains the same
solution for the control variable is still
(7.42 )
u ( t) = �.\(t)
as
in
-�
1,
.and
the
[ from (7 .39)]
Step ii Although the general solution for A is still
(7.43)
.\(t) = ke- t
[ from (7.40)]
we now can use the transversality condition .\(T) = 0 or A(l) - 0 to defini..
.
tize the arbitrary constant. The result is k = 0, so that
( 7.43')
A*(t) = 0
187
CHAPTER 7, OPTIMUM CONTROL, THE MAXIMUM PRINCIPLB .
It then follows from
(7.42) that
( 7 .44)
u* ( t )
Step iii
=0
y, we find
[ by (7.44)]
From the equation of motion for
y =y + u =y
The general solution o f this differential equation is
y ( t ) = ce1
, where the constant c can be definitized to
y(O) = 1. Thus the optimal state path is
·
(7 .45)
c
=1
by the initial condition
y* (t) = e1
Step iv It remains to check (7.45) against the terminal restriction. At the
fixed terminal time T = 1, (7.45) gives us y*(l) = e . This, unfortunately,
violates the terminal restriction y(1) ;;:.: 3. Thus, in order to satisfy the
transversality condition (7 35), we now have to set y(1) = 3 and resolve the
problem as one with a fixed terminal point. Note that had the terminal
restriction been T = 1, y(1) ;;:>:: 2, then (7.45) would have been an acceptable
solution.
.
·
-
'
, EXAMPL:& Jt.
V=
Maximize
' - -. :'_,_, ·.
� ,-,....
fT - 1 dt
0
y =y + u
y(O) = 5 y( T ) = 11
u(t) E [ - 1, 1]
subject to
and
T free
This example illustrates the problem with a horizontal terminal line. More­
over, it illustrates the type of problem known as a time-optimal problem,
, whose objective is to reach some preset target in the minimum amount of
time. The time-optimal nature of the problem is conveyed by the objective
functional:
Clearly, to
Step
maximize
fT - 1 dt = [ t T
-
0
this integral is to
]0 =
-T
minimize T.
i To begin with, form the Hamiltonian
- ( 7 .46)
H = -1
+ A(y + u )
188
PART 3: 0PT� CONTROL THEORY
Because this H function is linear in u , the iJHjiJu = 0 condition is inappli­
cable. And, with the control variable confined to the closed interval [ - 1, 1],
the optimal value of u at any point of time is expected to be a boundary
value, either - 1 or 1. Specifically, if A > 0 (H is an increasing function of
u ), then u* 1 (upper bound); but if A < 0, then u* = - 1. As a third
possibility, if A = 0 at some value of t, then the Hamiltonian will plot as a
horizontal line against u , and u* will become indeterminate at that point of
time. This relationship between u* and A �n be succinctly captured in the
so-called signum function, denoted by the symbol sgn, and defined as
follows:
=
( 7.47)
y=
y = sgn x
{ �7 }
indete
inate
if x
{;}
0
Note that if y is a signum function of x, then y (if determinate) can only
take one of two values, and the value of y depends on the sign (not
magnitude) of x.
Applied to the present problem, this function becomes
( 7 .48)
u*
= sgn
A
or
Once more, we find that a knowledge of
determined.
A is necessary before
u can
be
Step ii The equation of motion for the costate variable is, from (7.46),
iJH
ay
-A
which integrates to
( 7 .49)
A(t)
=
ke -t
( k arbitrary)
In this result, A(t), being exponential, can take only a single algebraic
sign-the sign of the constant k. Consequently, barring the eventuality of
k = 0 so that A(t) 0 for all t (which eventuality in fact does not occur
here), u* must be determinate and adhere to a single sign-nay, a single
constant value-in accordance with the signum function. For this reason,
even though the linearity of the Hamiltonian in the control variable u
results in a boundary solution in the present example, it produces no
bang-bang phenomenon.
It turns out that the clue to the sign of k lies in the transversality
condition [H]1_r
0. Using the H in (7.46), the A in (7.49), and the
terminal condition y(T)
11, we can write the transversality condition as
=
=
=
- 1 + ke- r ( l l + u*)
=
0
189
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
Since
u* is either 1 or - 1, the quantity (11
e - r. Therefore,
+
u*) must be positive, as is
k must be positive in order to satisfy this condition. It then
follows that A(t) > 0 for all t, and that
u* (t)
( 7.50)
=
1
Step iii With u * = 1 for all t, we can expre8s 'the equation of motion of
the state variable, y = y + u , as
y -y = 1
This fits the format of the first-order linear differential equation with a
constant coefficient and a constant term, dyjdt + ay = b-here with a =
- 1 and b = 1. Its definite solution gives us the optimal y path12
y*(t)
( 7.51)
=
=
[y(O) - � ]e-at + �
e
6 1- 1
= 5)
[y(O)
Step iv Having obtained the optimal control and state paths u*(t) and
y*(t), we next look for A*(t). For this purpose, we first return to the
transversality condition [ H lt - r 0 to fix the value of the constant k. In
view of (7.50) and (7.51), the transversality condition now reduces to
=
or
Thus k
path
=
6k
=
1
i. Substituting this result in (7.49) then yields the optimal
A*( t)
( 7 .52)
A
= ie- 1
Step v The three optimal paths in (7.50), (7.51), and (7.52) portray the
complete solution to the present problem except for the value of T*. To
calculate that, recall that the terminal state value is stipulated to be
y(T) = 1 1 . This, in conjunction with the y*(t) path obtained earlier, tells us
that 11 = 6e r - 1, or eT = 2. Hence,
T*
=
ln 2 (
==
0 .6931)
The optimal paths for the various variables are easily graphed. We shall
leave this to the reader.
12
This solution formula is derived in Alpha C. Chiang, Fundamental Methods of Mathematical
Economics, 3d ed., McGraw-Hill, New York, 1984, Sec. 14. 1.
190
PART 3: OPTIMAL CONTROL THEORY
The Constancy of the Hamiltonian
in Autonomous Problems
All
the examples discussed previously share the common feature that the
problems are "autonomous;" that is, the functions in the integrand and f
in the equation of motion. do not contain t as an explicit argument. An
important consequence of this feature is that the optimal Hamiltonian-the
Hamiltonian evaluated along the optimal paths of y, u , and A-will have a
constant value over time.
To see this, let us first examine the time derivative of the Hamiltonian
H(t, y, u , A) in the general case:
dH
-
dt
oH
=
-
at
+
oH
-.Y
ay
aH
+ -u +
au
oH .
-A
aA
When H is maximized, we have iJHjou 0 (for an interior solution) or
u = 0 (for a boundary solution). Thus the third term on the right drops out.
Moreover, the maximum principle also stipulates that y = oHjoA and A =
-aHjoy. So the second and fourth terms on the right exactly cancel out.
The net result is that H*, the Hamiltonian evaluated along the optimal
paths of all variables, satisfies the equation
=
( 7 .53)
dH*
aH*
dt
at
This result holds generally, for both autonomous and nonautonomous
problems.
In the special case of an autonomous problem, since t is absent from
the F and f functions as an explicit argument, the Hamiltonian must not
contain the t argument either. Consequently, we have oH* jot = 0, so that
( 7 .54)
dH*
--
dt
=0
or
H* = constant
[ for autonomous problems ]
This result is of practical use in an autonomous problem with a
horizontal terminal line. The transversality condition [ H ]1 � r 0 is nor­
mally expected to hold at the terminal time only. But if the Hamiltonian is a
constant in the optimal solution, then it must be zero for all t, and the
transversality condition can be applied at any point of time.
In Example 3, for instance, we find that
=
H*
=
- 1 + A* ( y*
+
u* ) = - 1 + �e-1(6 e1 - 1 + 1} = 0
This zero value of H* prevails regardless of the value of t, which shows
that the transversality condition is indeed satisfied at all t.
191
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
EXERCISE 7.4
1
Find the optimal paths of the control, state, and costate variables to
Maximize
2
foT - (t2 + u2) dt
u
subject to
y
and
y(O)
=
=
y( T )
4
T free
Find the optimal paths of the control, state, and costate variables to
Maximize
subject to
fo43y dt
y
=
y+u
y ( O)
and
3
= 5
=
5
0 :S u(t)
y(4)
:S
�
300
2
We wish to move from the initial point (0, in the ty plane to achieve the
terminal state value y(T) = 0 as soon as possible. Formulate and solve the
problem, assuming that dy1dt = 2 u , and that the control set is the closed
interval [ - 1, 1).
8)
4 Find the optimal control path and the corresponding optimal state path
that minimize the distance between the point of origin (0, 0) and a terminal
curve y(T ) = 10 - T 2, T > 0. Graph the terminal curve and the y * ( t)
path.
5
(7.37) for the
Demonstrate the validity of the transversality condition
problem with a truncated horizontal terminal line.
7.5 THE CALCULUS OF VARIATIONS
AND OPTIMAL CONTROL THEORY
COMPARED
We have shown earlier in (7.2) and (7.2') that a simple optimal control
problem can be translated into an equivalent problem of the calculus of
variations. One may wonder whether, in such a problem, the optimality
conditions required by the maximum principle are also equivalent to those
of the calculus of variations. The answer is that they indeed are.
For problem (7.2), the Hamiltonian function is
( 7 .55)
H = F( t,y, u ) + Au
Assuming this function to he differentiable with � tG u; We
;
�y list
:::. �"' :
.
PART 3: OPTIMAL CONTROL THEORY
192
the following conditions by the maximum principle:
aH
au
- =F + A = O
(7.56)
u
aH
aA
aH
ay
y= - =u
A=
A( T ) = 0
The first equation in (7.56) can be rewritten as A
- Fu . But in view of the
second equation therein, it can be further rewritten as
=
A = - F:r
( 7 .57)
Differentiation of (7.57) with respect to t yields
d
.
A = - - F.
dt y
However, the third equation in (7.56) gives another expression for
equating the two expressions, we end up with the single condition
A.
By
d
F - - F. = 0
y
dt y
which is identical with the Euler equation (2. 18).
When the Hamiltonian is maximized with respect to u, the condition
aH;au should be accompanied by the second-order necessary condition
a 2Hjau 2 � 0. Further differentiation of the aH;au expression in (7.56)
yields
a 2H
F
F
-au 2 = u u = yy
�
0
This, of course, is the Legendre necessary condition. Thus the maximum
principle is perfectly consistent with the conditions of the calculus of
variations.
Now, let us take a look at the transversality conditions. For a control
problem with a vertical terminal line, the transversality condition is A(T ) =
0. By (7.57), however, this may be written as [ - F:rlt - T = 0, or, equivalently,
{ F:rLr = o
Again, this is precisely the transversality condition in the calculus of
variations presented in (3.10).
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
193
For the problem with a horizontal terminal line, the optimal-control
transversality condition is [ H l1_r = 0 . In view of (7.55), this means
[ F + A u lt-T = 0. Using (7.56) again and after substituting y for u , how­
ever, we can transform this condition into
[ F - F-:Y ]
Y
t-T
=
0
Except for the slight difference in symbology, this is precisely the transver­
sality condition under the calculus of variations given in (3. 1 1).
It can also be shown that the transversality condition for the problem
with a terminal curve Yr = cf>(T ) under optimal control theory can be
translated into the corresponding condition under the calculus of variations,
and vice versa. The details of the demonstration are, however, left to the
reader.
7.6
THE POLITICAL BUSINESS CYCLE
Applications of the maximum principle to problems of economics mush­
roomed between 1965 and 1975, and the technique has become fairly
common. Its applications range from the more standard areas in macro- and
microeconomics all the way to such topics as fishery, city planning, and
pollution control. In the present section, we introduce an interesting model
of William Nordhaus,l3 which shows that, in a democracy, attempts by an
incumbent political party to prevent its rival party (or parties) from ousting
it from office may encourage the pursuit of economic policies that result in a
particular time profile for the rate of unemployment and the rate of
inflation within each electoral period. Over successive electoral periods, the
repetition of that pattern will then manifest itself as a series of business
cycles rooted solely in the play of politics.
The Vote Function and the Phillips
Tradeoff
The incumbent party, in control of the national government, is obliged in a
democracy to pursue policies that appeal to a majority of the voters in order
to keep on winning at the polls. In the present model, attention is focused
on economic policies only, and in fact on only two economic variables: U
(the rate of unemployment) and p (the rate of inflation). Since the ill effects
of unemployment and inflation seem to have been the primary economic
concerns of the electorate, this choice of focus is certainly reasonable. The
reaction of the voters to any realized values of U and p is assumed to be
13
William D. Nordhaus, " The Political Business Cycle," Review of Economic Studies, April
1975, pp. 169-190.
� .. ,.
'
.
�:� '
PAJtT s, OPTIMAL CONTRoL THEORY
194
p(%)
vote function
embodied in an (aggregate)
u = u( U, p)
( 7 .58)
where u is a measure of the vote-getting power of the incumbent party. The
partial derivatives of u with respect to each argument are negative, because
high values of U and p are both conducive to vote loss. This fact is reflected
in Fig. 7.6, where, out of the three isouote curves illustrated, the highest
one is associated with the lowest u. The notion of the isovote curve
underscores the fact that, on the political side, there is a tradeoff between
the two variables U and p. If the incumbent party displeases the voters by
producing higher inflation, it can hope to recoup the vote loss via a sufficient
reduction in the rate of unemployment.
Aside from the political tradeoff, the two variables under consideration
are also linked to each other in an economic tradeoff via the expectations­
augmented Phillips relation
( 7 .59)
p
=
cp ( U )
+
( q/ < 0, 0
arr
< a ::S;
1)
where rr denotes the expected rate of inflation. Expectations are assumed to
be formed adaptively, according to the differential equation
( 7 .60)
7T
= b( p -
rr
)
( b > 0)
All in all, we now have three variables, U, p, and rr. But which of these
should be considered as state variables and which as control variables? For a
variable to qualify as a state variable, it must come with a given equation of
motion in the problem. Since (7.60) constitutes an equation of motion for rr,
we can take rr as a state variable. The variable U, on the other hand, does
not come with an equation of motion. But since U can affect p via (7.59)
and then dynamically drive rr via (7.60), we can use it as a control variable.
To use U as a control variable, however, requires the implicit assumption
that the government in power does have the ability to implement any target
rate of unemployment it chooses at any point of time. As to the remaining
·
195
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
variable, p, (7.59) shows that its value at any time t will become determi­
nate once the values of the state and control variables are known. We may
thus view it as neither a state variable nor a control variable, but, like
just a function of the other two variables.
v,
The Optimal Control Problem
Suppose that a party has just won the election at t 0, and the next
election is to be held T years later at t T. The incumbent party then has
a total of T years in which to impress the voters with its accomplishments
(or outward appearances thereoD in order to win their votes. At any time in
the period [0, T ], the pair of realized values of U and p will determine a
Such values of v for different points of time must all
specific value of
enter into the objective functional of the incumbent party. But the various
values may have to be weighted differently, depending on the time of
occurrence. If the voters have a short collective memory and are influenced
more by the events occurring near election time, then the v values of the
later part of the period [0, T ] should be assigned heavier weights than those
that come earlier. We may then formulate the optimal control problem of
the incumbent party as follows:
=
=
v.
Maximize
( 7.61)
subject to
and
j rv ( U, p )
0
e
'1
dt
p = ¢( U ) + a1r
7T = b( p - 7r )
' ,; 1
.:
1r(T ) free
A few comments on (7.61) may be in order. First, the weighting system
for the
values pertaining to different points of time has been given the
specific form of the exponential function e '1, where r > 0 denotes the rate
of decay of memory. This function shows that the values at later points of
time are weighted more heavily. Note that, in contrast to the expression
- pt
e
, what we have here is not a discount factor, but its reverse. Second, we
have retained the expectations-augmented Phillips relation in the problem
statement. At the moment, however, we are not yet equipped to deal with
such a constraint. Fortunately, the variable p can easily be eliminated by
substituting that equation into the vote function and the equation of
motion. Then the p equation will disappear as a separate constraint. Third,
as the boundary conditions indicate, the incumbent party faces a vertical
terminal line, since T (election time) is predetermined. Fourth, although the
rate of unemployment is perforce nonnegative, no nonnegativity restriction
has actually been placed on the control variable U. The plan-and this is an
oft-used strategy-is to impose no restriction and just let the solution fall
v
v
196
PART 3: OPTIMAL CONTROL THEORY
out however it may. If U*(t) turns out to have economically acceptable
values for all t, then there is no need to worry; if not, and only if not, we
shall have to amend the problem formulation.
As stated in (7.61), the problem contains general functions, and thus
cannot yield a quantitative solution. To solve the problem quantitatively,
Nordhaus assumes the following specific function forms:
( 7.62)
( 7.63)
v( U, p) = - U2 - hp
p = (j - kU) + arr
( h > 0)
(j, k > 0, 0 < a � 1)
From (7.62), it can be seen that the partial derivatives of v are indeed
negative. In (7.63), we find that the Phillips relation l/>(U) has been lin­
earized. Using these specific functions, and after substituting (7.63) into
(7 .62), we now have the specific problem:
Maximize
( 7.64)
subject to
and
{(
- U2 - hj
0
+
hkU - harr)e'' dt
1T
= b(j - kU - ( 1 - a )rr]
rr( T ) free
rr(O) = rr0
Maximizing the Hamiltonian
.. ·
The Hamiltonian is
( 7 .65)
H = ( - U2 - hj
+
hkU - ha1T)e''
+
Ab [ j - kU - ( 1 - a)rr]
Maximizing H with respect to the control variable
i
U, we have the equation
aH
( - 2 U + hk )e'' - ltbk = o
au =
This implies the control path
( 7 .66)
U( t) = �k(h - Abe-'')
Since a2HjaU2 = - 2e71 < 0, the control path in (7.66) indeed maximizes
the Hamiltonian at every point of time, as the maximum principle requires.
The presence of A in the U(t) solution now necessitates a search for
the A(t) path.
The Optimal Costate Path
The search for the costate path begins with the equation of motion
aH
.
ll = - - = hae'1 + ..\b(l - a )
arr
197
CHAPTER 7: OPTIMUM CONTROL: THE MAXIMUM PRINCIPLE
When rewritten into the form
A - b(l - a ) A = hae rt
the equation is readily recognized as a first-order linear differential equation
with a constant coefficient but a variable term. Employing the standard
methods of solution,14 we can find the complementary function Ac and the
particular integral X to be, respectively,
( A arbitrary)
A c = Ae b( l - a )t
_
A=
ha
- e rt
It follows that the general solution for
( 7 .67)
A(t)
= r -
(B
B
A
is
_
=
b + ab )
Ac + A = Aeh<l -a)t
+
ha
ert
B
Note that the two constants A and B are fundamentally different in
nature; B is merely a shorthand symbol we have chosen in order to simplify
the notation, but A is an arbitrary constant to be definitized.
To definitize A, we can make use of the transversality condition for
the vertical-terminal-line problem, A( T ) 0. Letting t = T in (7.67), apply­
ing the transversality condition, and solving for A, we find that A
( - hajB)e 8 T. It follows that the definite solution-the optimal costate path
-is
=
=
( 7.67' )
A* ( t ) =
ha
- [ e rl - e B T + b(l - a )l
]
B
The Optimal Control Path
Now that we have found A*(t), all it takes is to substitute (7.67' ) into (7.66)
to derive the optimal control path. The result is, upon simplification,
( 7 . 68)
U*( t )
kh
=
- (( r 2B
b ) + bae B(T -1)]
It is this control path that the incumbent party should follow in the interest
of its reelection in year T.
14See Alpha C. Chiang, Fundamental Methods of Mathematical Economics, 3d ed., McGraw­
Hill, New York, 1984, Sec. 14.1 (for the complementary function) and Sec. 15.6 (for the
particular integral).
198
PART 3o OPTIMAL CONTROL THEORY
What are the economic implications of this path? First, we note that
U* is a decreasing function of t. Specifically, we have
dU*
dt
( 7 69)
--
.
=
1
- - khbae 8<T -t)
2
<
0
because k, h, b, and a are all positive, as is the exponential expression. The
vote-maximizing economic policy is, accordingly, to set a high unemploy­
ment level immediately upon winning the election at t 0, and then let the
rate of unemployment fall steadily throughout the electoral period [0, T ]. In
fact, the optimal levels of unemployment at time 0 and time T can be
exactly determined. They are
=
U * ( O)
=
U* ( T )
=
!
' •,
-; ·.}·
. . ...
,. /.•:::. ' , ,
kh
- [( r - b) + bae 8T]
2B
kh
kh
[ ( r - b) + ba ] = 2
2B
Note that the terminal unemployment level, khj2, is a positive quantity.
And since U*(T) represents the lowest point on the entire U*(t) path, the
U* values at all values of t in [0, T ] must uniformly be positive. This means
that the strategy of deliberately not imposing any restriction on the variable
U does not cause any trouble regarding the sign of U in the present case.
However, to be economically meaningful, U*(O) must be less than unity or,
more realistically, less than some maximum tolerable unemployment rate
Umax < 1. Unless the parameter values are such that U*(O) ::::;; Umax• the
model needs to be reformulated by including the constraint U(t) E [0, umaxl·
The typical optimal unemployment path, U* (t), is illustrated in Fig.
7.7, where we also show the repetition of similar U*(t) patterns over
successive electoral periods generates political business cycles. However, the
curvature of the U*(t) path does not always have to be concave as in Fig.
7. 7. For, by differentiating (7.69) with respect to t, we can see that
d 2 U*
dt 2
( 7.70)
--
=
1
0
Bkhbae B<T - t ) :2:
< .
2
-
88
;''
T
2T
3T
�:
FIGtJIU!: 7.7
CHAPTER 7: OPTIMUM CONTROL : THE MAXIMUM PRINCIPLE
r = 0.03, b
0.30, and a = 0.70, then B = r - b + ab
0.06, and the U * (t) path will be concave. But a positive value of B will
If, for illustration,
=
-
199
=
turn the curvature the other way. And, with parameter changes in different
electoral periods, the position as well as the curvature of the U * (t) paths in
succeeding electoral periods may very well change. Nevertheless, the politi­
cal business cycles will tend to persist.
The Optimal State Path
The politically inspired cyclical tendency in the control variable
spill over to the state variable
7T,
U
must also
and hence also to the actual rate of
inflation p. The general pattern would be for the optimal rate of inflation to
be relatively low at the beginning of each electoral period, but undergo a
steady climb. In other words, the time profile of p* tends to be the opposite
of that of U * . But we shall not go into the actual derivation of the optimal
inflation rate path here.
The reader is reminded that the conclusions of the present model-like
those of any other model-are intimately tied to the assumptions adopted.
In particular, the specific forms chosen for the vote function in (7.62) and
the expectations-augmented Phillips relation (7.63) undoubtedly exert an
important influence upon the final solutions. Alternative assumptions-such
as changing the linear term - hp in (7.62) to
hp
are likely to modify
significantly the
U *(t)
solution
as
-
2
-
well as the p * ( t ) solution. But the
problem formulation is also likely to become much more complicated .
EXERCISE 7.6
1
(a) What would happen in the Nordhaus model if the optimal control path
were characterized by dU* jdt = 0 for all t?
(b) What values of the various parameters would cause dU* jdt to become
zero?
(c) Interpret economically the parameter values you have indicated in part
(b).
2 What parameter values would, aside from causing dU*jdt 0, also cause
U *(t) = 0 for all t? Explain the economic implications and rationale for
=
,_ . ,
such an outcome.
3
How does a change in the value of parameter r (the rate of decay of voter
memory) affect the slope of the U*(t) path? Discuss the economic
implications of your result. [ Note: B = r - b + ab.]
4 Eliminate the e '1 term in the objective function in (7.64) and write out the
new problem.
(a) Solve the new problem by carrying out the same steps
illustrated in the text for the original problem.
as
those
PART 3, OPTIMAL CONTROL THEORY
200
(b)
7.7
Check your results by setting r = 0 in the results of the original model,
and (7.69).
especially (7.68)
ENERGY USE AND
ENVIRONMENTAL QUALITY
When an economy is faced with an essential resource that is exhaustible,
say, fossil fuel, it certainly behooves its citizenry to be concerned about the
question of how the limited supply of the resource is best to be allocated for
use over time. We have discussed some of the issues involved in Sec. 6.3
with the method of the calculus of variations. But the citizens of the
present-day world are also intensely concerned about the quality of the
environment in which they live. If the use of the exhaustible fuel generates
pollution as a by-product, then what is the optimal time path for energy
use? We shall illustrate how such a question can be tackled by optimal
control theory with a model of Bruce
A.
Forster.15
The Social Utility Function
Let
S(t) denote the stock of the fuel and E(t) the rate of extraction of fuel
t. Then we have
(and energy use) at any time
S
( 7 . 71)
= -E
Energy use, E , makes possible the production o f goods and services for
consumption, C, which creates utility, but also generates a flow of pollution,
P, which creates disutility. Instead of writing a simple utility function U( E )
a s we did i n the introductory section o f this chapter, therefore, our new
utility function should contain two arguments, C(E) and P ( E ). Forster
specifies the consumption function and the pollution function as follows:
( 7 . 72 )
C
( 7 .73)
P
=
=
C( E )
( C'
0, C "
<
0)
P( E )
( P' > 0, P"
>
0)
>
While energy use raises consumption at a decreasing rate, it generates
pollution at an increasing rate. In this particular model, pollution is as­
sumed for simplicity to be nonaccumulating; that is, it is a flow that
16
Bruce A. Forster, "Optimal Energy Use in a Polluted Environment," Journal of Environ­
mental Economics and Management, 1980, pp. 321-333. While this paper presents three
different models, here we shall confine our attention exclusively to the first one, which assumes
a single energy source producing a nonaccumulating pollutant. Another model, treating
pollution as a stock variable and involving two state variables, will be discussed in Sec. 8.5.
CHAPTER 7, OPTIMUM CONTROL, THE MAXIMUM PRINCIPLE
201
dissipates and does not build up into a stock. This is exemplified by the
auto-emission type of pollution.
The social utility function depends on consumption and pollution, with
derivatives as follows:
( 7 . 74)
U
=
U(C, P)
( Uc > 0 , Up < 0, Ucc < 0, Upp < 0 , Ucp
=
0)
The specification of Uc > 0 and Ucc < 0 shows that the marginal utility of
consumption is positive but diminishing. In contrast, the specification of
Up < 0 and Upp < 0 reveals that the marginal utility of pollution is negative
and diminishing (given a particular increase in P, Up may decrease from,
say, - 2 to - 3). In terms of the marginal dis utility of P( = - Up), there­
fore, Upp < 0 signifies increasing marginal disutility.
Since both C and P in turn depend on
the social utility hinges, in
the final analysis, on energy use alone-positively via consumption and
negatively via pollution. This means that C and
can both be substituted
out, leaving E as the prime candidate for the control variable. The only
other variable in the model, S , appears in (7.71) in the derivative form.
Since it is a variable dynamically driven by the control variable
it is clear
that S plays the role of a state variable here.
E,
P
E,
The Optimal Control Problem
If an Energy Board is appointed to plan and chart the optimal time path of
the energy-use variable
over a specified time period [0, T ], the dynamic
optimization problem it must solve may take the form
E
subject to
fu[C(E),
P(E)] dt
0
B = -E
and
S(O)
Maximize
( 7.75)
=
S0
S( T ) ;:::.:. 0
( S0, T given)
This particular formulation allows no discount factor in the integrand, a
practice in the Ramsey tradition. And it grants the Energy Board the
discretion of selecting the terminal stock S(T), subject only to the nature­
imposed restriction that it be nonnegative. Since the terminal time is fixed,
a truncated vertical terminal line characterizes the problem. With a single
control variable E and a single state variable S, the problem can be solved
with relative ease.
Maximizing the Hamiltonian
The Hamiltonian function
( 7 .76)
H = U(C( E ), P(E)] - A.E
202
PART 3: OPTIMAL CONTROL THEORY
involves nonlinear differentiable functions U, C, and P. Thus we may
maximize H with respect to the control variable simply by setting its first
derivative equal to zero:
aH
aE
(7.77)
=
Uc C'( E)
+
UpP'( E) - A = 0
When solved, this equation expresses E in terms of A.
To make sure that (7.77) maximizes rather than minimizes the Hamil­
tonian, we check the sign of a 2HjaE 2 . Since Uc and Up are, like U,
dependent on E, the second derivative is
a 2H
-2 = U.cc C' 2 + U.c C" + UPP P'2
aE
UP P" < 0
(by (7.72) , (7.73) ; and (7.74)]
+
Its negative sign guarantees that (7.77) does maximize
H.
The Optimal Costate and Control Paths
To elicit more information about E from (7.77), however, we need to look
into the time path of A. The maximum principle tells us that the equation of
motion for A is
(7.78)
aH
.
A = -- = 0
as
implying
A(t) = c ( constant)
To definitize the constant c, we can resort to the transversality condition.
For the problem at hand, with a truncated vertical terminal line, the
condition takes the form
(7.79) A( T ) ;:::: 0
A( T ) S ( T ) = 0
S ( T ) ;:::: 0
( by (7.35)]
In practical applications of this type of condition, the standard initial step is
to set A(T) = 0 (as if the terminal line is not truncated) to see whether the
solution will work. Since A(t) is a constant by (7. 78), to set A(T) = 0 is in
effect to set A(t) = 0 for all t.
With A(t) = 0, (7.77) reduces to an equation in the single variable E,
(7.80 )
Uc C'( E)
+
UpP'( E) = 0
which, in principle, can be solved for the optimal control path. Since this
equation is independent of the variable t, its solution is constant over time:
(7 .81 )
E* ( t) = E* ( a specific constant)
( if A*(t) = 0]
Whether this solution is acceptable from the standpoint of the S(T) �
restriction is, however, still a matter to be settled.
0
'
j '
'
l
j
203
CHAPTER 7: OPTrMUM CONTROL: THE MAXIMUM PRINCIPLE
Meanwhile, it is useful to examine the economic meaning of (7.80).
The first term, UcC'( E), measures the effect of a change in E on U via C.
That is, it represents the marginal utility of energy use through its contri­
bution to consumption. Similarly, the Up P'( E) term expresses the marginal
disutility of energy use through its pollution effect. What (7.80) does is,
therefore, to direct the Energy Board to select a E* value that balances the
marginal utility and disutility of energy use, much as the familiar MC = MR
rule requires a firm to balance the cost and revenue effects of production.
The Optimal State Path
It remains to check whether the E* solution in (7.81) can satisfy the
S ( T ) � 0 restriction. For this purpose, we must find the state path S(t).
With constant energy use, the equation of motion S
- E can be
readily integrated to yield
=
S(t)
=
- Et
+
k
( k arbitrary)
Moreover, by setting t 0 in this result, it is easily seen that k represents
the initial fuel stock S0• Thus the optimal state path can be written as
=
( 7.82)
S*(t)
=
S0 - E*t
The value of S*(t) at any time clearly hinges on the magnitude of E*.
Since the functions we have been working with-U(C, P), C(E), and P(E )
-are all general functions, E * cannot be assigned a specific numerical
value or parametric expression. Nonetheless, we can examine the S(T) � 0
restriction qualitatively.
Consider the three illustrative values of E* in Fig. 7.8, where E* 1 <
E* 2 < E* 3 • When the low rate of energy use E\ is in effect, the optimal
stock S* (t) appears as a gently downward-sloping straight line, such that
S * ( T ) is positive. With the higher rate of energy use E* 2, on the other
hand, the fuel stock is brought down to zero at time T. Even so, the Energy
Board would still be within the bounds of its given authority. But the other
case, E * 3 , entailing the premature exhaustion of the fuel endowment, would
patently violate the S(T) � 0 stipulation. Thus, if our solution E* in (7.81)
s•<tl
S*(tl 80 - E"t
(E•1 < E•2 < E•3)
=
204
PART 3 : OPTIMAL CONTROL THEORY
turns out to be like E* 1 or E* 2, then the transversality condition (7. 79) is
met, and the problem solved. But if it is like E* 3, then we must set
S(T) = 0, and solve the problem as one with a given ter.minal point. In that
event, the E* value can be directly found from (7.82) by setting t = T and
S(T) = 0:
( 7 .83)
S
E* = o
T
[ if ( 7 .81) violates S * ( T )
�
0]
This new E* can be illustrated by E* 2 in Fig. 7.8.
It is a notable feature of this model that E*, the optimal rate of energy
use, is constant over time. This constancy of E * prevails whether the
terminal-stock restriction, S(T) � 0, is binding [as in (7.83)] or nonbinding
[as in (7.81)]. What assumption in the model is responsible for this particu­
lar result? The answer lies in the absence of a discount factor. If a discount
factor is introduced [see Prob. 3 in Exercise 7.7], the E* path then will turn
out to be decreasing over time, provided that A*(t) > 0. However, in the
other case where A*(t) = 0, E* will still be constant.
EXERCISE 7. 7
1
Suppose that the solution of (7.80) turns out to be E*3, which fails to
satisfy the S(T) 2: 0 restriction, and consequently the Energy Board is
forced to select the lower rate of energy use, E* 2, instead.
(a) Does E* 3 satisfy the " marginal utility = marginal disutility" rule?
(b) Does E*2 satisfy that rule? If not, is E*2 characterized by " marginal
utility < marginal disutility" or " marginal utility > marginal
disutility"? Explain.
2 Let the terminal condition in the Forster model be changed from
S ( T ) 2: 0
to S(T) 2: Smin > 0. How should Fig. 7.8 be modified to show that E*1
results in S(T) > Smin• E*2 results in S(T) = Smin• and E*3 results in
S(T) < Smin?
3 Suppose that the Energy Board decides to incorporate a discount factor
e - pt into the objective functional.
(a) Write the new Hamiltonian, and find the condition that will maximize
the new Hamiltonian.
(b) Examine the optimal costate path. Do you still obtain a constant A
path as in (7. 78)?
(c) If the transversality condition A(T) = 0 applies, what will become of
the H-maximizing condition in part (a)? Can the condition be simplified
to (7.80)? What can you conclude about E* for this case?
(d) If the transversality condition is A(T) > 0 and S(T ) = 0 instead, what
will become of the H-maximizing condition in part (a)? Find the
derivative dE jdt and deduce the time profile of the E*(t} path for this
case.
CHAPTER
8
� •·
��'. >
.·.;
MORE
ON
OPTIMAL
CONTROL
For a better grasp of mathematical methods and results, it is always helpful
to understand the intuitive economic meanings behind them. For this
reason, we shall now seek to buttress our mathematical understanding of
the maximum principle by an economic interpretation of the various condi­
tions it imposes. Mter that is accomplished, we shall then move on to some
other aspects of optimal control theory such as the current-value Hamilto­
nian, concavity sufficient conditions, and problems with multiple state and
control variables.
8.1
ECONOMIC INTERPRETATION
OF THE MAXIMUM PRINCIPLE
AN
In a remarkable article, Robert Dorfman shows that each part of the
maximum principle can be assigned an intuitively appealing economic mean­
ing, and that each condition therein can be made plausible from a common­
sense point of view.1 We shall draw heavily from that article in this section.
1
Robert Dorfman,
Economic Review,
"An
Economic Interpretation of Optimal Control Theory,"
American
December 1969, pp. 817-831.
205
206
PART 3, OPTIMAL CONTROL THEORY
Consider a firm that seeks to maximize its profits over the time
interval [0,
There is a single state variable, capital stock
And there is
a single control variable
representing some business decision the firm has
T].
K.
u,
to make at each moment of time (such as its advertising budget or inventory
policy). The firm starts out at time 0 with capital
but the terminal
capital stock is left op�m. At any moment of time, the profit of the firm
K0,
depends on the amount of capital it currently holds
·
as
well
as
on the policy
u it currently selects. Thus, its profit function is 7T(t, K, u). But the policy
selection u also bears upon the rate at which capital K changes over time;
that is, K is affected by u. It follows that the optimal control problem is to
n = iT7T(t,
K, u) dt
0
Maximize
(8.1)
'' ,_� �. .· ·
subject to
K = f(t, K, u )
K(O) K0
and
=
The Costate Variable
as
K( T) free
( K0, T given)
a Shadow Price
The maximum principle places conditions on three types of variables:
u
K
control, state, and costate. The control variable
and the state variable
have already been assigned their economic meanings. What about the
costate variable >.?
As
intimated in an earlier section, >. is in the nature of a Lagrange
multiplier and, as such, it should have the connotation of a shadow price. To
confirm this, let us adapt (7 .22") to the present context, and plug in the
optimal paths or values for all variables, to get
n*
=
f[ H( t, K * , u* , A*) + K* (t)A* j dt - A* (T)K*(T) + A* (O)K0
0
n*
Partial differentiation of
with respect to the (given) initial capital and
the (optimal) terminal capital yields
(8.2)
an *
aK0 = A*(O)
and
an*
aK*(T) = - >.*(T)
Thus, >.*(0), the optimally determined initial costate value, i s a measure of
the sensitivity of the optimal total profit
to the given initial capital. If
we had one more (infinitesimal) unit of capital initially, ll * would be larger
n•
by the amount >.*(0). Therefore, the latter expression can be taken as the
imputed value or shadow price of a unit of initial capital. In the other
partial derivative in
(8.2),
the terminal value of the optimal costate path,
207
CHAPTER 8: MORE ON OPTIMAL CONTROL
A*(T), is seen to be the negative of the rate of change of IT* with respect to
the optimal terminal capital stock. If we wished to preserve one more unit
(use up one less unit) of capital stock at the end of the planning period, then
we would have to sacrifice our total profit by the amount A*(T). So, again,
the A* value at time T measures the shadow price of a unit of the terminal
capital stock.
In general, then, A* (t) for any t is the shadow price of capital at that
particular point of time. For this interpretation to be valid, however, we
have to write the Lagrange-multiplier expression as A(t)[ f(t, K, u ) - K }
rather than A(t)[ K f(t, K, u )]. Otherwise, A* would become instead the
negative of the shadow price of K.
-
The Hamiltonian and the Profit Prospect
The Hamiltonian of problem ( 8.1) is
( 8.3)
H = 1r(t, K, u ) + A ( t ) f( t, K , u )
The first component o n the right is simply the profit function at time t,
based on the current capital and the current policy decision taken at that
time. We may think of it as representing the "current profit corresponding
to policy u . " In the second component of (8.3), the f(t, K, u ) function
indicates the rate of change of (physical) capital, K, corresponding to policy
u , but when the f function is multiplied by the shadow price ,.\(t), it is
converted to a monetary value. Hence, the second component of the Hamil­
tonian represents the " rate of change of capital value corresponding to
policy u . " Unlike the first term, which relates to the current-profit effect of
u, the second term can be viewed as the future-profit effect of u, since the
objective of capital accumulation is to pave the way for the production of
profits for the firm in the future. These two effects are in general competing
in nature: If a particular policy decision u is favorable to the current profit,
then it will normally involve a sacrifice in the future profit. In sum, then,
the Hamiltonian represents the overall profit prospect of the various policy
decisions, with both the immediate and the future effects taken into ac­
count.2
The maximum principle requires the maximization of the Hamiltonian
with respect to u . What this means is that the firm must try at each point
2
Note that if the Hamiltonian is written out more completely as H
A(t)f(t, K, u) as in (7.4),
=
A01r(t, K, u)
+
and if it turns out that A0 = 0 , then this would mean economically
that the current-profit effect of u somehow does not matter to the firm, and does not have to
be weighed against the future-profit effect. Such an outcome is intuitively not appealing. In a
meaningful economic model, we would expect
normalized to one.
A0
to
be nonzero, in which
case A 0 can
be
208
PART 3: OPTIMAL CONTROL THEORY
of time to maximize the overall profit prospect by the proper choice of u .
Specifically, this would require the balancing of prospective gains in the
current profit against prospective losses in the future profit. To see this
more clearly, examine the weak version of the " Max H " condition:
iJH iJ7T
of
- = - + A(t) - = 0
iJu
ilu
au
When it is rewritten into the form
·' �- � -
.
iJ f
-A(t) ­
au
(8.4)
this condition shows that the optimal choice u * must balance any marginal
increase in the current profit made possible by the policy [the left-hand-side
expression in (8.4)} against the marginal decrease in the future profit that
the policy will induce via the change in the capital stock [the right-hand-side
expression in (8.4)}.
The Equations of Motion
The maximum principle involves two equations of motion. The one for the
state variable K, included as part of the problem statement (8.1) itself,
merely specifies the way the policy decision of the firm will affect the rate of
change of capital. The equation of motion for the costate variable is
iJ f
iJ7T
iJH
.
A = - - = - - - A(t) aK
aK
iJK
or,
after multiplying through by
(8.5)
-
1
,
' . •'/:/
07T
iJ f
- A = - + A( t ) aK
iJK
•
The left-hand-side of (8.5) denotes the rate of decrease of the shadow price
over time, or the rate of depreciation of the shadow price. The equation of
motion requires this rate to be equal in magnitude to the sum of the two
terms on the right-hand side of (8.5). The first of these, iJ1r jiJK, represents
the marginal contribution of capital to current profit, and the second,
A(iJ fjiJK), represents the marginal contribution of capital to the enhance­
ment of capital value. What the maximum principle requires is that the
shadow price of capital depreciate at the rate at which capital is contributing
to the current and future profits of the firm.
209
CHAPTER 8: MORE ON OPTIMAL CONTROL
Transversality Conditions
What about transversality conditions? With a free terminal state K(T ) at a
fixed terminal time T (vertical terminal line), that condition is
A( T ) = 0
This means that the shadow price of capital should be driven to zero at the
terminal time. The reason for this is that the valuableness of capital to the
firm emanates solely from its potential for producing profits. Given the rigid
planning horizon T, the tacit understanding is that only the profits made
within the period [0, T] would matter, and that whatever capital stock that
still exists at time T, being too late to be put to use, would have no economic
value to the firm. Hence, it is only natural that the shadow price of capital
at time T should be set equal to zero. In view of this, we would not expect
the firm to engage in capital accumulation in earnest toward the end of the
planning period. Rather, it should be trying to use up most of its capital by
time T. The situation is not unlike that of a pure egotist-a sort of Mr.
Scrooge-who places no value on any material possessions that he himself
cannot enjoy and must leave behind upon his demise.
For a firm that intends to continue its existence beyond the planning
period [0, T ], it may be reasonable to stipulate some minimum acceptable
level for the terminal capital, say, K min · In that case, of course, we would
have a truncated vertical terminal line instead. The transversality condition
now stipulates that
A( T ) � 0
and
[ K * ( T ) - K min ] A ( T )
=
0
[ from ( 7 .35)]
If K*(T ) turns out to exceed K min • then the restriction placed upon the
terminal capital stock proves to be nonbinding. The outcome is the same as
if there is no restriction, and the old condition A(T ) = 0 will still apply. But
if the terminal shadow price A(T ) is optimally nonzero (positive), then the
restriction K min indeed is binding, in the sense that it is preventing the firm
from using up as much of its capital toward the end of the period as it would
otherwise do. The amount of terminal capital actually left by the firm will
therefore be exactly at the minimum required level, K min ·
Finally, consider the case of a horizontal terminal line. In that case,
the firm has a prespecified terminal capital level, say K ro• but is free to
choose the time to reach the target. The transversality condition
simply means that, at the (chosen) terminal time, the sum of the current
and future profits pertaining to that point of time must be zero. In other
words, the firm should not attain K ro at a time when the sum of immediate
and future profits (the value of H) is still positive; rather, it should-after
210
PART 3, 0PTUMAL CONTROL THEORY
having taken advantage of all possible profit opportunities-attain Kro at a
time when the sum of immediate and future profits has been squeezed down
to zero.
8.2
THE CURRENT-VALUE
HAMILTONIAN
,,;
In economic applications of optimal control theory, the integrand function
F often contains a discount factor e-pt. Such an F function can in general
be
expressed as
F( t, y, u) = G(t, y, u ) e - p1
(8.6)
so that the QPti:mal control problem is to
·,
'
Maximize
(8.7)
subject to
:Y = f( t, y, u )
and
boundary conditions
By the standard definition, the Hamiltonian function takes the form
H = G(t,y, u )e-p1 + A {(t,y, u )
(8.8)
·"
But since the maximum principle calls for the differentiation of H with
respect to u and y, and since the presence of the discount factor adds
complexity to the derivatives, it may be desirable to define a new Hamilto­
nian that is free of the discount factor. Such a Hamiltonian is called the
current-value Hamiltonian, where the term "current-value" (as against
" present-value") serves to convey the "undiscounted" nature of the new
Hamiltonian.
The concept of the current-value Hamiltonian necessitates the com­
panion concept of the current-value Lagrange multiplier. Let us therefore
first define a new (current-value) Lagrange multiplier m :
(implying A = me-P1)
{8.9)
Then the current-value Hamiltonian, denoted by
(8.10)
He = He P1 = G(t, y, u ) + mf(t,y, u )
As intended,
(8.10')
He, can be written as
[ by (8.8) and {8.9)]
He is now free of the discount factor. Note that (8.10) implies
'f
211
CHAPTER 8: MORE ON OPTIMAL CONTROL
The Maximum Principle Revised
If we choose to work with
H.
H, then all the conditions of the
instead of
maximum principle should be reexamined to see whether any revision is
needed.
The first condition in the maximum principle is to maximize
respect to
u
H
with
at every point of time. When we switch to the current-value
H. = HeP1, the condition is essentially unchanged except for
H. for H. This is because the exponential term eP1 is a
constant for any given t. The particular
that maximizes H will therefore
also maximize H Thus the revised condition is simply
for all t E [0, T ]
(8.11)
Hamiltonian
the substitution of
u
•.
The equation of motion for the state variable originally appears in the
canonical system as j
= iJHjiJA.
(8.8) and (8.10)], this equation
(8.12)
Since
iJHjiJA = f(t, y, u ) iJH0jiJm
=
[by
should now be revised to
aH.
y. = am
[ equation o f motion for y]
T o revise the equation o f motion for the costate variable,
,\
=
-aHjoy,
we shall first transform each side of this equation into an expression
m,
involving the new Lagrange multiplier
and then equate the two results.
For the left-hand side, we have, by differentiating
(8.9),
H in (8.10'), we can rewrite the right-hand side as
aH
aH
- __c e -pt
iJy
y
iJ
Upon equating these two results and canceling the common factor e-pt, we
Using the definition of
arrive at the following revised equation of motion:
( 8.13)
aH.
m. = - -- + pm
ay
[ equation of motion for
m]
A,
Note that, compared with the original equation of motion for
the new one
for m involves an extra term,
It remains to examine the transversality conditions. We shall d o this
pm.
for the vertical terminal line and the horizontal terminal line only. For the
former, we can deduce that
(8.14)
A{T) = 0
[ me -p1]1-T = 0
m ( T ) e -pT = 0
[ by ( 8.9) }
PART 8: OPTIMAL CONTROL THEORY
212
For the latter, similar reasoning shows that
[ He e-P1] t - T = 0
(8.15)
[ by (8.10')]
Autonomous Problems
As a special case of problem (8. 7), both the G and f functions may contain
no t argument; that is, they may take the forms
.. , .
G = G(y, u )
and
· Then the problem becomes
f = f(y, u )
Maximize
( 8.16)
subject to
:Y = f(y, u )
and
boundary conditions
,
;..
__
Since the integrand G(y, u)e -P1 still explicitly contains t, the problem is,
strictly speaking, nonautonomous. However, by using the current-value
Hamiltonian, we can in effect take the discount factor e -pt out of considera­
tion. It is for this reason that economists tend to view problem (8.16) as an
autonomous problem-in the special sense that the t argument explicitly
enters in the problem via the discount factor only.
The current-value Hamiltonian is, of course, as applicable to problem
(8.16) as it is to (8.7). And all the revised maximum-principle conditions
(8.11) to (8.15) still hold. But the current-value Hamiltonian of the au­
tonomous problem (8.16) has an additional property not available in prob­
lem (8.7). Since He now specializes to the form
He = G(y, u ) + mf (y, u )
which is free of the t argument, its value evaluated along the optimal paths
'of all variables must be constant over time. That is,
(8.17)
dH*
c
=0
dt
__
or
He* = constant
[autonomous problem]
This result is, of course, nothing but a replay of (7.54).
Another View of the Eisner-Strotz Model
Consider the following autonomous control problem adapted from the
Eisner-Strotz model, originally studied as a calculus-of-variations problem
CHAPTER s, MORE ON OPTIMAL CONTROL
2 13
with an infinite horizon in Sec. 5.2:
Maximize
subject to
( 8.18)
and
for[1r(K)' .:... C( l)]e-P' dt
K=I
boundary conditions
The meanings of the symbols are: 'll" profit,
capital stock,
=
adjustment cost, and
net investment. The only state variable is
and
the only control variable is The and functions have derivatives
I=
I.
C'(I) > 0
1r"(K) < 0
1r
and
=
K=
C
C"(I) > 0
K,
C
[see Fig. 5 . 1]
From the Hamiltonian function
H = [1r (K) - C(I)]e-pt + AI
'
'
there emerge the maximum-principle conditions (not including the transversality condition)
·
::o···
aH
aI = -C'(I)e-p1 + A = 0
aH = I
K. = aA
aH = -1r'(K)e-pt
.A = - aK
If we decide to work with the current-value Hamiltonian, however, we have
( 8 . 19)
He = 1r(K) - C(I) + mi
[ by ( 8 . 1 0) ]
with equivalent maximum-principle conditions
( 8 .20)
( 8.21)
( 8 .22 )
aHe = - C'(I) + m = 0
BI
[by ( 8 . 1 1 ) ]
K. = aHc
am = I
aHe + pm = -1r'(K) + pm
m=
aK
- -
[by ( 8 . 1 2) ]
[by ( 8. 13 ) ]
The latter version is simpler because it is free of the discount factor e -�'1•
From (8.20), we see that
m = C'(I) > O
dm = C"(l) > 0
di
214
PART 3 , OPTIMAL CONTROL THEORY
that is, m is a monotonically increasing function of I. Accordingly, we
should be able to write an inverse function 1/1 =
C'-1:
I = 1/J ( m )
(8.23}
( 1/1' > 0)
Substituting (8.23) into (8.21), we can express K as
K
(8.24}
=
1/!( m )
Paired together, (8.24) and (8.22) constitute a system of two simultaneous
equations in the variables K and m. After solving these for the K*(t) and
m*(t) paths, and definitizing the arbitrary constants by the boundary and
transversality conditions, we can find the optimal control path I*(t) via
(8.23).
EXERCISE 8.2
Find the revised transversality conditions stated in terms
Hamiltonian for the following:
1 A problem with terminal curve Yr = tf>(T).
or• ��
·
i
'
·
··, ·
·
'··
2 A problem with a truncated vertical terminal line.
S A problem with a truncated horizontal terminal line.
8.3
SUFFICIENT CONDITIONS
The maximum principle furnishes a set of necessary conditions for optimal
control. In general, these conditions are not sufficient. However, when
certain concavity conditions are satisfied, then the conditions stipulated by
the maximum principle are sufficient for maximization. We shall present
here only two such sufficiency theorems, those of Mangasarian and Arrow.
The Mangasarian Sufficiency Theorem
A basic sufficiency theorem due to 0. L. Mangasarian3 �
optimal control problem
Maximize
(8.25)
subject to
and
uud for the
u) dt
V = fTF(t,y,
0
y = f(t,y , u )
y( O) = Yo ( y0 , T given)
30. L. Mangasarian, "Sufficient Conditions for the Optimal Control oC NonliDear �."
SIAM JtJUrnal on Control, Vol. 4, February 1966, pp. 139-152.
\
2 15
CHAPTER 8: MORE ON OPTIMAL CONTROL
the necessary conditions of the maximum principle are also sufficient for the
global maximization of V, if (1) both the F and f functions are differen­
tiable and concave in the variables (y, u) jointly, and (2) in the optimal
solution it is true that
(8.26)
for all
A(t) ;::: 0
t
E
[0, T ]
if f is nonlinear in y or in
u
[If f is linear in y and in u, then A(t) needs no sign restriction.]
As a preliminary to demonstrating the validity of this theorem, let us
first remind ourselves that with the Hamiltonian
H = F( t,y, u ) + A f( t , y, u )
the optimal control path u* (t)-along with the associated
paths-must satisfy the maximum principle, so that
aH
0U
-
This implies that
Iu* = Fu ( t , y* , u* ) + A*fu ( t , y* , u* )
=
y*(t) and A*(t)
0
Fu ( t, y* , u* ) = - A*fu(t,y*, u* )
(8.27)
Moreover, from the costate equation of motion,
have
A
=
-iJHjily,
we should
A* = - Fy( t, y* , u* ) - A*fy( t , y*, u* )
which implies that
(8.28)
Fy( t, y* , u* )
=
-A* - A*fy( t, y*, u* )
Finally, assuming for the time being that the problem has a vertical
terminal line, the initial condition and the transversality condition should
give us
(8.29)
Yo* = Yo (given )
and
A*( T ) = 0
These relations will prove instrumental in the following development.
Now let both the F and f functions be concave in (y, u ). Then, for two
distinct points (t, y*, u*) and (t, y, u ) in the domain, we have
(8.30)
(8.30' )
f( t , y, u )
-
:$;
Fy(t, y* , u* ) ( Y - y* )
+ Fu ( t , y* , u*) ( u - u* )
f( t , y * , u* ) 5, fy(t,y*, u*)(y - y * )
+ fu ( t , y * , u* ) ( u - u* )
F(t,y, u ) - F(t , y* , u* )
[ cf. (4.4)]
PART 3 : OPTIMAL CONTROL THEORY
216
Upon integrating both sides of (8.30) over
[0, T ], that inequality becomes
(8.3 1)
V - V* � JT [ Fy (t,y* , u* ) (y - y* )
0
=
+
Fu(t, y*, u* ) ( u - u* )] dt
fT[ -A*(y - y*) - A.*{y(t, y* , u*)(y - y* )
0
(by (8.28) and ( 8.27)]
-A.*fu(t, y* , u* ) ( u - u*)j dt
The first component of the last integral, relating to the expression
y*), can be integrated by parts to yield4
JT0 - A*(y - y*) dt = {0 A.* [ f( t, y, u) - f( t,y* , u* ) ] dt
This result enables us, upon substitution into
(8.3f )
(8.31),
- A* (y . I�
.
to write
V - V* � {A.* [ f(t, y , u) - f(t,y*, u*) - {y (t,y* , u*)(y - y* )
0
-fu(t, y*, u* ) ( u - u*)] dt
�0
The last inequality follows from the assumption of
;:::.: 0 in
fact that the bracketed expression in the integrand is
Consequently, the final result is
)..*
(8.26), and the
by (8.30').
�0
v � V*
(8.3f' )
V*
which establishes
to be a (global) maximum, as claimed in the theorem.
Note that if the
function is linear in (y,
then (8.30') becomes a
strict equality. In that case, the bracketed expression in the integrand in
f
u),
4Let u = - A* and v = y - y*. Then du = -A* dt and dv = (y - y * ) dt. So,
f - A* (y - y* ) dt ( = j0Tvdu )
= ( -A*(y - y* )]� - r - A* (y - y*)
.. :·
dt
= {"-* (:Y - :Y* ) dt
[by (8.29)]
- {A* [ f( t , y , u ) - f(t , y* , u* )] dt
0
[by (8.25)]
0
CHAPTER 8: MORE ON OPTIMAL CONTROL
217
(8.3Y) is zero, and we can have the desired result V - V * � 0 regardless of
the sign of A*, so the restriction in (8.26) can be omitted.
The above theorem is based on the F and f functions being concave.
If those functions are strictly concave, the weak inequalities in (8.30) and
(8.30' ) will become strict inequalities, as will the inequalities in (8.31),
(8.3Y), and (8.3f' ). The maximum principle will then be sufficient for a
unique global maximum of V.
Although the proof of the theorem has proceeded on the assumption of
a vertical terminal line, the theorem is also valid for other problems with a
fixed T (fixed terminal point or truncated vertical terminal line). To see
this, recall that in the proof of this theorem the transversality condition
A*(T) 0 in (8.29) is applied in the integration-by-parts process (see the
associated footnote) to make the expression - A*(T)(yr - Yr* ) vanish. But
the latter expression will also vanish if the problem has a fixed terminal
state Yro• for then the said expression will become - A*(T)(Yro - Yr* ), and
it must be zero because Yr* has to be equal to Yr o· Moreover, if the problem
has a truncated vertical terminal line, then either the transversality condi­
tion A* ( T ) 0 is satisfied (if the truncation point is nonbinding) or we must
treat the problem as one with a fixed terminal state at the truncation point.
In either event, the said expression will vanish. Thus the Mangasarian
theorem is applicable as long as T is fixed.
In the application of this theorem, it is possible to combine Mangasar­
ian's conditions (1) and (2) into a single concavity condition on the Hamilto­
nian. If the F and f functions are both concave in (y, u ), and if A is
nonnegative, then the Hamiltonian, H = F + A {, being the sum of two
concave functions, must be concave in (y, u ), too. Hence, the theorem can be
restated in terms of the concavity of H.
=
=
The Arrow Sufficiency Theorem
Another sufficiency theorem, due to Kenneth J. Arrow,5 uses a weaker
condition than Mangasarian's theorem, and can be considered as a general­
ization of the latter. Here, we shall describe its essence without reproducing
the proof.
At any point of time, given the values of the state and costate variables
y and A, the Hamiltonian function is maximized by a particular u , u*,
6
This theorem appears without proof as Proposition 5 in Kenneth J . Arrow, " Applications of
Control Theory to Economic Growth, " in George B. Dantzig and Arthur F. Veinott, Jr., eds.,
Mathematics of the Decision Sciences, Part 2, American Mathematical Society, Providence, RI,
1968, p. 92. It also appears with proof as Proposition 6 in Kenneth J. Arrow and Mordecai
Kurz, Public Inue�tment, The Rate of Return, and Optimal Fiscal Policy, published for
Resources for the Future, Inc., by the Johns Hopkins Press, Baltimore, MD, 1970, p. 45. A
proof of the theorem is also provided in Morton I. Kamien and Nancy L. Schwartz, "Sufficient
Conditions in Optimal Control Theory," Journal of Economic Theory, Vol. 3, 1971, pp.
207-�14.
PAltT S: OPTIMAL CONTROL THEORY
218
which depends on
(8.32)
t, y, and A:
u* = u*(t,y, A)
When (8.32) is substituted into the Hamiltonian, we obtain what is referred
to as the maximized Hamiltonian function
H0(t, y, A) = F( t , y , u* ) + A f(t,y , u*)
Note that the concept of H0 is different from that of the optimal Hamilto­
nian H* encountered in (7.53) and (7.54). Since H* denotes the Hamilto­
nian evaluated along all the optimal paths, that is, evaluated at y*(t), u*(t),
and A*(t) for every point of time, the y, u, and A arguments can all be
substituted out, leaving H* as a function of t alone: H* H*(t). In
contrast, H0 is evaluated along u*(t) only; thus, while the u argument is
substituted out, the other arguments remain, so that H0 = H0(t, y, A) is
( 8.33)
=
still a function with three arguments.
The Arrow theorem states that, in the optimal control problem (8.25),
the conditions of the maximum principle are sufficient for the global maxi­
mization of V, if the maximized Hamiltonian function H0 defined in (8.33)
is concave in the variable y for all t in the time interval [0, T], for given A .
The reason why the Arrow theorem can b e considered as a generaliza­
tion of the Mangasarian theorem-or the latter as a special case of the
former-is as follows: If both the F and f functions are concave in (y, u)
and A � 0, as stipulated by Mangasarian, then H = F + A f i s also concave
in (y, u), and from this it follows that H0 is concave in y, as stipulated by
Arrow. But H0 can be concave in y even if F and f are not concave in
(y, u), which makes the Arrow condition a weaker requirement.
Like the Mangasarian theorem, the validity of the Arrow theorem
actually extends to problems with other types of terminal conditions as long
as T is fixed. Also, although the theorem has been couched in terms of the
regular Hamiltonian H and its " maximized" version H0, it can also be
rephrased using the current-value Hamiltonian He and its " maximized"
version Hc0, which differ from H and H0, respectively, only by a discount
factor e-pt.
EXAMPLE 1
problem
Maximize
subject to
and
In Sec. 7 .2, Example 1, we discussed the Bh�ce
[T
2V = ), - ( 1 +
0
j=u
y(O) = A
u
2
)
1/2
y(T ) free
Let us apply both sufficiency theorems.
dt
( A, T given)
[ from ( 7.7)]
219
CHAPTER 8: MORE ON OPTIMAL CONTROL
For the Mangasarian theorem, we note that neither the F function
nor the f function depends on y , so the concavity condition relates to u
alone. From the F function, we obtain
-1
and
Fu = u ( 1 + U 2 ) /2
Fuu = - ( l + u 2 ) -3/2 < 0
-
[by the product rule]
Thus F is concave in u . As to the f function, f = u , since it is linear in u ,
it is automatically concave in u . Besides, the fact that f is linear makes
condition (8.26) irrelevant. Consequently, the conditions of Mangasarian are
satisfied, and the optimal solution found earlier does maximize V (and
minimize the distance) globally.
Once the Mangasarian conditions are satisfied, it is no longer neces­
sary to check the Arrow condition. But if we do wish to apply the Arrow
theorem, we can proceed to check whether the maximized Hamiltonian H 0
is concave in y. In the present example, the Hamiltonian is
H
=
-(1
+
1
u 2 ) 12 + A u
When the optimal control
u (t)
=
1
A ( 1 - A2 ) - 12
[from (7.9)]
is substituted into H to eliminate u , the resulting H 0 expression contains
A alone, with no y. Thus H 0 is linear and hence concave in y fpr given A,
and it satisfies the Arrow sufficient condition.
·
EXAMPLE 2
In the problem of Example 2 in Sec.
. ··· ··;
Maximize
.·.·
,,.> ·
subject to
and
7.2:
V = J \2y - 3 u ) dt
0
y =y + u
y(O) = 4 y(2) free
[from (7.13)]
u(t) E [0, 2]
both the F and f functions are linear in (y, u ). As a result, all the
second-order partial derivatives of F and f are zero. With reference to the
test for sign semidefiniteness in (4. 12), we have here ID1 1 ID2 1 0,
which establishes the negative semidefiniteness of the quadratic form in­
volved. Hence, both F and f are concave in (y, u ). Moreover, as in Example
1, the stipulation in (8.26) is irrelevant. The conditions of the Mangasarian
theorem are therefore satisfied.
As to the Arrow condition, we recall from (7.14) that the optimal
control depends on A, and that u * takes the boundary values 2 and 0 in the
=
=
220
PART S: OPT04AL CONTROL THEORY
two phases of the solution as follows:
u*11
and
=0
[ from (7 . 17)]
Therefore, we should have two maximized Hamiltonian functions, one for
each phase. From the Hamiltonian
H = 2y - 3u + A(y + u )
we obtain, upon eliminating u ,
H 01 = 2y - 6 + A ( y + 2 ) = (2 + A )y - 6 + 2A
H 0 11 = 2y + Ay = (2 + A )y
In either phase, H 0 is linear in y for given A, so the Arrow condition
satisfied.
iiJ
Let us now check whether the maximum principle is suffi­
cient for the control problem adapted from the Eisner-Strotz model (Sec.
8.2, Example 1):
EXAMPLE 3
Maximize
{[77'(K) - C ( I ) ] e-P1 dt
0
subject to
and
boundary conditions
[ from (8. 18)]
7T"(K)
where
< 0, C '(I) > 0, and C"(l) > 0.
For the Mangasarian conditions, we can immediately see that the f
function, f = I, is linear and concave in I. To check the concavity of the F
function in the variables
1 ), we need the second-order partial derivatives
of F =
C(l)}e -P1• These are found to be
[7T(K) -
(K,
Fxx = 1r" ( K ) e-pt < 0
Fxr = FIK = 0
Fu = - C"( I )e-P1 < 0
Thus, with reference to the test for sign definiteness in (4.8) and (4.9), we
find here ID11 < 0 and ID2 1 > 0, indicating that F is strictly concave in
1). Since condition (8.26) is again irrelevant, the Mangasarian condi­
tions are fully met.
To check the Arrow sufficient condition, we recall from (8.20) and
(8.23) that, with the current-value Hamiltonian
(K,
He = 77'(K)
- C( I) + ml
[ from (8.19)]
m in terms of I,
we can, by setting aHjai = 0, solving for
and then
writing I as an inverse function of m, express the optimal control
the
in
,. .,,,,;;<·'··;
221
CHAPTER 8: MORE ON OPTIMAL CONTROL
form
I = 1/J ( m )
[ from ( 8 .23)]
Substitution o f this optimal control into He yields the
value Hamiltonian
maxithize(feurrent- ''
Hc0 = rr ( K) - C (l/f( m )] + m l/f( m )
Since aHc0faK = rr'(K) and a2Hc0faK 2 = rr"(K) < 0, the maximized cur­
rent-value Hamiltonian is strictly concave in the state variable K. The
Arrow sufficient condition is thus satisfied.
The checking of concavity in the present example is slightly more
laborious than in the preceding two examples, because even though the
model is simple, it involves general functions. For more elaborate
general-function models, checking concavity-especially for the maximized
Hamiltonian-can be a tedious process.6 Frequently, therefore, models are
constructed with the appropriate concavity;convexity properties incorpo­
rated or assumed at the outset, thereby obviating the need to check suffi­
cient conditions.
EXERCISE 8.3
Check the Mangasarian and Arrow sufficient conditions for the following:
1 Example 1 in Sec. 7.4.
2 Example 3 in Sec. 7.4.
� ' .' ' .
3 Problem 1 in Exercise 7 .4.
4 Problem 3 in Exercise 7.4.
5 The Nordhaus model in Sec. 7.6.
6 The Forster model in Sec. 7.7.
8.4 PROBLEMS WITH SEVERAL STATE
AND CONTROL VARIABLES
For simplicity, we have so far confined our attention to problems with one
state variable and one control variable. The generalization to problems with
multiple state variables and control variables is, in principle, very straight­
forward. But the solution procedure naturally becomes more complicated.
6
For more examples of checking the Arrow sufficient condition, see Morton I. Kamien and
Nancy L. Schwartz,
Dynamic Optimization: The Calculus of Variations and Optimal Control
in Economics and Management, 2d ed., Elsevier, New York, 1991, pp. 222-225; Atle Seierstad
and Knut Sydsreter, Optimal Control Theory with Economic Applications, Elsevier, New York,
1987, pp. 112-129.
222
PART 3: OPTIMAL CONTROL THEORY
The Problem
be n state variables, y1 , . . . , y,. , and
control variables,
u 1, . . . , u m. The two numbers n and are not restricted in relative
magnitudes; we can have either n < m , n = m, or n > m. Each state
variable yi must come witp an equation of motion, describing how yi
specifically depends on t, on the values of Yi and the other state variables at
time t, as well on the values of each control variable u chosen at time t.
Let there
·
·
m
m
as
Thus, there should be in the problem
n
i
differential equations in the form
y
There should be n initial conditions on the
variables. Assuming the
problem to be one with fixed terminal points, we should also have n given
terminal conditions on the state variables. In contrast, the control variables
are not accompanied by equations of motion, but each control variable
may be subject to the restriction of a control region
Assuming fixed
initial and terminal points for the time being, the optimal control problem
can be expressed as
ui
�i .
Maximize
subject to
(8.34)
Y 1( 0) = Yw. , y,. (O) = Y,.o
y,(T) = y1T , . . . , y,. (T) = y,. T
u 1(t} �1, . . . , um(t) �m
· · ·
and
E
E
This rather lengthy statement of the problem can be shortened some­
what by the use of index subscripts. Using the index j for the state
variables and the index i for the control variables, we may restate the
problem as
Maximize
( 8.34')
subject to
and
V = /,T0 F(t, y1, . . . , y ,., u 1, . . . , u m ) dt
i
um )
Yi = f ( t, yl , . . . , y,., ul,
yj{ O) = Yio yj(T ) = YiT
ui( t) E �i ( i = l, . . . , m , j = l, . . , n )
. . . •
.
223
CHAPTER 8: MORE ON OPTIMAL CONTROL
To have an even more compact statement of the problem, we may
employ vector notation. Define a y vector and a u vector
Then the integrand function F in (8.34') can be abbreviated to F(t, y, u ) By
the same token, the arguments in the fi functions can be simplified to
(t, y, u ). If we define two more vectors
.
then the equations of motion can be collectively written in a single vector
equation: :Y = f(t, y, u ). Obvious extensions of the idea to the boundary
conditions would enable us to write the vector equations y(O) = y 0 and
y(T ) = Y T• where y(O), y0, y(T ), and Y r are all n X 1 vectors. And, finally,
we can express the control-region restrictions by the vector statement
u(t) E �. where u(t) and � are both m X 1 vectors.
In vector notation, therefore, the statement of the optimal control
problem is simply
Maximize
(S.M"}
subject to
and
v=
fTF( t, y' u ) dt
0
:Y = f( t, y , u )
y(O) = Y o y(T ) = Yr
u(t) E �
which, in appearance, is not at all different from the problem with one state
variable and one control variable. The only difference is that, in (8.34"),
several symbols represent vectors, and therefore involve much more than
what is visible on the surface. Note, however, that even in the vector
statement of the problem, not all the symbols represent vectors. The symbol
V is clearly a scalar, and so are the symbols t and T.
The Maximum Principle
The extension of the maximum principle to the multivariable case is compa­
rably straightforward. First of all, to form the Hamiltonian, we introduce a
224
PART 3, OPTIMAL CONTROL THEORY
Lagrange multiplier for every
( 8.35)
{J
function in the problem. Thus, we have
H = F( t' y' u )
+
n
L Aj fj ( t' y ' u )
j- 1
where y and u are the state- and costate vectors defined earlier. The
summation term on the right can, if desired, also be written in vector
notation. Once we define another n X 1 vector
with transpose
X =
[ A1( t)
the Hamiltonian can be rewritten as
( 8 .35')
H( t , y , u , A) = F( t , y , u ) + X f( t , y , u )
which looks strikingly similar to the Hamiltonian i n a one-variable case,
except that, here, the last term is a scalar product-the product of the row
vector X and the column vector f(t, y, u ). Note that the Hamiltonian itself
is also a scalar.
The requirement that the Hamiltonian be maximized at every point of
time with respect to the control variables stands as before. We can thus still
write the condition
Max H
(8 .36)
u
But u is now an m-vector, so we must at each point of time make a choice
of m control values, u t, . . . , u m* ·
As to the equations of motion, those for the state variables always
come with the problem; they are just the n differential equations appearing
in (8.34). Moreover, these n equations are still expressible in terms of the
derivatives of the Hamiltonian:
(8 .37)
ilH
yj = �J
(j = l , . . . , n )
Similarly, the equations of motion for the costate variables are simply
( 8.38)
iJH
.
AJ· = - iJy
j
(j = l, . . . , n )
Together, (8.37) and (8.38)-representing a total of 2n differential equa­
tions-constitute the canonical system of the present problem. The 2n
arbitrary constants that are expected to arise from these differential equa­
tions can be definitized by using the 2n boundary conditions.
225
CHAPTER 8: MORE ON OPTIMAL CONTROL
If desired, the canonical system can also be translated into vector
notation. The equivalent statement for (8.37) is
(8.37' )
Y=
aH
ax
[X
=
transpose of A ]
The noteworthy thing about this is that H is differentiated with respect to
(a row vector), not A (a column vector). A look at (8.35') would make it
clear why this should be the case. More to the point, however, is the fact
that (8.37') is in line with the mathematical rule that the derivative of a
scalar with respect to a row (column ) vector is a column (row) vector.
Accordingly, the vector-notation translation of (8.38) should be
X
iJH
iJy
(8 .38')
[ X : transpose of A]
Transversality Conditions
Problem (8.34) assumes fixed endpoints. I f the terminal point i s variable, we
again need appropriate transversality conditions.
Referring back to (7.30), we see that, for the one-state-variable prob­
lem, the transversality conditions are derived from the last two terms of the
dVjd E expression set equal to zero: [H ]1_ r tlT - A(T) tlyr 0. When
there are n state variables in the problem, those two terms will expand to
an expression with (n + 1) terms:
=
From this, the following two basic transversality conditions arise:
( 8.40)
[ H Jt- T
( 8.41)
A1( T )
=0
=
[ if T is free ]
[ if YJ T is free)
0
Clearly, these conditions are essentially no different from the transver­
sality conditions for the one-state-variable case. The only differences are
that (1) the Hamiltonian H in (8.40) for the present n-variable problem
contains more terms than its one-variable counterpart, and (2) there will be
as many conditions of the A/T ) 0 type in (8.41) as there are state
variables with free terminal states.
The transversality conditions for the terminalccurve and truncated­
terminal-line problems are also essentially similar to those for the one­
state-variable case. Suppose that the terminal time is free, and two of the
state variables, y1 and y2, are required to have their terminal values tied to
the terminal time by the relations
=
( 8.42 )
Y1r
=
f/>1( T )
and
226
PART 3: OPTIMAL
CONTIIOL 1'HB()BY
AT, we may expect the following to hold:
Ay1r = cfJ/ ( T ) AT and
Using these to eliminate Ay 1r and Ay2r in (8.39), we can rewrite the latter
Then, for a small
as
The transversality condition that emerges for this terminal-curve problem is
[ cf. ( 7.32) ]
(8 .44)
Condition (8.44)-which should replace (8.40)-together with the two equa­
tions in (8.42) provide three relations to determine the three unknowns T,
y 1T , and y2r. The other state variables, (y3, . . . , Yn), assumed here to have
free terminal values, are still subject to the type of transversality condition
in (8.41 ).
If the terminal time is fixed, and all the state variables have truncated
vertical terminal lines, then the transversality condition is
(YJT - YJ,min)..\J( T ) = 0,
(8.45)
(j = l , . , n )
. .
[ cf. ( 7 .35) ]
This condition serves as a replacement for (8.41).
Finally, for the problem with a maximum permissible terminal time,
Tmax• the transversality condition is
(8.46)
This condition is another type of replacement for (8.40).
An Example from Pontryagin
While the conceptual extension of the maximum principle to the multivari­
able problem is straightforward, the actual solution process can become
much more involved. For this reason, economic applications with more than
one state variable are not nearly as common as those with one state
variable. A taste of the increasing complexity can be had from a purely
mathematical example adapted from the work of Pontryagin et al.,? with
two state variables and one control variable.
This example actually starts out with a single state variable y, which
we wish to move from an initial state y(O) = Yo to the terminal state
7L. S. Pontryagin et a!.,
York, 1962, pp. 23-27.
·TAe MatherraaliCGl T"-Y of Optimal �. In�
N­
227
CHAPTER 8: MORE ON OPTIMAL CONTROL
y(T) = 0 in the shortest possible time. In other words, we have a time-opti­
mal problem. Instead of the usual equation of motion in the form of
y = f(t,y, u), however, we have this time a second-order differential equa­
tion, ji = u , where ji = d 2yjdt 2, and where u is assumed to be constrained
to the control region [ - 1, 1]. Since the second-order derivative ji is not
admissible in the standard format of the optimal control problem, we have
to translate this second-order differential equation into an equivalent pair of
first-order differential equations. It is this translation process that causes
the problem to become one with two state variables. Thus, this example also
serves to illustrate how to handle a problem that contains a higher-order
differential equation in the state equation of motion.
To effect the translation, we introduce a second state variable z:
z =j
i = ji
implying
Then the equation ji = u can be rewritten as
first-order equations of motion
y=z
�ne
i=u
for each state variable. The problem is then to
Maximize
'
and
i = u . And we now have two
(
.
subject to
(8.47)
and
.'·· -..
lT0 - 1 dt
y =z
i=u
y(O) = Yo (given)
z(O) = z0
u(t) E [ - 1, 1]
y( T)
-=
0
T free
The boundary condition on the new variable z is equivalent to the specifi­
cation of an initial value for j. The specification of both y(O) and j(O) is a
normal ingredient of a problem involving a second-order differential equa­
tion in y.
Step
i
The Hamiltonian function is
. ( 8.48)
Being linear in u, this function leads to corner solutions for
the given control region, the optimal control should be
(8.49)
u* = sgn A2
[ cf. ( 7 .48)]
u.
Jn view of
228
PART a, OPTIMAL CONTROL THEORY
Step ii Since
u
*
depends on A2, we must next turn to the equations of
motion for the costate variables:
aH
A1 = - ay
=
o
implying A1( t) = c1 (constant)
aH
.
A2 = - - = - A l
ilz
=
Thus, the time path for A2 is linear:
( 8 . 50 )
( c 1 , c2 arbitrary)
Excepting the special case of c 1 = 0, this path plots as a straight line
that intersects the t axis. At that intersection, A2 switches its algebraic sign
and, according to (8.49), the optimal control u* must make a concomitant
switch from 1 to - 1 or the other way around, thereby yielding a bang-bang
solution. However, since the linear function A it) can cross the t axis at
most once, there cannot be more than one switch in the value of u* .
,
· Step iii Next, we examine the state variables. The variable z hae the
equation of motion i = u. Inasmuch as u can optimally take only two
possible values, u * = ± 1, there are only two possible forms that equation
can take: i = 1 and i = - 1.
Possibility 1
(8 .51)
u
*
= 1 and
i
= 1. In this case, the time path for z is
z(t) = t + c3
Then, since y = z = t + c3, straight integration yields the following
quadratic time path for y:
(8 .52)
The initial condition y(O) = y0 tells us that c4 = y0• But c3 is not as
easy to definitize from the terminal condition y( T ) = 0, because T is not
yet known. Glancing at (8.51), however, we note that c3 represents the
value of z at t = 0. If we denote z(O) by z0, then c3 = z0• Thus, (8.51) and
(8.52) may be definitized to
(8.53)
(8 .54)
z(t) = t + z0
2
y ( t) = �t + z0t + Yo
The way these last two equations are written, y and z appear as two
separate functions of time. But it is possible to condense them into . one
single equation by eliminating t and expressing y in terms of z. This will
enable us to depict the movement of y and z in a phase diagram in the yz
229
CHAPTER 8: MORE ON OPTIMAL CONTROL
Possibility 1
u* = 1
FIGURE 8.1 .
plane.
Since
2
t 2 + 2z0 t
z 2 ( t + z0)
2
we can solve this for t and rewrite (8.54) as
=
=
+
z02
·,
( 8 .54' )
In the yz plane, this equation plots as a family of parabolas as illustrated in
Fig. 8.1. Each parabola is associated with a specific value of k which is
based on the values of Yo and z0• In particular, when k 0, we get the
parabola that passes through the point of origin. Of course, once y0 and z0
are numerically specified, we can pinpoint not only the relevant parabola,
but also the exact starting point on that parabola.
Figure 8.1 is a one-variable phase diagram, with z = dy jdt on the
vertical axis and y on the horizontal axis. Above the horizontal axis, with
y > 0, the directional arrowheads attached to the curves should uniformly
point eastward. The opposite is true below the horizontal axis. Note that,
according to these arrowheads, the designated terminal state y(T ) = 0 is
attainable only if we start at some point on the heavy arc of the parabola
that passes through the point of origin. Note also that the terminal value of
z is z(T) 0.
=
=
Possibility 2 u* = - 1 and i - 1. Under the second possibility, the
equation of motion for z integrates to
=
( 8.55)
z(t)
=
-t
+
c5
=
-t
+
z0
[ cf. ( 8.53)]
Since this result also represents the j path, further integration and defini-
PAR',I' 8: OPTlMAL CONTROL THEORY
230
y
= -
1
- z'
Possibility
u' = -1
2
2
y
.� •. '
..
FIGURE 8.2
tization of the constant gives us
(8 .56)
y(t)
=
- it 2 + Zot + Yo
[ cf. ( 8 .54))
Combining the last two equations, we obtain the single result
which is comparable to (8.54') in nature. Note that the two shorthand
symbols k and h are not the same. Like (8.54'), this new condensed
equation plots as a family of parabolas in the yz plane. But because of the
negative coefficient of the z2 term, the curves should all bend the other way,
as shown in Fig. 8.2. Each parabola is uniquely associated with a value of h;
for example, the one passing through the origin corresponds to h = 0. As
before, the directional arrowheads uniformly point eastward above the
horizontal axis, but westward below. Here, again, only the points located on
the heavy arc can lead to the designated terminal state y(T ) = 0. The
terminal value of z is, as may be expected, still z(T ) = 0.
Although we have analyzed the two cases of u * = 1 and u * = - 1 as
separate possibilities, both possibilities will, in a bang-bang solution, become
realities, one sequentially after the other. It remains to establish the link
between the two and to find out how and when the switch in the optimal
control is carried out, if needed.
Step iv This link can best be developed diagrammatically. Since the heavy
arcs in Figs. 8.1 and 8.2 are the only routes that can lead ultimately to the
desired terminal state, let us place them together in Fig. 8.3. Even though
the two arcs appear to have been merged into a single curve, they should be
CHAPTER 8: MORE ON OPTIMAL CONTROL
231
z (ey)
I
(u•
=
+ 1)
FIGURE 8.3
kept distinct in our thinking; the arrowheads go in opposite directions on
the two arcs, and they call for different optimal controls-u* = + 1 for arc
AO and u * = - 1 for arc B O.
Should the actual initial point be located anywhere on arc AO or arc
B 0, the selection of the proper control would start us off on a direct journey
toward the origin. No switch in the control is needed. For an initial point
located off the two heavy arcs, however, the journey to the origin would
consist of two legs. The first leg is to take us from the initial point to arc AO
or B 0, as the case may be, and the second leg is to take us to the origin
along the latter arc. For this reason, there is no need to reproduce in Fig.
8.3 every part of every parabola in Figs. 8.1 and 8.2. All we need are the
portions of the parabolas flowing toward and culminating in arc AO and arc
B O, respectively. If, for instance, point C is the initial point, then the first
leg is to follow a parabolic trajectory to point D. Since the parabola involved
is taken from Fig. 8.2 under possibility 2, the control appropriate to the
circumstance is u *
- 1. Once point D is reached, however, we should
detour onto arc AO. Since the latter comes from Fig. 8.1 under possibility 1,
the control called for is u* = 1 . By observing these u* values, and only by
so doing, we are able to proceed on course along the optimal path CDO.
Other initial points off the heavy arcs (such as point E) can be
similarly analyzed. For each such initial point, there is a unique optimal
path leading to the designated terminal state, and that path must involve
one (and only one) switch in the optimal control. Thus, in such cases, the
optimal movement of y over time is never monotonic; an initial rise must be
followed by a subsequent fall, and vice versa. The optimal movement of the
control variable over time, on the other hand, traces out a step function
composed of two linear segments.
=
�'.· .
232
PART 3: OPTIMAL CONTROL THEORY
More about the Switching
Having demonstrated the general pattern of the bang-bang solution, Pon­
tryagin et al. dropped the example at this point. However, we can proceed a
step further with the discussion of the CD O path, to find the coordinates of
the switching point D and..also to determine the exact time,
r,
at which the
switch in the control should be made.
Point D is the intersection of two curves. One of these, arc AO, being
part of a parabola taken from Fig. 8.1, falls under possibility
described by (8.54') with k = 0. Specifically, we have
1,
and is
2,
and is
( z < 0)
( 8 .57)
8.2
The other is a parabola taken from Fig.
described instead by
(8.56').
is positive. Hence, we have
under possibility
Since the intercept on the y axis is positive,
·
(h >
( 8 .58)
At point D, both
(8.57) and (8.58) are satisfied.
simultaneously yields
ble). Thus,
y=
4h
and
Point
(8 .59)
And the switch in
- /h .
u*
z=
D =
-
0)
Solving these two equations
fh (the positive root is inadmissi­
(�h , - /h )
should be made precisely when
The calculation of the switching time
r
From the initial point C, the movement of
=
- t + z0.
At time
r,
z
y=
th
and
z( = y) =
can be undertaken from the
knowledge that, at point D, and hence at time r,
(8.55): z(t)
h
z
attains the value
- /h.
follows the pattern given in
its value should be
z( r ) = - r + z0
Setting
z( T) equal to
-
lh ,
(8 .60)
we then find that
T = z0
+
/Ji
This result shows that the larger the values of
z0
and
h, the longer it would
take to reach switching time. A look at Fig. 8.3 will confirm the reasonable­
ness of this conclusion: A larger z0 and/or a larger
would mean an initial
point farther away from arc AO, and hence a longer journey to reach arc
h
AO, where the switching point is to be found. It should be realized, of
course, that
(8.60)
is appropriate only for switching points located on arc
AO, because the assumptions of a negative
z
and a positive
h
used in its
derivation are applicable only to arc AO. For switching points located on arc
B O, the expression for
the same.
T
is different, although the procedure for finding it is
233
CHAPTER 8: MORE ON OPTIMAL CONTROL
It is also not difficult to calculate the amount of time required to travel
from point D to the final destination at the origin. On that second leg of
the journey, the value of z goes from z( T) = - /h to zero. The relevant
equation for that movement is (8.53), where, for the present purpose, we
should consider T as the initial time and z( -r) = - lh as the initial value
z0• In other words, the equation should be modified to� z(t) = t - /h. At
the terminal time-which we shall denote for the time being as T'-the
left-hand-side e_!Pression becomes z(T') 0, and the right-hand-side one
becomes T' - -/h . Thus, we obtain
=
T' = {h
( 8 .61)
The reason we use the symbol T ' instead of T is that the symbol T is
supposed to represent the total time of travel from the initial point C. to the
origin, whereas the quantity shown in (8.61) tells only the travel time from
point D to the origin. To get the optimal value of T, (8.60) and (8.61)
should be added together. That is,
T*
( 8 .62)
= -r
+ T' = z0 +
2/h
The preceding discussion covers most aspects of the time-optimal
problem under consideration. About the only thing not yet resolved is the
matter of the arbitrary constants c1 and c2 in (8.50). The purpose of finding
the values of c1 and c2 is only to enable us to definitize the A 2( t ) path, so as
to know when A 2 will change sign and thereby trigger the switch in the
optimal control. Since we have already found -r in (8.60) via another route,
there is little point in worrying about c 1 and c2• For the interested reader,
however, we mention here that c1 and c2 can be definitized by means of the
following two relations: First, when t = T, A2 must be zero (at the juncture
of a sign change); hence, from (8.50), we have - c1T + c2 0. Second, the
transversality condition
=
( H ]t=T
=
[ - 1 + A 1 z + A 2u ]1_T
=
0
reduces to - 1 + A 2( T ) 0, since z(T) 0 and u(T) = 1. Substituting
(8.50) into this simplified transversality condition yields another equation
relating c1 to c2• Together, these two relations enable us to definitize c 1
and c2•
=
=
EXERCISE 8.4
1 On the basis of the optimal path CDO in Fig. 8.3, plot the time path u*(t ).
2 On the basis of the optimal path CDO in Fig. 8.3, describe the way y*
changes its values over time, bearing in mind that the value of z represents
the rate of change of y. From your description, plot the time path y*(t).
PART s, OPTIMAL CONTROL THEORY
234
3 For each of the following pairs of Yo and z0 values, indicate whether the
resulting optimal path in Fig. 8.3 would contain a switch in the control,
and explain why:
(a) (y0 , z0) = (2, 1)
(b) (y0, z0) = (2, 2)
(c) (y0, z0) = ( - i , l)
4 Let point E in Fig. 8.3 be the initial point.
(a) What is the optimal path in the figure?
(b) Describe the time path u * (t ).
(c) Describe the time path y*(t), taking into account the fact that z "' j.
Plot the y*(t ) path.
5 Let point E in Fig. 8.3 be the initial point.
(a) Find the coordinates of the switching point on arc B O.
(b) Find the switching time T.
(c) Find the optimal total time for traveling from point E to the destination
at the origin.
-
_
8.5
ANTIPOLLUTION POLICY
In the Forster model on energy use and environmental quality in Sec. ·7 .7,
pollution is taken to be a flow variable.8 This is exemplified by auto
emission, which, while currently detrimental to the environment, dissipates
quickly and does not accumulate into a long-lasting stock. But in other
types of pollution, such as radioactive waste and oil spills, the pollutants do
emerge as a stock and produce lasting effects. A formulation with pollution
as a stock variable is considered by Forster in the same paper cited
previously. This formulation contains two state variables and two control
variables.
The Pollution Stock
As before, we use the symbol E to represent the extraction of fuel and
energy use. But the symbol P will now denote the stock (rather· than flow)
of pollution, with P as its flow. The use of energy generates a flow of
pollution. If the amount of pollution flow is directly proportional to the
amount of energy used, then we can write P = a E, (a > 0). Let A stand for
the level of antipollution activities, and assume that A can reduce the
pollution stock in a proportionate manner. Then, from this consideration,
we have P = J3A, ((3 > 0). Further, if the pollution stock is subject to
-
8Bruce A. Forster, " Optimal Energy Use in a Polluted ED"rironment," Journal ofEnvii'"01IIIWIItal Economics and Management, 1980, pp. 321-333.
' ,, �� . .-.
·.
CHAPTER s, MORE ON OPTIMAL CONTROL
235
exponential decay at the rate
> 0, then we have P
=
Combining these factors that affect
we can write
P
S
-8P.
(8.63)
P
=
P,
aE - {3A - SP
( a , f3
> 0, 0
jP = -S,
so that
< S < 1)
The Stock of Energy Resource
A in itself requires the use of
A causes a reduction in S, the stock of the energy resource.
Assuming a proportional relationship between A and S, we can, by the
proper choice of unit A, simply write S = -A. But since S is also reduced
by the use of energy in other economic activities, we should also have
S = -E [same as (7.71)]. Combining these considerations, we have
The implementation of antipollution activities
energy. That is,
S
( 8 .64)
= -A -
E
The Control Problem
(8.63) and (8.64) delineate the dynamic changes of P
S, respectively, these equations can evidently serve the equations of
motion in this model. This then points to P (pollution stock) and S (fuel
stock) as the state variables. A further look at (8.63) and (8.64) also reveals
that E (energy use) and A (antipollution activities) should play the role of
Since the relations in
as
and
control variables in the present analysis.
If we adhere to the same utility function used before:
U = U[C(E) , P]
(Uc > 0, Up < 0, Ucc < 0 , Upp < 0 , C' > 0 , C" < 0)
[ from ( 7.74) and (7.72)]
then the dynamic optimization problem may be stated as
�·::�
Maximize
subject to
(&.�}_
�- ·
l ·-
and
fTU[C(
E), P) dt
0
P = aE - {3A - 8P
S = -A - E
P(O) = P0 > 0 P(T) � 0 free
S(O) = 80 > 0 S(T) � 0 free
E � O O�A�A
(T given)
Two aspects of this problem are comment·worthy. First, while the
P
T,
initial values of
and S are fixed, as is the terminal time
the terminal
and the energy resource stock S are
values of both the pollution stock
P
236
PART 3: OPTIMAL CONTROL THEORY
left free, subject only to nonnegativity. This means that there is a truncated
vertical terminal line for
and another one for S. Second, both control
variables
and
are confined to their respective control regions. For
the control region is [0, oo). And for A, the control region is [0,
where A
denotes the maximum feasible level of antipollution activities. Since budget
E
A
P,
E,
A],
considerations and other factors are likely to preclude unlimited pursuit of
environmental purification, the assumption of an upper bound
seem unreasonable.
A
does not
Maximizing the Hamiltonian
As usual, we start the solution process by writing the Hamiltonian function:
(8.66)
H
=
U[C(E), P] + Ap(aE - {3 A - oP) - A s( A + E )
A
where the subscript of each costate variable
indicates the state variable
associated with it. To maximize H with respect to the control variable
E,
E � 0, the Kuhn-Tucker condition is aH;aE � 0, with the comple­
mentary-slackness proviso that E(oHjoE) = 0. But inasmuch as we can
rule out the extreme case of E = 0 (which would totally halt consumption
production), we must postulate E > 0. It then follows from complementary
where
slackness that we must satisfy the condition
aH
= Uc C'( E ) + aAp - As = 0
aE
Note that o 2HjoE 2 = UccC' 2 + UcC" < 0; so H indeed is maximized rather
( 8.67 )
than minimized.
In addition, H should be maximized with respect to A . As
shows, H is linear in the variable
with
A,
(8.66)
aH
= -{3Ap - As
oA
A].
�
Besides, E is restricted to the closed control set (0,
Thus to maximize
H, the left-hand-side boundary solution A* = 0 should be chosen if aH;aA
is negative, and the right-hand-side boundary
oHjoA is positive. That is,
-
(8.68)
From
(8.67), however, we
see that
A*
=
A* =
A
should be selected if
{�}
237
CHAPTER 8: MORE ON OPTIMAL CONTROL
Substitution of this into (8.68) results in the alternative condition
A* = { 1}
( 8 .68' )
The optimal choice of A thus depends critically on A P·
The Policy Choices
Categorically, the optimal choice of antipollution activities A is either an
interior solution or a boundary solution. Forster shows that it is not
possible to have an interior solution in the present model.
To see this, consider the equations of motion of the costate variables:
( 8.69)
(8 .70)
If
v+•
aH
.
Ap = = - Up + 8Ap
iJP
As =
aH
-
-
as
=
=
0
As = constant
A* is an interior solution, then
{3Ap + As = 0
[ by ( 8 .68)]
Since As is a constant by (8.70), this last equatio n shows that Ap must also
be a constant, which in turn implies that
[by (8.69)]
(8.71)
But the constancy of A p requires Up to be constant, too. Since U is
monotonic in P, there can only be one value of P that would make Up take
any particular constant value. Thus P, too, must be a constant if
is an
interior solution.
Given the initial pollution stock P0 > 0, to have a constant P is to fix
the terminal pollution stock at P(T) = P0 > 0. For a problem with a
truncated terminal line, the transversality condition includes the stipulation
that
A*
( 8 . 72)
P ( T ) Ap( T ) = 0
With a positive P(T ), it is required that Ap(T) = 0, which, since Ap is a
constantby (8. 71), means that
for all t E {0 , T ]
Ap ( t ) = 0
But the zero value for Ap would imply, by (8.68'), that UcC'(E) = 0, which
contradicts the assumptions that Uc and C' are both positive. Conse­
quently, an interior solution for
must be ruled out in the present model.
A*
PART 3: OPTIMAL CONTROL 'ftB!QilY
238
Boundary Solutions for Control
Variable A
The only viable policies are therefore the boundary solutions A* = 0 (jO
attempt to fight pollution) or A* = A (abatement of pollution at the ma:IJ.
mal feasible rate). These two policies share the common feature that
[from ( 8 .67)]
( 8.73)
Interpreted economically, the effect of energy use on utility through con­
sumption, UcC'(E), should under both policies be equated to the shadow
value of the depletion of the energy resource, measured by A8, adjusted for
the shadow value of pollution via the -aAp term. But the two policies differ
in that
( 8.74)
A* =
{1}
is associated with
>.. 8
{ � } - f3>.. p
[by (8.68)]
The economic meaning of the first line in (8. 74) is that a hands-off policy on
pollution is suited to the situation where the shadow price of energy
resource, As, exceeds that of pollution abatement, measured by -f3Ap. In
this situation, it is not worthwhile to expend the resource on antipollution
activities because the resource cost is greater than the benefit. But, as
shown in the second line, fighting pollution at the maximal rate is justified
in the opposite situation. In distinguishing between these two situations,
the parameter /3, which measures the efficacy of antipollution activities,
plays an important role.
Let us look further into the implications of the A* = 0 case. Since
the pollution stock is left alone in this case, the terminal stock of pollu­
tion is certainly positive. With P(T) > 0, the transversality stipulation
P(T)Ap(T) = 0 in (8.72) mandates that Ap(T) = 0. This suggests that Ap,
the shadow price of pollution, which is initially negative, should increase
over time to a terminal value of zero to satisfy the complementary-slackness
condition. That is,
Ap > o
Now if we differentiate (8. 73) totally with respect to t, we find
(UccC' 2 + UcC")E
= -aAp
Since the expression in parentheses is negative, E
sign. So we can conclude that
and
A
p
have the same
The increasing use of energy over time will, in the present case, lead to the
exhaustion of the energy resource. As (8.70) shows, As is a constant-a
239
CHAPTER 8: MORE ON OPTIMAL CONTROL
positive constant because As denotes the shadow price of a valuable re­
source. The positivity of the As constant means that, in order to satisfy the
transversality-condition stipulation
S ( T ) As ( T )
=
0
[cf. ( 8.72)]
S(T) must be zero, signifying the exhaustion of the energy resource stock at
the terminal time T.
Turning to the other policy, we have pollution abatement at the
maximal feasible rate, A* = A. Despite the maximal effort, however, we do
not expect the antipollution activities to run the pollution stock P all the
way down to zero. Upon the assumptions that Up < 0 and Upp < 0, we
know that, as P steadily diminishes as a result of antipollution activities,
Up will steadily rise from a negative level toward zero. When the P stock
becomes sufficiently small, Up will reach a tolerable level where we can no
longer justify the resource cost of further pollution-abatement effort. There­
fore, we may expect the terminal pollution stock P(T ) to be positive. If so,
then Ap(T) 0 by the complementary-slackness condition, and the rest of
the story is qualitatively the same as the case of A* = 0.
=
EXERCISE 8.5
1
2
;' ,f . -
•
S
What will happen to the optimal solution if the control region in the
Forster model for control variable A in (8.65) is changed to A ;;::; 0?
Let a discount factor e -P•, (p
> 0), be incorporated into problem (8.65).
(a) Write the current-value Hamiltonian He for the new problem.
(b) What are the conditions for maximizing He?
(c) Rewrite these conditions in terms of Ap and A s (instead of m p and
m8), and compare them with (8.67) and (8.68) to check whether the
new conditions are equivalent to the old.
In the new problem with the discount factor in Prob. 2, check whether it is
possible to have an interior solution for the control variable A. In answering
this question, use the regular (present-value) Hamiltonian H, and assume
that there exists a tolerably low level of P below which the marginal
resource cost of further pollution abatement effort does not compensate for
the marginal environmental betterment.
' ;�
·. ••
:�
;;.!
CHAPTER
9
INFINITEHORIZON
PROBLEMS
In our discussion of problems of the calculus of variations, it was pointed
out that the extension of the planning horizon to infinity entails certain
mathematical complications. The same can be said about problems of
optimal control. One main issue is the convergence of the objective func­
tional, which, in the infinite-horizon context, is an improper integral. Since
we have discussed this problem in Sec. 5.1, there is no need to elaborate on
the topic here. But the other major issue-whether finite-horizon transver­
sality conditions can be generalized to the infinite-horizon context-deserves
some further discussion, because serious doubts have been cast upon their
applicability by some alleged counterexamples in optimal control theory. We
shall examine such counterexamples and then argue that the counterexamples may be specious.
9.1 TRANSVERSALITY CONDITIONS
In their celebrated book, Pontryagin et al. concern themselves almost
exclusively with problems with a finite planning horizon. The only digres­
sion into the case of an improper-integral functional is a three-page discus­
sion of an autonomous problem in which the " boundary condition at
240
CHAPTER 9: INFINITE-HORIZON PROBLEMS
241
infinity" is assumed to take the specific form1
lim y(t) = yoo
( 9 .1)
t -><<>
In other words, the infinite horizon is specified to be accompanied by a fixed
terminal state. For that case, it is shown that the maximum-principle
condition-including the transversality condition for what we have been
referring to as the horizontal-terminal-line problem-apply as they do to a
finite-horizon problem. For the latter, the transversality condition is
[ H ]t-T = 0
When the problem is autonomous, moreover, the maximized value of the
Hamiltonian is known to be constant over time, so the condition
H = 0 can
be checked not only at t = T, but at any point of time at all. Accordingly,
the transversality condition pertaining to an infinite-horizon autonomous
problem with the boundary condition
lim
( 9 .2)
t � oo
(9.1) can be expressed not only as
H=O
but more broadly as
H= 0
( 9 .3)
for all
t
E
[ 0 , oo)
The fact that the transversality condition for the horizontal-terminal­
line problem can be adapted from the finite-horizon to the infinite-horizon
context makes it plausible to expect that a similar adaptation will work for
other problems. If the terminal state is free as t -> oo, for instance, we may
expect the transversality condition to take the form
( 9.4)
lim
t � oo
A(t)
=
0
[ transversality condition for free terminal state]
Similarly, i n case the variable terminal state i s subject to a prespecified
minimum level Ymin as t -> oo, we may expect the infinite-horizon transver­
sality condition to be
lim
(9 .5)
t -+<>0
A(t)
;::-:
0
and
lim
t � oo
A(t) [y( t) - Ymin] = 0
The Variational View
That
(9.4)
from an
finite-horizon
is a reasonable transversality condition can be gathered
adaptation of the procedure used in Sec.
7.3
to derive the
1L. S. Pontryagin et a!., The Mathematical Theory of Optimal Processes, Interscience, New
York, 1962, pp. 189- 191.
.
242
',. ,
PART 3: OPTIMAL CONTROL THEORY
transversality conditions for problem (7.20). There, we first combined-via
a Lagrange multiplier-the original objective functional with the equa­
tion of motion for into a new functional
y
V = t0 H(t,y, u ,) .) dt - (A(t)ydt
0
V
[from ( 7.22' ) ]
Then, after integrating the second integral by parts, we rewrote
V = ((H(t,y,
u , A) + y(t),\) dt - A(T)yr + A(O)y0
0
V as
[from ( 7.22'')]
To generate neighboring paths for comparison with the optimal control path
and the optimal state path, we adopted perturbing curves
for and
for to write
p(t) u
q (t ) y ,
u (t) = u* (t) + Ep (t) and y(t) = y*(t) + Eq(t)
[from ( 7.24) and ( 7 .25)]
Similarly, for variable T and
we wrote
[from ( 7 .26)]
T = T* + E ilT and
As a result, the functional V was transformed into a function �f E. The
Yr•
Jr = Yr* + E ilYr
first-order condition for maximizing V then emerged as
, !JJ
�·I:;
::
=
�T[(:� + A) q(t) + :�p(t) ] dt + [H]1_r!lT - A(T) !lyr = O
[from ( 7 .30)]
When adapted to the infinite-horizon framework, this equation be·I f1· comes
(9.6)
aHP (t) dt + lim H!lT - lim A(t) !lyT
aH + A } q(t) + &;;
dV ®[ ( ay
d E J0
J
.
t�®
�} jl
·· · ; -·
, '"
=0
nl
!1 2
t � oo
!13
In order to satisfy this condition, each of the three component terms
(01, !12, !13) must vanish individually. It is the vanishing of the last two
terms, !12 and !13, that gives rise to transversality conditions.
The Variability of T and Yr
For an infinite-horizon problem, the terminal time T is" not fixed, so that
!lT is nonzero. Thus, to make the fi2 term in (9.6) vanish, we must impose
CHAPTER 9: INFINITE-HORIZON PROBLEMS
243
the condition
( 9 . 7)
lim H = O
[ infinite-horizon transversality condition]
/ -HX>
This condition appears to be identical with (9.2). Unlike the latter, however,
(9.7) plays the role of a general transversality condition for infinite-horizon
problems, regardless of whether the terminal state is fixed. It therefore
carries greater significance.
We have based the justification of this condition on the simple fact that
the terminal time is perforce not fixed in infinite-horizon problems. But a
full-fledged proof for this condition has been provided by Michel.2 The proof
is quite long, and it will not be reproduced here. Instead, we shall add a few
words of economic interpretation to clarify its intuitive appeal. If, following
Dorfman (see Sec. 8.1), we let the state variable represent the capital stock
of a firm, the control variable the business policy decision, and the F
function the profit function of the firm, then the Hamiltonian function
sums up the overall (current plus future) profit prospect associated with
each admissible business policy decision. So long as H remains positive,
there is yet profit to be made by the appropriate choice of the control. To
require H ---> 0 as t ---> oo means that the firm should see to it that all the
profit opportunities will have been taken advantage of as t ---> oo . Intuitively,
such a requirement is appropriate whether or not the firm's terminal state
(capital stock) is fixed.
With regard to the 03 term in (9.6), on the other hand, it does matter
whether Y r is fixed or free. Suppose, first, that the terminal state is fixed.
Then we have lim 1 - ., Yr = y.,, same as in (9. 1). Since this implies ll.yr = 0
at time infinite, the 03 term will vanish without requiring any restriction
on the terminal value of A. In contrast, with a free terminal state, we no
longer have ll.yr = 0 at time infinite. Consequently, to make 03 vanish, it is
necessary to impose the condition lim 1 _., A(t) = 0, and this provides the
rationale for the transversality condition (9.4).
Despite the preceding considerations, many writers consider the ques­
tion of infinite-horizon transversality conditions to be in an unsettled state.
The transversality condition (9. 7) is not in dispute. However, the transver­
sality condition (9.4)-lim1 _., A(t)
0-has been thrown into doubt by
several writers who claim to have found counterexamples in which that
condition is violated. If true, such counterexamples would, of course, dis­
qualify that condition as a necessary condition. We shall argue, however,
that those are not genuine counterexamples, because the problems involved
do not have a truly free terminal state, so that (9.4) ought not to have been
applied in the first place.
=
2
Phillipe Michel, "On the Transversality
Econometrica, July 1982,
pp.
975-985.
�- a IJIAlaite Horiaon Optimal Prablema,"
' . ' • ' · ".·..
244
9.2
PART 3, OPTIMAL CONTROL THEORY
SOME COUNTEREXAMPLES
REEXAMINED
The Halkin Counterexample
"'
Perhaps the most often-cited counterexample is the following autonomous
control problem constructed by Halkin:3
subject to
(9.8)
and
{ \..>
{(1 - y ) u dt
Maximize
0
y = ( l - y) u
y(O) = 0
u(t) E [0, 1]
Inasmuch as the integrand function F is identical with the f function
in the equation of motion, the objective function can be rewritten as
( 9.9)
1"'y dt = [y(t)] � =
0
lim y ( t )
t -HIO
- y(O) =
lim y(t)
t -t oo
' '
Viewed in this light, the objective functional is seen to depend exclusively on
the terminal state at the infinite horizon. For such a problem, known as a
terminal control problem, the initial and intermediate state values attained
on an optimal path do not matter at all, except for their role as stepping
stones leading to the final state value. Hence, to maximize (9.9) is tanta­
mount to requiring the state variable to take one specific value-the upper
bound of y-as its terminal value.
To find the upper bound of y, let us first find the y(t) path. The
equation of motion for y, which can be written as
y + u (t)y = u (t)
i s a first-order linear differential equation with a variable coefficient and a
3
Hubert Halkin, "Necessary Conditions for Optimal Control Problems with Infinite Horizons,"
Econometrica ,
March 1974, pp. 267-272, especially p. 271.
A slightly
different version of the
Halkin counterexample is cited in Kenneth J. Arrow and Mordecai Kurz,
the Rate of Return , and Optimal Fiscal Policy,
Public Investment,
Johns Hopkins Press, Baltimore, MD, 1970,
p. 46 (footnote). The Halkin counterexample has also been publicized in Akira Takayama,
Mathematical Economics,
2d ed., Cambridge University Press, Cambridge, 1985, p. 625; and
Ngo Van Long and Neil Vousden, " Optimal Control Theorems," Essay 1 in John D. Pitchford
and Stephen J. Turnovsky, eds.,
Applications of Control Theory to Economic Analysis,
Holland, Amsterdam, 1977, p. 16.
North­
245
CHAPTER 9: INFINITE-HORIZON PROBLEMS
variable term. Its general solution is4
y(t)
(9.10)
ce - fu dt + 1
= ke - f6 u d t + 1
( c arbitrary)
( k arbitrary)
=
I
�)111'•'•' ' '
Next, by setting t = 0 in the general solution and making use of the initial
condition y(O) = 0, we find
0
so that
(9.10')
k
=
=
y(O)
=
ke 0
+
1
=
k
+
1
- 1 . Therefore, the definite solution is
y(t) = 1 e - Jb u dt = 1 e - Z(t )
_
_
where Z(t) is a shorthand symbol for the definite integral of u .
Since u(t) E [0, 1], Z(t) is nonnegative. Hence, the value of e - Z<t> must
lie in the interval (0, 1], and the value of y(t) must lie in the interval [0, 1).
It follows that any u(t) path that makes Z(t) --> oo as t --> oo-thereby
making e - Z<t > --> 0 and y(t) --> 1-will maximize the objective functional in
(9.10') and be optimal. In other words, there is no unique optimal control.
Halkin himself suggests the following as an optimal control:
u*
u*
=
=
0
for t
1
for
E
[0, 1 ]
( 1 , oo)
tE
This would work because J� 1 dt is divergent, resulting in an infinite Z(t).
But, as Arrow and Kurz point out (op. cit . p. 46), any constant time path
for u ,
,
u* ( t )
=
u0
for all
t
(O < u 0 < 1)
would serve just as well. The fact that Arrow and Kurz chose u(t) = u 0,
where u 0 is an interior point in the control region [0, 1], makes it possible
to use the derivative condition aH;au = 0. Since the Hamiltonian for this
problem is
(9 .11)
H
=
( 1 - y)u
+
..\(1 - y) u
=
(1
+
..\ ) ( 1 - y) u
the condition aH;au = 0 means ( 1 + ..\ )(1 - y) 0, or more simply (1 + ..\ )
= 0, inasmuch as y i s always less than one. Hence, we find ,.\* (t) = - 1,
,
=
4
See Alpha C. Chiang, Fundamental Methods ofMathematical Economics, 3d ed., McGraw-Hill,
New York, 1984, Sec. 14.3, for the standard formula for solving such a differential equation.
The rationale behind the conversion of the indefinite integral Ju dt to the definite integral
J�u dt in (9.10) is discussed in op. cit., Sec. 13.3 (last subsection). The constant k differs from
the constant c, because it has absorbed another constant that arises in connection with t-he
lower limit of integration of the definite integral.
246
PART 3; OPTIMAL CONTROL THEORY
which contradicts the transversality condition (9.4), and seemingly makes
this problem a counterexample.
As pointed out earlier, however, the structure of the Halkin problem is
such that it requires the choice of the upper bound of y as the terminal
state, regardless of the y*(t) path chosen. Put differently, there is an
implicit fixed terminal state, even though this is not openly declared in the
problem statement. In view of this, the correct transversality condition is
not lim 1 _, ,. ,\(t) = 0 as in (9.4), but rather lim 1 _, ,. H = 0 as given in (9.2).
More specifically, we should use the condition H = 0 for all t as in (9.3),
because the problem is autonomous.
Using the transversality condition H = 0 for all
we can approach the
Halkin problem as follows. First, noting that the Hamiltonian is linear in u ,
we only need to consider the boundary values of the control region as
t,
1.
candidates for u*(t), namely, either u = 0 or u =
The consequence of
choosing u = 0 at any point of time, however, is to make y = 0 (by the
equation of motion) at that point of time, thereby preventing y from
growing. For this reason, we must avoid choosing u (t) = 0 for all t.5
Suppose, instead, we choose the control
u*(t)
=
for all
1
t
Then we have the equations of motion
aH
A = -ay
y=
whoee
aH
-
o,\
( 1 + ,\ ) u
=
=
( 1 - y) u =
=
1+A ' ·
1 -y
� solutions are, respectively,
,\ ( t ) = Ae1
y(t)
By the initial condition
= Be -
y(O)
=
-1
1
+
0,
=
( '"·····
.::•;
.'
.. .,. . . ,
:·
·
( A arbitrary)
1
we
(B
can
:lefinite solution of the state variable is
y(t)
:
1 - e- 1
arbitrary)
determine that· B
= - 1,
·
·
with lim
1 ->«>
y(t)
=
·
1!10
r ·.·
the
.-
'
1
fhis latter fact shows that the control w e have chosen,
u*(t) =
1, can
naximize the objective functional just as those of Halkin and Arrow and
Kurz. What about the arbitrary constant A? To definitize A, we call upon
'The choice of
u
-0
for some limited initial period
� � dOiee Jio· harm, ���ne·e ollly tbe
IIIJIIIIItlded. by Halldn doel �
erminal state counts in the present problem. The eohrtion
he feature that u
- 0 initially, for t E [0, 1].
'
CHAPTER 9: INFINITE-HORIZON PROBLEMS
247
the transversality condition H = 0 for all t. From (9.11), we see that, with
u = 1 and y < 1 for all t, the only way to satisfy the condition H = 0 is to
have ,.\
1 for all t. Hence, A = 0.
Even though our optimal control differs from that of Halkin and that
of Arrow and Kurz, the conclusion that ,\*(t) = - 1 is the same. Yet, we do
not view this as a violation of an expected transversality condition. Rather,
it represents a perfectly acceptable conclusion drawn from the correct
transversality condition. In this light, Halkin's problem does not constitute
a valid counterexample.
Some may be inclined to question this conclusion on grounds that the
requirement on the terminal state, being based on the maximization of
(9.9), is in the nature of an outcome of optimization rather than an implicit
restriction on the terminal state. A moment's reflection will reveal, how­
ever, that such an objection is valid only for a static optimization problem,
where the objective is merely to choose one particular value of y from its
permissible range. But in the present context of dynamic optimization, the
optimization process calls for the choice of, not a single value of a variable ,
but an entire path. I n such a context, any restriction o n the state variable at
a single point of time (here, the terminal time) must, as a general rule, be
viewed as a restriction on the problem itself. The fact that the problem is a
special case, with the objective functional depending exclusively on the
terminal state, in no way disqualifies it as a dynamic optimization problem;
nor does it alter the general rule that any requirement placed on the
terminal state should be viewed as a restriction on the problem.
=
-
Other Counterexamples
In another alleged counterexample, Karl Shell6 shows that the transversal­
ity condition (9.4) does not apply when the problem is to maximize an
integral of per-capita consumption c(t) over an infinite period of time:
J; c(t) dt. But we can again show that the problem has an implicit fixed
terminal state, so that (9.4) should not have been expected to apply in the
first place.
The Shell problem is based on the neoclassical production function
Y Y( K, L), which, on the assumption of linear homogeneity, can be
rewritten in per-capita (per-worker) terms as
=
y=
where y = YjL and
q,(k )
with
f/J' ( k )
> 0,
f/J"( k )
<0
k = KjL . Let n denote the rate of growth of labor, and
6Kari Shell, "Applications of Pontryagin's Maximum Principle to Economics," in H. W. Kuhn
and G. P. Szegii, eds.,
Mathematical Systems Theory and Economics,
New York, 1969, pp. 241-292, especially pp. 273-275.
Vol. 1, Springer-Verlag,
248
PART 3: OPTIMAL CONTROL THEORY
let � denote the rate of depreciation of capital. Then the rate of growth of k,
the capital-labor ratio, is
(9.12)
,.,.
k = cP( k ) - c - ( n
+
8)k
which is similar to the funoamental differential equation of the Solow
growth model.7 In a steady state, we should have k = 0. By setting k = 0
and transposing, we obtain the following expression for the steady-state
per-capita consumption:
(9.13)
c = q,( k ) - (n + �)k
[steady-state value of c ]
Note that the steady-state value of c is a function of the steady-state value
of k. The highest possible steady-state c is attained when dcjdk = q,' (k) ­
> ( n + �) = 0, that is, when
(9.14)
q,'( k ) = n + �
The condition given in (9.14) is known as the golden rule of capital
.>·d•· .. accumulation. 8 Since q,(k) is monotonic, there is only one value of k that
·
·
will satisfy (9.14). Let that particular value of k-the golden-rule value of
k-be denoted by k , and let the corresponding golden-rule value of c be
denoted by c. Then
(9.15)
c = q,( k ) - ( n + �)k
[by (9.13)]
This value of c represents the maximum sustainable per-capita consump­
tion stream, since it is derived from the maximization condition dcjdk = 0.
Now consider the problem of maximizing J; c(t) dt. Since this integral
obviously does not converge for c(t) ;;::, 0, Shell adopts the Ramsey device of
rewriting the integrand as the deviation from " Bliss"-with the " Bliss"
identified in this context with c. In other words, the problem is to
(9.16)
Maximize
{0 < c - c) dt
subject to
k = q,( k ) - c - ( n + �) k
k(O) = k0
0 :o; c(t) :S q, [ k ( t)]
and
7
For details of the derivation, see the ensuing discussion leading to (9.25).
8See E. S. Phelps, " The Golden Rule of Capital Accumulation: A Fable for Growthmen,"
American Economic Review, September 1961, pp. 638-643. This rule is christened the "golden
rule" because it is predicated upon the premise that each generation is willing to abide by the
golden rule of conduct ("Do unto others as you would have
uniform savings ratio to be adhered to by all generations.
others do unto you.") and adopt a
249
CHAPTER 9: INFINITE-HORIZON PROBLEMS
Although the state variable, k , has a given initial value k0, no terminal
value is explicitly mentioned. The control variable,
is confined at every
point of time to the control region [0, <f>(k)], which is just another way of
saying that the marginal propensity to consume is restricted to the interval
[0, 1] . For this autonomous problem, the Hamiltonian is
c,
H = c - c + -\(<f>( k ) - c - ( n + o)k]
( 9 . 17)
Shell shows that the optimal path o f the costate variable has the limit value
1
as
t -+ oo, and the optimal state path is characterized by lim 1 � ., k(t) = k .
A(t) optimally tends to a unit value instead of zero leads Shell
The fact that
to consider this a counterexample against the transversality condition (9.4).
However, the Shell problem in fact also contains an implicit fixed
terminal state, so that the appropriate transversality condition is
all t, rather than lim1
A(t)
0.
H = 0 for
To see this, recall that the very reason for
rewriting the objective functional as in (9.16) is to ensure convergence.
Since the integrand expression
- c), a continuous function of time,
maintains the same sign for all
and offers no possibility of cancellation
_ .,
=
(c
t
between positive and negative terms associated with different points of time,
the only way for the integral to converge is to have
- c) -> 0 as -> oo.9
Hence,
must tend to c and, in line with (9. 15),
must tend to k , as
-> oo. What this does is to fix the terminal state, if only implicitly, in the
(c
k(t)
c(t)
t
t
form of lim 1 � ,.,
k, thereby disqualifying (9.4) as a transversality
condition. The important point is that this terminal state does not come
k(t) =
about as the result of optimization; rather, it is a condition dictated by the
convergence requirement of the objective functional.
We can easily check the correct transversality condition
0 for
t ->
oo. As
t
becomes infinite,
c
and
k
will take the values
H=
c and
k,
respectively; moreover, c and k will be related to each other by (9_15).
Hence, from (9.17), it i s clear that
will tend t o zero. Since, for this
H
H
autonomous problem,
should be zero for all
by setting (9. 17) equal to zero, to get
t, we can actually solve for A
c - c(t)
A( t) -- ---;-:-....,.,--.--;
----:-:--:4> [ k ( t)] - c(t)- (n +---:o)k(t)
( 9 . 18)
By taking the limit of this expression, we can verify Shell's result that
1 as -> oo. 1° From our perspective, however, this result by no
A(t) ->
t
9
See the discussion of Condition II in Sec. 5.1.
10
To do so, we can resort to L'Hopital's rule, since both the numerator and the denominator in
(9.18) approach zero
as
lim
t - oo
t
becomes infinite. Recalling that
A ( t)
=
lim
t - oo
.
- i:
f'(k)k - i: - ( n
+
(9.14) is· satisfied as t -+ oo, we have
a) k
.
=
- i:
lim -;- = 1
t - oo - c
PART 3: OPTIMAL CONTROL THEORY
250
means contradicts any applicable transversality condition. Rather, it is
perfectly consistent with, and indeed follows from, the correct transversality
condition, H = for all t.
Another intended counterexample is found in the regional investment
model of John Pitchford.U Suppressing population growth, he considers the
problem of choosing
and 12, the rate of investment in each of two
regions, so as to
0
11
Maximize
subject to
{9.19)
{lc(t ) - c] dt
K l = [1 - 5K l
/(2 = 12 - 5Kz
c �1( K1) + �2(K2) - l1 - l2 � 0
0
· ·£ "
=
and
K1(0) > 0 K2 (0) > 0
11 � 0 12 � 0
The symbol c again denotes the maximum sustainable level of consumption.
The model is essentially similar to the Shell counterexample, ·except that
there are here two control variables,
and 12 , and two state variables, K1
and
as well as certain inequality constraints (a subject to be discussed
in the next chapter). Pitchford also assumes implicitly that the terminal
states are constrained to be nonnegative.
The solution to this problem involves the terminal states
> and
> defined by the relations
[cf. (9.14); here,
n = Since the terminal value of each of the costate variables is found to
be one, the transversality condition implied by (9.5) for nonnegative termi­
nal capital values, namely, lim , - �
is not satisfied (Pitchford,
op .
p. 143). This, of course, would not have come as a surprise, if the
objective functional had been required to converge. For then c(t) must tend
to c, and
must tend to
thereby making the problem one with fixed
terminal states, to which a different type of transversality condition should
apply. However, not wanting to rule out the possible equilibria
and
a priori, Pitchford decides to proceed on the assumption
that the integral is not required to converge-which, however, makes one
wonder why he would then still employ the Ramsey device of expressing the
integrand as c(t) - c, a device explicitly purported to ensure convergence.
Because of this assumption, it takes a rather lengthy excursion before he is
[1
K2 ,
K2 0,
0].
cit.,
(0, K2 ),
�1'(K1) �2'(K2)
=
=
5
K1 0
Ai(t)Ki(t) 0,
=
Ki
(K11 0)
Ki,
(0, 0),
� ._d
11John D. Pitchford, " Two State Variable Problems," Essay 6 in John D.
Stephen J. Turnovsky, eds. , Applications of Control Theory to Economic An8l,ai., North­
Holland, Amsterdam, 1977, pp. 127-154, especially pp. 134-143.
251
CHAPTER 9: INFINITE-HORIZON PROBLEMS
led back to the conclusion that the objective functional has to converge after
all. The upshot is then the same: The terminal states are implicitly fixed.
And the model is not a genuine counterexample.
The Role of Time Discounting
In a noteworthy observation, Pitchford points out (op. cit., p. 128, footnote)
that all the known counterexamples apparently share one common feature:
No time discounting is present in the objective functional. While Pitchford
offers no reason for this phenomenon, he reports a finding of Weitzman12
that, for discrete-time problems, the transversality condition involving the
costate variables indeed becomes necessary when there is time discounting
and the objective functional converges. Moreover, Pitchford conjectures that
a similar result would hold for the continuous-time case. The relevant work
on the continuous-time case comes later in a paper by Benveniste and
Scheinkman. 13
What our foregoing discussion can offer is a simple intuitive explana­
tion of the link between the absence (presence) of time discounting on the
one hand and the failure (applicability) of the transversality condition (9.4)
or (9.5) on the other. Consider the case of maximizing J;[c(t) - c] dt. If
this functional is required to converge, and given that no possibility exists
for cancellation between positive and negative terms, the integrand [c(t) - c]
must tend to zero as t � oo. Since the only way for this to occur is for c(t) to
approach c, the terminal state for the problem is in effect fixed, whether or
not explicitly acknowledged in the problem statement. But if time discount­
ing is admitted, so that the integral is, say, J;[c(t) - c]e-pl dt, then conver­
gence will merely call for the vanishing of [c(t) - c]e-p1 as t -> oo, which can
be achieved as long as [ c(t) - c] tends to some finite number, not necessar­
ily zero. Inasmuch as c(t) is no longer obliged to approach a unique value c
for convergence, and nor is K(t) obliged to approach a unique value K, the
problem with time discounting would indeed be one with a free terminal
state, with (9.4) or (9.5) as its expected transversality condition. The Shell
and Pitchford models omit time discounting, and thus do not fall into this
category. The case where the objective functional does contain a discount
factor is illustrated by the neoclassical optimal growth model which we shall
discuss in Sec. 9.3.
12
M. L. Weitzman, " Duality Theory for Infinite Horizon Convex Models,"
Science, 1973,
pp.
Management
783-789.
13L. M . Benveniste and J. A. Scheinkman, " Duality Theory for Dynamic Optimization Models
of Economics: The Continuous Time Case,"
Journal of Economic Theory, 1982,
pp.
1-19.
PART 3o OPTIMAL CONTROL THEORY
252
The Transversality Condition as Part
of a Sufficient Condition
Although the transversality condition lim,
A(t) = 0 is not universally
accepted as a necessary condition for infinite-horizon problems, a limit
condition involving A does enter as an integral part of a concavity sufficient
condition for such problems. In an amalgamated form-combining the
concavity provisions of Mangasarian as well as those of Arrow-the suffi­
ciency theorem states that the conditions in the maximum principle are
sufficient for the global maximization of V in the infinite-horizon problem
�oo
{F(t , y, u ) dt
Maximize
V=
subject to
y = f( t , y, u )
and
0
y(O) = Yo
( Yo given)
provided
( 9.20)
either H = F(t, y, u) + A f(t, y, u) is concave in (y, u ) for all
t E [0, T], or H 0 = F(t, y, u*) + A f(t, y, u*) is concave in y
for all t, for given A
and
( 9 .21 )
lim
1 -><0
A(t) [y(t) - y* (t)]
� 0
where y*(t) denotes the optimal state path and y(t) i s any other admissible
state path. The terminal state can be either fixed or free.
Note that in (9.20) we have merged Mangasarian's concavity condi­
tions on the F and f functions and nonnegativity condition on A(t) into a
single concavity condition on the Hamiltonian H. The concavity of H is in
(y, u), jointly. In contrast, Arrow's condition is that H 0 be concave in the y
variable alone.
The limit expression in (9.21) is the infinite-horizon counterpart of the
expression A*(TXyr - Yr *) which we have encountered earlier in the proof
of the Mangasarian theorem [see footnote related to (8.31) and (8.3Y)]. In
that proof, the expression just cited is shown to vanish, so that the result
V s; V* emerges, establishing V* to be a global maximum. But, to establish
the maximality of V*, it is in fact also acceptable to have A*(T )(yr - Yr *)
positive. It is along the same line of reasoning that condition (9.21) restricts
the limit of A(t)[y(t) - y*(t)] to be either positive or zero as t ...... oo. The
reader may also find it of interest to compare (9.21) with (5.44), the
corresponding condition in the calculus-of-variations context. At first glance,
condition (5.44) may strike one as entirely different from (9.21) because it
shows the " s; " rather than the " � " inequality. However, from (7.57) we
CHAPTER 9: INFINITE-HORIZON PROBLEMS
253
recall that A
- FY
Since multiplying an inequality by a negative number
reverses the sense of inequality, it � ·� that (9;tl) aMI (1t44) are
;.. -- · :· · ,;,'· ·· ::.,. ... ;_, · •,-,:·. ·y ; . . ,
.. ,
indeed exactly the same.
=
•.
,_ .
9.3
THE NEOCLASSICAL THEORY
OF OPTIMAL GROWTH
The Ramsey model of saving behavior (Sec. 5.3), which deals with the
important issue of intertemporal resource allocation, has exerted a strong
influence on economic thinking, although this influence did not come into
being until after World War II, when that model was "rediscovered" by
growth theorists long after its publication. In a more recent development,
the same basic issue is formulated as a problem of optimal control rather
than the calculus of variations. Moreover, the new treatment-labeled as
" the neoclassical theory of optimal growth" -extends the Ramsey model in
two major respects: (1) The labor force (identified with the population) is
assumed to be growing at an exogenous constant rate n > 0 (the Ramsey
model has n = 0), and ( 2) the social utility is assumed to be subject to time
discounting at a constant rate p > 0 (the Ramsey model has p
0). Our
discussion of this subject will be based primarily on a classic paper by David
Cass.14
=
The Model
This theory is labeled a " neoclassical" theory, because its analytical frame­
work revolves around the neoclassical production function Y Y( K, L),
assumed to be characterized by constant returns to scale, positive marginal
products, and diminishing returns to each input. 15 Such a production
function, being linearly homogeneous, can be rewritten in per-worker terms
-or per-capita terms, as we shall draw no distinction between the popula­
tion and the labor force. Letting the lowercases of Y and K be defined,
respectively, as
=
(average product of labor)
( capital-labor ratio)
14 David Cass, " Optimum Growth in an Aggregate Model of Capital Accumulation," Review of
1965, pp. 233-240.
15
In discussing the Ramsey model, we denoted output by Q. Here we use instead the notation
Y, which is another commonly used symbol for income and output, especially at the macro
leveL
Economic Studies, July
PART 3: OPTIMAL C9NTROL THEORY
254
0 .______
___ __ k = !£
L
FIGURE 9.1
we can express the production function by
(9.22)
y
=
cf>( k )
for all k > 0
with cf>'( k ) > 0 and cf>"( k ) < 0
Additionally, it is stipulated that
lim cf>' ( k )
k --+ 0
= oo
and
lim cf>'( k)
k .-+ CO
=
0
The graph of cf>(k)-the APPL curve plotted against the capital-labor ratio
-has the general shape illustrated in Fig. 9.1.
The total output Y is allocated either to consumption C or gross
investment Ig. Therefore, net investment, I = K, can be expressed as
K=
Ig - oK =
Y-
C
-
[ o = depreciation rate]
oK
Dividing through by L, and using the symbol c = CjL for per-capita
consumption, we have
(9.23)
1 .
-K = y
L
-
c
-
ok
=
cf>( k )
-
c
-
ak
The right-hand side now contains per-capita variables only, but the left-hand
side does not. To unify the two sides, we make use of the relation
. dK d
dL
dk
K = dt = ( kL) = k dt + L
dt
dt
= knL + Lk
[product rule]
[ n dLidt ]
=
= L( kn + k )
By substituting this last result into (9.23) and rearranging, we 6nally obtain
an equation involving per-capita variables only:
·
(9. 24)
k = cf>( k ) - c - ( n + o ) k
· · '·. ' ·
255
CHAPTER 9: INFINITE-HORIZON PROBLEMS
This equation, describing how the capital-labor ratio k varies over time, is
the fundamental differential equation of neoclassical growth theory which
we have already encountered in (9.12).
The level of per-capita consumption, c, is what determines the utility
or welfare of society at any time. The social utility index function, U(c), is
assumed to possess the following properties:
U"( c ) < 0
U ( c) > 0
and
lim U'(c) = oo
c>0
lim U'(c)
0
for all
'
( 9.25)
=
c --> oo
c --+ 0
The U(c) function is, of course, to be summed over time in the dynamic
optimization problem. But, since the population (labor force) grows at the
rate n, Cass decides that the social utility attained at any point of time
should be weighted by the population size at that time before summing.
Hence, with a discount rate p, the objective function takes the form
...... ,.:
( 9 .26 )
= L0
j"" U( c ) e-<p-n)t dt
0
To ensure convergence, Cass stipulates that p - n > 0. It might be pointed
out, however, that this is equivalent to stipulating a single positive discount
rate r, where r = p - n . If, further, we let L 0
1 by choice of unit, then
the functional will reduce to the simple form
=
(r = p
( 9 .26' )
-
n > 0) ·
In other words, weighting social utility by population size and simultane­
ously requiring the discount rate p to exceed the rate of population growth
n, is mathematically no different from the alternative of not using popula­
tion weights at all but adopting a new, positive discount rate r. For this
reason, we shall proceed on the basis of the simpler alternative (9.26' ).
The problem of optimal growth is thus simply to
_, _ ,,.,.
:
'
, .. • ::_ ; . : , ,_ ,
Maximize
subject to
· , .
and
k = 4>( k ) - c - ( n
k(O) =
k0
0 ::::; c( t)
+
6) k
.; '�
::::; <i> [ k ( t)]
This resembles the Shell problem (9.16), but here the integrand function is
not (c - c) (the deviation of c from the golden-rule level), but a utility index
256
PART 3, 0PTOlAL CONTROL THEORY
H (and components)
H
\','· ''- :;
- U(c)e-rt '·''''
FIGURE 9.2
function of per-capita consumption, discounted at the rate
one state variable, k, and only one control variable, c.
r.
There is only
The Maximum Principle
The Hamiltonian for this problem,
(9.28)
H = U( c)e-rt + A [ 4>( k ) - c - ( n
+
o)k]
is nonlinear in c. More specifically, at any given point of time, the first
additive component of H would plot against c as the illustrative broken
curve in Fig. 9.2, and the second component would appear as the broken
straight line. 16 Their sum, H, is seen to contain a hump, with its peak
occurring at a c value between c = 0 and c = c1. Since c 1 is the solution
of the equation [4>(k) - c - (n + o)k] = 0, it follows that cl = cjJ(k) ­
(n + o)k, so that c 1 < <f>(k). Hence, the maximum of H corresponds to a
value of c in the interior of the control region [0, <f>(k)]. We can accordingly
find the maximum of H by setting
aH
iJc
- =
U'(c)e-rt - A = 0
From this, we obtain the condition
(9.29)
U'(c) = Aert
which states that, optimally, the marginal utility of per-capita consumption
should be equal to the shadow price of capital amplified by the exponential
16 The shape of the broken curve is based on the specification for U(c) in (9.25). The broken
straight line has vertical intercept A[<f>(k) - (n + 8)k) and slope - A . We take A to be positive
at any point of time, since it represents the shadow price (measured in utility) of capital at that
time; hence, the negative slope of the broken straight line.
257
CHAPTER 9: INFINITE-f!ORIZON PROBLEMS
term er t . Since a 2Hjac 2 = U"(c)e - r t is negative by (9.25), H is indeed
maximized.
The maximum principle calls for two equations of motion. One of
these, A = - aH;ak, entails for the present model the differential equation
A = - A [ cP' ( k ) - ( n + 8)]
( 9.30)
And the other,
( 9.31)
k = aH;aA,
simply restates the constraint
h = cP( k ) - c - ( n + o ) k
The three equations (9.29) through (9.31) should in principle enable us to
solve for the three variables c, A, and k. Without knowledge of the specific
forms of the U(c) and cP(k ) functions, however, we can only undertake a
qualitative analysis of the model.
It is, of course, also possible to work with the current-value Hamiltonian
(9.32)
He = U( c) + m [ cP( k ) - c - ( n + 8 ) k ]
In this case, the maximum principle requires that
or
(9.33)
[ by (8.10) ]
aHjac
=
U'(c)
-
m = 0,
m = U'( c)
This condition does maximize He, because a 2Hjac 2 = U"(c) < 0 by (9.25).
The equation of motion for the state variable k can be read directly
from the second line of (9.27), but it can also be derived as
(9.34)
. aHe
k = - = cP( k ) - c - ( n + o ) k
am
And the equation of motion for the current-value multiplier
( 9.35)
a He
m= + rm = - m [ cP' ( k ) - ( n + o ) ] + rm
ak
- m [ cP' ( k ) - ( n + a + r ) ]
m
is
[ by (8.13)]
The ensuing discussion will be based on the current-value maximum­
principle conditions (9.33) through (9.35). Since there is no explicit t
argument in these, we now have an autonomous system. This makes
possible a qualitative analysis by a phase diagra� , as we did in Sec. 5.4.
Constructing the Phase Diagram
Since the two differential equations (9.34) and (9.35) involve the variables k
and m , the normal phase diagram would be in the km space. To use such a
diagram, it would be necessary first to eliminate the other variable, c. But
PART So OPTIMAL CONTROL THEORY
258
(9.33),
function
since the other equation,
contains a
of c-to wit,
U '(c)- instead of the plain c itself, the task of eliminating c is more
complicated than that of eliminating m . We shall, therefore, depart from
Cass's procedure and seek instead to eliminate the m variable. In so doing,
we shall create a differential equation in the variable
as a by-product. The
analysis can then be carried out with a phase diagram in the kc space.
We begin by differentiating
with respect to
to obtain an
expression for rh :
c
(9.33)
m
t,
= U"( c)c
This, together with the equation m
enables u s t o
rh and m expressions. The result, after rearrangement, is
= U'(c),
rid (9.35) of the
which is a differential equation in the variab le c. Consequently, we now can
work with the differential equation system
k
=
cf>( k ) - c - ( n + 8)k
U' ( c)
c = U" [cf>'(,lt) - ( n + 8 + r)]
( 9.36)
-
(c)
T o construct the phase diagram, we first draw the
c = 0 curve. These are defined by the two equations
(9.37)
c = cf>( k) - ( n + 8)k
[
(9.38)
cf>'( k ) = n + 8 + r
[equation for
equation for
k
k
=
= 0 curve and tJie
0 curve]
c = 0 curve] 17
The k
0 curve shows up in the
space as a concave curve, as illustrated
in Fig.
As
indicates, this curve expresses c as the difference
between two functions of
The graph of
already
encountered earlier in Fig.
is reproduced in Fig.
And the
term simply gives us an upward-sloping straight line. Plotting the difference
between these two curves then yields the desired k
curve in Fig.
As to the
curve,
requires that the slope of the l/J(k) curve
assume the specific value
The
curve being monotonic, this
requirement can be satisfied only at a single point on that curve, point B,
=
9.3b.
c=0
17The
kc
k: cf>(k) and (n + 8)k.
9.1,
(9.38)
n + 8 + r.
9.3a.
l/J(k),
(n + 8)k
=0
: ,·f· ':_.
9.3b.
cf>(k)
U'(c)/U"(c) term is always positive, by (9.25), and can never be zero .
to vanish is to let the bracketed expression in (9.36) be zero.
only way for c
-
(9.37)
��
·
·
·
CHAPTER 9: INFINITE-HORIZON PROBLEMS
259
(n
+
li)k
�-""?r"- 1/J(k)
(a)
0
k
c
"
k
/,c=O
I
I
I
I
1\
c
1/J'(k)
D
(b) c
0
(Modified
golden
rule)
(Golden
rule)
=
n + li +
r
/h=O
c = 1/J(k) - (n
I
I
I
I
I
I
I
I
I
I
I
I
I
I
k
+
li)k
k
FIGURE 9.3
which corresponds to a unique k value, k. Thus the c = 0 curve must plot
as a vertical straight line in Fig. 9.3b, with horizontal intercept k.
The intersection of the two curves in Fig. 9.3b, at point E, determines
the steady-state values of k and c. Denoted by k and c, respectively, these
values are referred to in the literature as the modified-golden-rule values of
capital-labor ratio and per-capita consumption, as distinct from the golden­
rule values k and c discussed earlier in the Shell counterexample. As
indicated in (9.14), k is defined by 4>'<fl) = n + B. Thus it corresponds to
point A on the 4>(k) curve in Fig. 9.3a , where the tangent to that curve is
parallel to the (n + B)k line. In contrast, k is defined by 4>'(k ) = n + B + r ,
by (9.38), which involves a larger slope of 4>(k). This is why point B (for k )
must b e located to the left of point A (for k ). I n other words, the
modified-golden-rule value of k (based on time discounting at rate r) must
be less than the golden-rule value of k (with no time discounting). By the
same token, the modified-golden-rule value of c-the height of point E in
Fig. 9.3b-must be less than the golden-rule value-the height of point D.
PART 3: OPTIMAL CONTROL THEORY
260
Analyzing the Phase Diagram
To prepare for the analysis of the phase diagram, we have, in Fig. 9.4, added
vertical sketching bars to the k = 0 curve, and horizontal ones to the c = 0
curve. These sketching bars are useful in guiding the drawing of the
streamlines-to remind us that the streamlines must cross the k = 0 curve
with infinite slope, and cross the c = 0 curve with zero slope.
For clues about the general directions the streamlines should take, we
partially differentiate the two differential equations in (9.36), to find that
(9.39)
ak
ac
ac
-1 < 0
U'( c)
4>"( k ) < 0
(by (9 .25) and (9.22)]
ak
- U"(c)
According to (9.39), as c increases (going northward), k should follow the
( + , 0, - ) sign sequence. So, the k-arrowheads must point eastward below
the k = 0 curve, and westward above it. Similarly, (9.40) indicates that c
should follow the ( + , 0, -) sign sequence as k increases (going eastward).
Hence, the c-arrowheads should point upward to the left of the c 0 curve,
(9.40)
=
=
and downward to the right of it.
The streamlines drawn in accordance with such arrowheads Yield a
saddle-point equilibrium at point E, (k, c), where k and c denote the
intertemporal equilibrium values of k and c, respectively. The constancy of
k implies that y = cf>(k) is constant, too. Since k = KjL and y = YjL, the
simultaneous constancy of k and y means that, at E, the variables Y, K,
and L all grow at the same rate. The fact that Y and K share a common
growth rate is especially important as a sine qua non of a steady state or
c
FIGURE 9.4
261
CHAPTER 9: INFINITE-HORIZON PROBLEMS
growth equilibrium. Given the configurations of the k = 0 and c = 0 curves
in this problem, the steady state is unique.
Note that the only way the economy can ever move toward the steady
state is to get onto one of the stable branches-the "yellow brick
road"-leading to point E. This means that, given an initial capital-labor
ratio k0, it must choose an initial per-capita consumption level c0 such that
the ordered pair (k0, c 0 )- not shown in the graph-lies on a stable branch.
Otherwise, the dynamic forces of the model will lead us into a situation of
either (1) ever-increasing k accompanied by ever-decreasing c (along the
streamlines that point toward the southeast), or (2) ever-increasing c
accompanied by ever-decreasing k (along the streamlines that point toward
the northwest). Situation (1) implies progressively severe belt-tightening,
culminating in eventual starvation, whereas situation (2) implies growing
overindulgence, leading to eventual capital exhaustion. Since neither of
these is a viable long-run alternative, the steady state at E, with a sustain­
able constant level of per-capita consumption, is the only meaningful long­
run target for the economy depicted in the present model.
It may be noted that even at E, the per-capita consumption becomes
constant, and its level cannot be raised further over time. This is because a
static production function, Y = Y(K, L ), is assumed in the model. To make
possible a rising per-capita consumption, technological progress must be
introduced. This will be discussed in the next section.
Transversality Conditions
To select a stable branch from the family of streamlines is tantamount to
choosing a particular solution from a family of general solutions by defini­
tizing an arbitrary constant. This is to be done with the help of some
boundary conditions. The requirement that we choose a specific initial c0
such that the ordered pair (k0, c0) sits on the stable branch is one way of
doing it. An alternative is to look at an appropriate transversality condition.
Since we are dealing with general functions and not working with quantita­
tive solutions of differential equations, however, it is difficult to illustrate
the use of a transversality condition to definitize an arbitrary constant.
Nevertheless, we can, in light of the phase-diagram analysis, verify that the
steady-state solution indeed satisfies the expected transversality conditions.
One transversality condition we may expect the steady-state solution
to satisfy is that
(9.41)
A -> 0
as t -> oo
[by (9.4)]
This is because the objective functional does contain a discount factor and
the terminal !)tate is free. Since the solution path for A in the Cass model is
A* = U'(c* )e-rt
[ by (9.29)]
262
PART 3: 0PTDMAL CONTROL THEORY
and since the limit of U ' (c*) is finite as t --+ oo/8 this expression does satisfy
the transversality condition (9.41).
Another condition we may expect is that, in the solution,
H --+ 0
(9.42)
(by (9.7)]
as t -+ oo
For the present problem, the solution path for H takes the form
(by (9.28)]
H* = U( c* )e -'1 + A* ( <f>( k * ) - c* - ( n + B ) k * ]
Since U(c*) is finite as t -+ oo, the exponential term U(c*)e -'1 tends to zero
as t becomes infinite. In the remaining term, we already know from (9.41)
that A* tends to zero; moreover, the bracketed expression, representing k
by (9.36), is equal to zero by definition of steady state. Therefore, the
transversality condition (9.42) is also satisfied.
Checking the Saddle Point
by Characteristic Roots
In Fig. 9.4, the pattern of the streamlines drawn leads us to conclude that
the equilibrium at E is a saddle point. Since the sketching of the stream­
lines is done on a qualitative basis with considerable latitude in the position­
ing of the curves, it may be desirable to check the validity of the conclusion
by another means. We can do this by examining the characteristic roots of
the linearization of the (nonlinear) differential-equation system of the
model. 19
On the basis of the two-equation system (9.36):
L -
k = 4>( k ) - c - ( n + o) k
U ' ( c)
c = - " [ c/l' ( k ) - ( n + 5 + r)]
U (c)
l ]
we first form the Jacobian matrix and evaluate it at the steady-state point
E, or (k, c),
( 9 .�)
18
ak
ak
JE =
ac
ak
!
· ·:':.
From (9.25), we see that
does not tend to zero
as t
to have U'(c)
....,
...., oo,
ak
ac
ac
ac
;• I '
O< , c>
we must have
oo; thus the limit of
i'
<.i· ;·
U'(c*) is
c
....,
finite
0. In the present problem, c*
as t
...., oo.
19 For more details of the procedure of linearization of a nonlinear system, see Alpha C. Chiang,
Fundamental Methods of Mathematical Economics,
Sec. 18.6.
3d ed., McGraw-Hill, New York, 1984,
'
263
CHAPTER 9, INFINITE-HORIZON PROBLEMS
The four partial derivatives, when evaluated at E, where
turn out to be
::IE =
IE
cP'(k) - ( n + ll}
ak
=
ac
ac
ak
ac
-
ac
I
I
E
E
r > 0 '·· ·· · ·
--ll.·h
n
+S
+
r,
":·, ·_ , , ,�, -,��,·� -
-1 < o
U'( c)
cP"( k ) < 0
U"( c)
- [ U" ( c) ] 2 + U "' ( c) U'(c)
= =
=
4''(k) =
_
( U"(c)]
2
( cP' ( k ) - ( n + S + 1')] = 0
_
It follows that the Jacobian matrix takes the form
(9 .43')
The qualitative information we need about the characteristic roots r1
and r2 to confirm the saddle point is conveyed by that result that
(9 .44)
This implies that the two roots have opposite signs, which establishes the
steady state to be locally a saddle point.
EXERCISE
1
9.3
. '·
A
b
2
. . �-·
Let the production function and the utility function be
1 b
U = U - -c -
. -� •,
.,
{0 < a < 1)
( b > O)
..
( a ) Find the y = ,P(k ) function and the U'(c) function. Check whether
these conform to the specifications in (9.22) and (9.25).
(b) Write the specific optimal control problem.
(c) Apply the maximum principle, using the current-value Hamiltonian.
(d) Derive the differential-equation system in the variables k and c. Solve
for the steady-state values Oi, c).
In the phase-diagram analysis of the Cass model, directional arrowheads in
Fig. 9.4 are determined from the signs of the partial derivatives ak joe and
iJcjiJk in (9.39) and (9.40).
264
PART 3: OPTIMAL CONTROL THEORY
(a) Is it possible andjor desirable to accomplish the same purpose by using
\
okjiJk in lieu of ak;ac? Why?
(b) Is it possible andjor desirable to consider the ·sign of iJcjiJc in place of
ac;ak? Why?
In Fig. 9.4, suppose that the economy is currently located on the lower
stable branch and is moving toward point E. Let there be a parameter
change such that the c 0 curve is shifted to the right, but the k 0
curve is unaffected.
(a ) What specific parameter changes could cause such a shift?
(b) What would happen to the position of the steady state?
(c) Can we consider the preshift position of the economy-call it
( k 1, c1)-as the appropriate initial point on the new stable branch in
the postshift phase diagram? Why?
Check whether the Cass model (9.27) satisfies the Mangasarian sufficient
conditions.
=
4
9.4
=
EXOGENOUS AND ENDOGENOUS
TECHNOLOGICAL PROGRES S
The neoclassical optimal growth model discussed i n the preceding section
provides a steady state in which the per-capita consumption, c, stays
constant at c, leaving no hope of any further improvement in the average
standard of living. The culprit responsible for the rigid capping of per-capita
consumption is the static nature of the production function, Y = Y(K, L ).
Since the same production technology holds for all t, no upward shift over
time can occur. Once technological progress is allowed, however, we can
easily remove the cap on per-capita consumption.
"-j!
Dynamic Production Functions
For a general representation of a dynamic production function, we can
simply write
(9.4 5 )
Y = Y( K, L , t)
( ¥; > 0 )
The positive sign of ¥; shows that technological progress does take place,
but it offers no explanation of how the progress comes into being. This
representation can thus be used only for exogenous technological progress.
An alternative way of writing a dynamic production function is to introduce
a technology variable explicitly into the function. Let A = A(t) represent
the state of the art, with dAjdt > 0, then we can write
(9.46)
Y = Y( K, L , A)
The advantage of (9.46) over (9.45) is that, with an explicit variable A, we
265
CHAPTER 9: INFINITE-HORIZON PROBLEMS
now can either leave A as an exogenous variable or �e it endogenous by
postulating how A is determined in the model..
. ·. :�
Neutral Exogenous Technological
Progress
Technological progress is said to be " neutral" when it leaves a certain
economic variable unaffected under a stipulated circumstance. Specifically,
technological progress is Hicks-neutral if it leaves the marginal rate of
technical substitution (MRTS = MPPLfMPPK) unchanged at the same K1L
ratio. That is, if we hold KjL constant and examine the MRTS's before and
after progress, the two MRTS 's will be identicaL To capture this feature, we
can use the following special version of (9.46):
Y = A( t ) Y( K, L )
( 9 .47)
[Hicks-neutral]
where Y(K, L) is taken to be linearly homogeneous. The reason why the
variable A will not disturb the ratio of the marginal products of K and L is
that it lies outside of the Y(K, L) expression.
In contrast, technological progress is Harrod-neutral if it leaves the
output-capital ratio (YjK) unchanged at the same MPPK . The special
version of (9.46) that displays this feature is
Y = Y [ K, A( t)L]
(9.48)
[Harrod-neutral]
where Y is taken to be linearly homogeneous in K and A(t)L. Since A(t) is
attached excbsively to L, this type of technological progress is said to_ be
purely labor-augmenting. Considering the way A(t) and L are combined, we
may view technology and labor as perfect substitutes in the production
process. It is the fact that A is totally detached from the K variable that
explains how the YjK ratio can remain undisturbed at the same MPPK as
technological progress occurs.
A third type of neutrality, Solow neutrality, is the mirror image of
Harrod neutrality, with the roles of K and L interchanged. The function
can then be written as
Y = Y [ A( t ) K , L]
( 9.49)
where
·y
[Solow-neutral]
is taken to be linearly homogeneous in
AK
and
L.
Neoclassical Optimal Growth with
Harrod-Neutral Progress
In growth models with exogenous technological progress, Harrod neutrality
is frequently assumed. The main reason for the popularity of Harrod
neutrality is that it is perfectly consistent with the notion of a steady state;
it can yield a dynamic equilibrium in which Y and K may grow apace with
266
PART 3: OPTIMAL CONTROL THEORY
each other. As a bonus, it turns out that Harrod-neutral technological
progress entails hardly any additional analytical complexity compared with
the case where a static production function is used.
Let us define efficiency labor, 71, by
( 9 .50)
Then the production function (9.48) becomes
Y = Y( K, 71 )
( 9 .51)
If we now consider efficiency labor 71 (rather than natural labor L) as the
relevant labor-input variable, then (9.51) can be treated mathematically just
as a static production function, despite that its forebear-(9.48)-is really
dynamic in nature. By the assumption of linear homogeneity, we can
rewrite (9.51) as
where y
( 9 .5f )
'I
""
y
- and k ""
71
'I
K
-
71
This is identical in nature with (9 . 22), and we shall indeed retain all the
assumed features of cf> enumerated in connection with (9.22). The equation
of motion for k'l, the new state variable, can be found by the same
procedure used to derive (9.24). The result is
( 9 .52)
c
�
A
a=A
where c" "" 71
t
and n "" ­
L
The optimal control problem is therefore
Maximize
(9.53)
subject to
k"
=
cf>(k,.J - c'l - (e� + n + 3)i.,
k'I(O)
=
k'l0
and
The similarity between this new formulation and the old problem (9.27)
should be patently clear.
Since the current-value Hamiltonian in the new problem is
. '· ,-:.
( 9.54)
where
m' is the new current-value costate variable, the maximum principle
267
CHAPTER 9: INFINITE-HORIZON PROBLEMS
calls for the following conditions:
( 9 .55)
(9.56)
( 9.57)
m' =
U'( c..,)
[cf. ( 9 .33) ]
k.., = c/J( k..,) - c.., - (a + n + S ) k ..,
rh' =
-
m'
[ c/J ( k ..,)
'
-
[ cf. (9 .34)]
(a + n + a + r)j
[cf. ( 9 .35)]
By eliminating the m' variable, the last three equations can be condensed
into the differential-equation system
k.., = lf>( k..,) - c.., - ( a + n + S) k..,
( 9 .58)
c.., = -
U'( c.,)
[c/J' ( k .., ) - ( a + n + a + r)j
"
U ( c.., )
[cf. ( 9 .36)]
These then give rise to the following pair of equations for the new phase
diagram:
(9.59)
c.., = lf>(k..,) - (a + n + S)k..,
( 9.60)
c/J'(k..,) = a + n + a + r
( equation for k.., = 0 curve]
[ equation for c.., = 0 curve]
Since the phase-diagram analysis is qualitatively the same as that of
Fig. 9.4, there is no need to repeat it. Suffice it to say that the intersection
of the k.., = 0 curve and the c.., = 0 curve delineates a steady state in which
Y, K, and 7J all grow at the same rate. But one major difference should be
noted about the new steady state. Previously, we had c = CjL constant in
the steady state. Now we have instead
(9.61)
c
c
c.., = - = 7J
=
AL
c
L
.
:.. �.1 .
which implies that
(9.6f)
;_.":· ·,,�,.,.,,
conllt.ut'
- = c
.., A
.
J
� j>'
--.-. ..
Thus, as long as A increases as a result of technological progress, per-capita
consumption C1L will rise over time pari passu . The cap on the average
standard of living has been removed.
Endogenous Technological Progress
While exogenous technological progress has the great merit of simplicity, it
shirks the task of explaining the origin of progress. To rectify this shortcom­
ing, it is necessary to endogenize technological progress. As early as the
1960s, economists were already exploring the economics of knowledge ere-
PART 3o OPTIMAL CONTROL THEORY
268
ation. The well-known " learning-by-doing" model of Kenneth Arrow,20 for
example, regards gross investment as the measure of the amount of " doing"
that results in " learning" (technological progress). As another example, a
model of Karl Shell,21 which we shall now briefly discuss, makes the
accumulation of knowledge explicitly dependent on the amount of resources
devoted to inventive activity.
The production function in the Shell model,
Y(t )
=
Y [ K( t ) , L ( t ) , A(t ) ]
is the same as (9.46). But now the variable A(t), denoting the stock
knowledge, is given the specific pattern of change
(9.62)
of
(0 < u s 1 , 0 s a s 1 , {3 ;;:: 0)
A(t) = ua( t) Y( t ) - {3A( t )
where u is the research success coefficient, a(t) denotes the fraction of
output channeled toward inventive activity at time t, and f3 is the rate of
decay of technical knowledge. Out of the remaining resources, a part will be
saved (and invested). The K(t) variable thus changes over time according to
(9.63)
K(t) = s ( t ) [ 1 - a(t) ] Y( t) - 8K(t)
where s denotes the propensity to save and 8 denotes the rate of deprecia­
tion. In order to focus attention on the accumulation of knowledge and
capital, we assume that L is constant, and (by choice of unit) set it equal to
one.
In a decentralized economy, the dynamics of accumulation can be
traced from the equation system (9.62) and (9.63) in the two variables A
and K. If a government authority seeks to maximize social utility, on the
other hand, there then arises the optimal control problem
(9.64)
Maximize
['u[ ( 1 - s ) ( l - a ) Y ] e -p' dt
0
subject to
A = uaY( K, A ) - {3A
K = s( 1 - a ) Y( K, A) - 8K
and
A(O) = A0
K(O) = K0
The integrand in the objective functional may look unfamiliar, but it is
merely another way of expressing U(C)e -P1, because C is the residual of Y
20
Kenneth J. Arrow, " The Economic Implications of Learning by Doing,"
Studies, June 1 962,
21
pp. 155-173.
Karl Shell, " Towards a Theory of Inventive Activity
Economic Review,
May 1966, pp. 62-68.
·
and
Review of Economic
Capital Accumulation,"
American
CHAPTER 9: INFINITE-HORIZON PROBLEMS
269
after deducting what goes to inventive activity and capital accumulation:
C = Y - aY - s ( l - a ) Y
= ( 1 - s){l - a)Y
This problem contains two state variables ( A and K ) and two control
variables (a and s).
The maximum principle is, of course, directly applicable to this prob­
lem. But, with two state variables, the solution process is not simple. Shell
presents no explicit solution of the dynamics of the model, but shows that A
and K will approach specific constant limiting values A and K . There is no
need to enumerate these limiting values here, for their significance lies not
in their exact magnitudes, but in the fact that they are constant. Since
A _. A and K _. K as t _. oo, the specter of a capped standard of living is
here to visit us again.
One may wonder whether allowing a growing population and making
the social utility a function of per-capita consumption will change the
outcome. The answer is no. The difficulty, as Kazuo Sato (a discussant of
the Shell paper) points out,22 may lie in the specification of A in (9.62),
which ties A to Y. If (9. 62) is changed to
(9.65)
A = ua A - {3A
or
A
- = ua A
{3
then no cap on the growth of knowledge will exist so long as ua - {3 > 0.
Interestingly, a similar idea is featured in a recent model of endogenous
technological progress by Paul Romer to ensure unbounded progress.
Endogenous Technology a Ia Romer
Knowledge, according to the Romer model,23 can be classified into two major
components. The first component, which may be broadly labeled as human
capital, is person-specific. It constitutes a " rival good" in the sense that its
use by one firm precludes its use by another. The other component, to be
referred to as technology, is generally available to the public. It is a
" nonrival good" in the sense that its use by one firm does not limit its use
by others. Because of the rivalous nature of human capital, the person who
invests h,l the accumulation of human capital receives the rewards arising
therefrom. In contrast, the nonrivalry feature of technology implies knowl­
edge spillovers, such that the discoverer of new technology will not be the
22American Economic Review, May 1966, p. 79.
23 Paul M. Romer, " Endogenous Technical Change," Journal of Political Economy,
No. 5, Part 2, October 1990, pp. 871-8102.
Vol. 98,
270
PART 3, OPTIMAL CONTROL THEORY
sole beneficiary of that discovery. The inability of the discoverer to reap all
the benefits creates an economic externality that causes private efforts at
technological improvement to fall short of what is socially optimal.
Both human capital and technology are created by conscious action. To
reduce the number of state variables, however, human capital is simply
assumed to be fixed and inelastically supplied, although its allocation to
different uses is still to be endogenously determined. Romer denotes human
capital by H, but to avoid confusion with the symbol for the Hamiltonian,
we shall use S (skill or skilled labor) instead, with S 0 as its fixed total.
Since S can be used for the production of the final good, Y, or for the
improvement of technology, A (state of the art), we have
(9.66)
Technology A, on the other hand, is not fixed. It can be created by engaging
human capital SA in research and applying the existing tecliliology A u
follows:
(9.67)
where u is the research success parameter. This equation is closely similar
to (9.65), except that, here, human capital SA has replaced a (the fraction
of output channeled to research), and /3 = 0 (technology does not depreci­
ate). Note that A;A = uSA > 0 as long as both u and SA are positive.
Thus technology can grow without bound. Note also that research activity is
assumed to be human-capital-intensive and technology-intensive, with no
capital (K) and ordinary unskilled labor ( L) engaged in that activity.
In the production of the final good Y, however, K and L do enter as
inputs along with human capital Sy and technology A. The production
function for Y is developed in several steps. First, think of technology as
made up of an infinite set of "designs" for capital,
including those that have not yet been invented at the present time. If we
let x, = 0 for i � A, then A can serve as the index of the current technol­
ogy. The production function for the final good is assumed to be of the
Cobb-Douglas type:
where L0 indicates a fixed and inelastically supplied amount of ordinary
labor. For simplicity, Romer allows all the designs x, to enter in an
additively separable manner. As the second step, let the design index i be
271
CHAPTER 9: INFINITE-HORIZON PROBLEMS
turned into a continuous variable. The production function then becomes
(9.68)
Since all x(i) enter symmetrically in the integrand, we may deduce that
there is a common level of use, x, of all x(i). It follows that
)[AX ( t" ) l -a-f3 dt·
,0
=
=
Therefore, (9.68) can be simplified to
[A-l-a-f3 d"
)
,
0
t
X
;xt-a-{3 iA di
0
= AXJ -a-{3
(9.68')
Next, assume that (1) capital goods are nothing but foregone consumption
goods, both types of goods being subject to the same final-good production
function, and (2) it takes -y units of capital goods to produce one unit of any
type of design. Then the amount of capital actually used will be
K
=
-yAX
=>
K
x= -yA
Substituting"this into (9.68') results in
( 9.68" )
Y = Sy"'L /A
=
=
··· ,.
( K ) l -a - {3
-yA
-J
SyaL /Aa + f3K l - a-f3 -y a +{3
a
+ f3 - !
( S yA ) ( L o A ) f3 K l -a -f3-ya
In this last function, all four types of inputs are present: human capital Sy,
technology A , labor L 0 , and capital K. More importantly, we observe from
the (SyA) and ( L 0 A) expressions that technology can be viewed as human­
capital-augmenting as well as labor-augmenting, but it is detached from K.
In other words, it is characterized by Harrod neutrality-here endogenously
introduced rather than exogenously imposed. Our experience with Harrod
neutrality suggests that this production function is consistent with a steady
state in which technology-and, along with it, output, capital, and con­
sumption-can all grow without bound.
In this model, consumption C does not need to be expressed in
per-capita terms because L 0 (identified with population and measured by
head count) is constant. By the same token, K rather than K1L can
'
PART s, OPTIMAL CONTROL THEORY
272
appropriately serve as the variable for analysis. Net investment, as usual, is
just output not consumed. Thus, recalling that Sy = 80 - SA, we have
(9.69)
K= Y- C
The Optimal Control Problem
Facing this background, society may consider an optimal control problem
with two state variables, A and K-with (9.67) and (9.69) as their equa­
tions of motion-and two control variables, C and SA . Adopting the specific
constant-elasticity utility function
U( C ) =
c1-a
(0 < 8 < 1)
1-8
the control problem takes the form
(9.70)
"" c l - e
Maximize
f --e
-pt dt
1-8
subject to
A = uSA A
0
"
K = y"+P - !A<> +il ( S0 - SA) L rfKl -a-13 - C
A(O)
and
=
K(O) = K0
A0
For convenience, define the shorthand symbol
(9.71)
Then we have the current-value Hamiltonian
He =
c1-e
1
_
8
+ AA( uSAA ) + AK(d - C )
where AA and AK represent the shadow prices of A and K , respectively.
From this, we get the conditions
(9.72)
(9 .73)
oHc
ac
- =
c - e - ,\K =
0
=
CHAPTER 9: INFINITE-HORIZON PROBLEMS
273
In addition to the A and K equations given in the problem statement, the
maximum principle requires the following equations of motion for the
costate variables:
.
(9.74)
aHc
aA
aHc
AA = - - + pAA = -AAuSA - A K ( a + I') A - 1 A + pA A
.
Ax = -
aK
+ pAx = - A x ( l - a - 13 ) K - 1 A + pAx
The Steady State
Since there are four differential equations, the system cannot be analyzed
with a phase diagram. And solving the system explicitly for its dynamics is
not simple. Romer elects to focus the discussion on the properties of the
balanc�d growth equilibrium inherent in the model: a steady state with
Harrod-neutral technological progress. Questions of interest include: What
is the rate of growth in that steady state? How is the growth rate affected by
the various parameters? What economic policies can be pursued to promote
growth?
The basic feature of such a steady state is that the variables Y, K, A,
and C all grow at the same rate. We thus have in the steady state
( 9.75)
¥
K
6 A
- = - = - = - = uSA
y
K
c A
( by (9.67)]
But we want to express the growth rate in terms of the parameters only,
with SA substituted out. Since Ax = c-8 [by (9.72)], we can calculate
(9.76)
- oc-8- 1 6
c 8
6
c
= -0- =
- OuSA
If we get another Ax/Ax expression from the second equation in (9.74),
then we can equate it to (9. 76) and solve for SA' But that equation is not
convenient to work with. Instead, Romer first calculates AA /AA from the
first equation in (9.74), then takes advantage of the relation ix/Ax = >.AlA A
that characterizes the steady state. By dividing the AA expression in (9.74)
through by AA and simplifying, it is found that
(9.77)
Equating (9.76) and (9. 77) and solving for SA, we then ascertain that SA
PAR"r S: OJ"'''MAL COHTROL THBOKY
274
attains in steady state the constant value
(9 .78)
SA =
a ( a + {3) S0 - ap
-------
a ( a8 + {3 )
I t follows that the parametriC&llY expressed steady-state growth rate i s
Y
K
C
A
- =- = - = - =
(9.79)
Y
K
C
a ( a + {3 ) S0 - ap
A
aO + {3
The result in (9.79) shows not only the growth rate, but also how the
various parameters affect that rate. Visual inspection is sufficient to estab­
lish that human capital S 0 has a positive effect on the growth rate, as does
the research success parameter a, but a negative effect is exerted by the
discount rate p. More formally, the effects of the various parameters can be
found by taking the partial derivatives of (9.79).
To conclude this discussion, we should point out a complication regard­
ing the rate-of-growth expression in (9.79): A rate of growth should be a
pure number, and economic commonsense would lead us to expect it to be a
positive fraction. Yet the expression in (9.79) comprises not only the pure­
number parameters a, a, {3, p, and 8, but also the parameter S0 which has
a physical unit. The magnitude of the expression thus becomes problematic.
The problem may have started at an early stage with the specification of
(9.67), where SA• a variable with a physical unit, enters into the rate of
growth A;A. If so, the remedy would be to replace SA with a suitable ratio
figure in (9.67).
EXERCISE 9.4
Y = A(t)K"LfJ, the
1
Show that, in the Cobb-Douglas production function
2
Derive the equation of motion (9.52) by following the same procedure used
3
4
5
6
\·. .
technological progress can be considered to be either
( a ) Hicks-neutral, ( b )
Harrod-neutral, o r ( c ) Solow-neutral.
to derive (9.24).
Verify the validity of the maximum-principle conditions in (9.55), (9.56),
and (9.57).
Verify that the differential-equation system (9.58) is equivalent to the set
of equations (9.55), (9.56), and (9.57).
What would happen if (9.67) were changed to
concave function?
A
=
liSAI{l( A ), where 1{1 is a
In the Romer model, how is the steady-state growth rate specifically
affected by the following parameters?
(a} a
( b ) {3
(c) IJ
! '
1!1
�1
o il
n
�.�{
��
��
CHAPTER
10
OPTIMAL
CONTROL
WITH
CONSTRAINTS
Constraints have earlier been encountered in Sec. 7.4, where we discussed
various types of terminal lines, including truncated ones. Those constraints
are only concerned with what happens at the endpoint of the path, and the
conditions that are developed to deal with them are in the nature of
transversality conditions. In the present chapter, we turn to constraints
that apply throughout the planning period [0, T ].
As in the calculus of variations, the treatment of constraints in optimal
control theory relies heavily on the Lagrange-multiplier technique. But
since optimal control problems contain not only state variables, but also
control variables, it is necessary to distinguish between two major categories
of constraints. In the first category, control variables are present in the
constraints, either with or without the state variables alongside. In the
second category, control variables are absent, so that the constraints only
affect the state variables. '1\s we shall see, the methods of treatment for the
two categories are different.
10. 1
CONSTRAINTS INVOLVING
CONTROL VARIABLES
Within the first category of constraints-those with control variables pres­
ent-four basic types can be considered: equality constraints, inequality
constraints, equality integral constraints, and inequality integral con275
PART 3, OPTIMAL CONTROL THEORY
276
straints. We shall in general include the state variables alongside the control
variables in the constraints, but the methods of treatment discussed in this
section are applicable even when the state variables do not appear.
Equality Constraints
in
Let there be two control variables
required to satisfy the condition
u 1 and u2, that
a problem,
· :.,.
•
are
>)
We shall refer to the g function as the constraint function, and the constant
c as the constraint constant. The control problem may then be stated as
( 10.1)
,.
Maximize
j F( t, y, u 1, u 2) dt
subject to
Y
T
0
=
f( t, y , u1, u 2)
g( t, y, u 1, u2) = c
and
boundary conditions
This is a simple version of the problem with m control variables and q
equality constraints, where it is required that q < m .
The maximum principle calls for the maximization o f the Hamiltonian
( 10.2)
for every t E [0, T ]. But this time the maximization of H is subject to the
constraint g(t, y, u 1 , u 2) = c . Accordingly, we form the Lagrangian expres­
sion
J= H + 8(t) [ c - g(t , y, u 1 , u2) }
( 10.3)
= F( t , y , u 1 , u 2) + >. ( t) f( t, y , u 1 , u 2)
+ 8(t) [ c - g(t , y , u 1 , u2 )}
where the Lagrange multiplier 8 is made dynamic, as a function of t. This is
necessitated by the fact that the g constraint must be satisfied at every t in
the planning period. Assuming an interior solution for each u J • we require
that
( 10.4)
iJ../
=
iJu1
iJg
iJF
iJ f
+ >. - - 8 - = 0
iJ u i
iJ u 1
iJ u i
for all
t E [O, T ]
( j = 1, 2)
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
277
Simultaneously, we must also set
i)..£'
aiJ
( 10.5)
= c -
g(
t , y , u 1, u 2 )
=
0
for all t
E
[0, T ]
to ensure that the constraint will always be in force. Together, (10.4) and
(10.5) constitute the first-order condition for the constrained maximization
of H. This must be supported, of course, by a proper second-order condition
or a suitable concavity condition.
The rest of the maximum-principle conditions includes:
( 10.6 )
and
( 10.7)
y
=
( = ilH )
a../
ilA
[equation of motion for y]
aA
(
). = _ a../ = _ ilH
ay
ay
+
8 ag
ay
)
[equation of motion for A ]
plus an appropriate transversality condition. Note that the equation of
motion for y, (10.6), would turn out the same whether we differentiate the
Lagrangian function (the augmented Hamiltonian) or the original Hamilto­
nian function with respect to A . On the other hand, it would make a
difference in the equation of motion for A, (10. 7), whether we differentiate
.../" or H with respect to y. The correct choice is the Lagrangian expression
..£'. This is because, as the constraint in problem (10.1) specifically pre­
scribes, the y variable impinges upon the range of choice of the control
variables, and such effects must be taken into account in determining the
path for the costate variable A .
While it i s feasible to solve a problem with equality constraints in the
manner outlined above, it is usually simpler in actual practice to use
substitution to reduce the number of variables we have to deal with.
Substitution is therefore recommended whenever it is feasible.
Inequality Constraints
\
The substitution method is not as easily applicable to problems with in­
equality constraints, so it is desirable to have an alternative procedure for
such a situation.
We first remark that when the g constraints are in the ineq uality
form, there is no need to insist that the number of control variables exceed
the number of constraints. This is because inequality constraints allow us
much more latitude in our choices than constraints in the form of rigid
equalities. For simplicity, we shall illustrate this type of problem with two
PART 8: OPl'IMAL CONTROL TJJE<)RY
278
control
variables
and two inequality constraints:
Maximize
J TF( t,y, u1, u 2 ) dt
subject (o
Y
and
boundary conditions
( 10.8)
0
=
f(t,y, U v u 2 )
gl(t,y, u l, u 2 ) :;; cl
g 2 ( t, y, ul> u 2 ) :;; c 2
The Hamiltonian defined in (10.2) is still valid for the present problem.
But since the Hamiltonian is now to be maximized with respect to u 1 and
u 2 subject to the two inequality constraints, we need to invoke the Kuhn­
Tucker conditions. Besides, for these conditions to be necessary, a con­
straint qualification must be satisfied. According to a theorem of Arrow,
Hurwicz, and Uzawa, any of the following conditions will satisfy the con­
straint qualification1:
All the constraint functions gi are concave in the control variables uJ
[here, concave in (u1, u 2 )].
(2) All the constraint functions gi are linear in the control variables u J
[here, linear in (u 1 , u 2)]-a special case of (1).
(3) All the constraint functions gi are convex in the control variables u 1.
And, in addition, there exists a point in the control region u 0 E U
[here, u 0 is a point (u 10, u 20)] such that, when evaluated at u0, all
constraints gi are strictly < c ; . (That is, the constraint set has a
nonempty interior.)
(1)
{4) The gi functions satisfy the rank condition: Taking only those con­
straints that turn out to be effective or binding (satisfied as strict
equalities), form the matrix of partial derivative [agijau ). (where e
indicates "effective constraints only"), and evaluate the partial deriva­
tives at the optimal values of the y and u variables. The rank
condition is that the rank of this matrix be equal to the number of
effective constraints.
.... .
1K. J. Arrow, L. Hurwicz, and H. Uzawa, " Constraint Qualifications in Nonlinear Program­
ming," Naval Research Logistics Quarterly, January 1961. The summary version here is
adapted from Akira Takayama, Mathematical Economics, 2d ed., Cambridge University Press,
Cambridge, 1985, p. 648. Note that our constraints are written as g :s; c (rather than g ;;,; c as
in Takayama).
;;;
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
279
We now augment the Hamiltonian into a Lagrangian function:
J= F( t, y, u 1 , u 2 )
( 10.9)
+ A(t) f( t, y , u 1 , u 2 )
+ 0 1( t) [ c1 - g 1 (t, y, u 1 , u 2 )]
+
02 (t) [ c2 - g 2 ( t , y, u 1 , u 2 ) ]
The essence of _/ may become more transparent if we suppress all the
arguments and simply write
( 10.9')
The first-order condition for maximizing _/ calls for, assuming interior
solutions,
( 10.10)
as well
as
( 10.11)
iJJ
j)_/
.
- = C·' - g ' � 0
iJO;
( i = 1 , 2 and j
=
=0
0' iJO;
1 , 2)
for all t
E
[ 0, T ]
Condition (10.1 1) differs from (10.5) because the constraints in the present
problem are inequalities. The iJ.LjiJO; � 0 condition merely restates the i th
constraint, and the complementary-slackness condition O;(iJd'jiJO) = 0 en­
sures that those terms in (10.9) involving 0; will vanish in the solutio·n, so
that the value of _/ will be identical with that of H = F + A f after
maximization.
Note that, unlike in nonlinear programming, we have in (10. 10) the
first-order conditions iJJjiJ u i = 0, not iJd'jiJ u i � 0. This is because the u i
variables are not restricted to nonnegative values in problem (10.8). If the
latter problem contains additional nonnegativity restrictions
ui ( t) � O
then, by tht;, . �n-Tucker conditions, we should replace the /Jd'fiu i
conditions b{ ('Ib.10)
with
.
=0
'
( 10.12)
It should be pointed out that the symbol .L' in (10.12) denotes the same
Lagrangian as defined in (10.9), without separate O(t) type of multiplier
terms appended on account of the additional constraints u 1(t) � 0. This
procedure is directly comparable to that used in nonlinear programming.
The alternative approach of adding a new multiplier for each nonnegativity
restriction u i(t) � 0 is explored in a problem in Exercise 10.1.
280
PART 3, OPTIMAL CONTROL THEORY
Other maximum-principle conditions include the equations of motion
for y and A. These are the same as in (10.6) and (10. 7):
ad'
y= ­
aA
( 1 0 . 13)
ad'
.
A = -­
ay
and
Wherever appropriate, of course, transversality conditions must be added,
too.
Isoperimetric Problem
When an equality integral constraint is present, the control problem is
known as an isoperimetric problem. Two features of such a problem are
worth noting. First, the costate variable associated With the integral con­
straint is, as in the calculus of variations, constant over time. Second,
although the constraint is in the nature of a strict equality, the integral
aspect of it obviates the need to restrict the number of constraints relative
to the number of control variables. We shall illustrate the solution method
with a problem that contains orie state variable, one control variable, and
one integral constraint:
( 10.14)
Maximize
tF( t , y , u ) dt
subject to
y = f(t , y , u )
0
{G(t, y, u ) dt = k
0
and
y(O)
y(T) free
= Yo
( k given)
(y0, T given)
The approach to be used here is to introduce a new state variable r(t)
into the problem such that the integral constraint can be replaced by a
condition in terms of f(t). To this end, let us define
f(t)
( 10 . 15)
=
-
{G(t, y, u ) dt
0
where, the reader will note, the upper limit of integration is the variable t,
not the terminal time T. The derivative of this variable is
( 10 . 16)
t=
[equation of motion for r 1
- G(t,y, u )
and the initial and terminal values of
( 10 . 17 )
f(O)
=
-
r(t) in the planning period are
ta(t,y, u ) dt = 0
0
28 1
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
and
( 10 . 18)
f( T ) = - jTG(t , y , u ) dt = - k
0
[by ( 10 . 14) ]
From (10. 18), it is clear that we can replace the given integral constraint by
a terminal condition on the r variable.
By incorporating r into the problem as a new state variable, we can
restate (10. 14) as
Maximize
jTF(t, y, u ) dt
subject to
y
(10.19)
and
0
= f(t, y, u )
f = - G(t, y, u )
y(O) = Yo y( T ) free
f( T ) - k
f(O) = 0
=
(y0, T given)
( k given)
This new problem is an unconstrained problem with two state variables, y
and r. While the y variable has a vertical terminal line, the new r variable
has a fixed terminal point. Inasmuch as this problem is now an uncon­
strained problem, we can work with the Hamiltonian without first expand­
ing it into a Lagrangian function.
Note that this procedure of substituting out the constraint can be
repeatedly applied to additional integral constraints, with each application
resulting in a new state variable for the problem. This is why there is no
need to limit the number of integral constraints.
Defining the Hamiltonian as
( 10 .20)
H = F( t , y , u ) + >.. f(t,y, u ) - p. G(t, y , u )
we have the following conditions from the maximum principle:
Max H
u
aH
a>..
aH
A = -ay
aH
f=
aiL
aH
it = - -
y= -
( 10.21)
ar
>.. ( T ) = 0
for all
t E [ O, T ]
'
[equation of motion for y ]
[equation of motion for >.. ]
[equation of motion for r ]
[equation of motion for IL ]
[ transversality condition]
What distinguishes (10.21) from the conditions for the usual unconstrained
PART 3: OPTIMAL CONTROL THEORY
282
problem is the presence of the pair of equations of motion for r and J.L .
Since the r variable i s an artifact whose mission i s only to guide u s to add
the -J.LG(t, y, u) term to the Hamiltonian-a mission that has already been
accomplished-and whose time path is of no direct interest, we can safely
omit its equation of motion from (10.21) at no loss. On the other hand, the
equation of motion for J.L does impart a significant piece of information.
Since the r variable does not appear in the Hamiltonian, it follows that
aH
-
Jl = - ar- = 0
( 10.22)
J.L(t) = constant
This validates our earlier assertion that the costate variable associated with
the integral constraint is constant over time. But as long as we remember
that the J.L multiplier is a constant, we may omit its equation of motion from
(10.21) as well.
Inequality Integral Constraint
Finally, we consider the case where the integral constraint enters
problem as an inequality, say,
1TF(t,y, u) dt
y = f(t,y, u)
Maximize
.( 10.23 )
tile
0
subject to
jTG(t,y, u) dt :::;; k
0
y(O) = Yo
and
y(T)
free
(k given)
(y0, T given)
Taking a cue from the isoperimetric problem, we can again dispose of the
inequality integral constraint by making a substitution.
Define a new state variable r the same as in (10.15):
r(t)
=
- J'G(t,y, u) dt
0
where the upper limit of integration is
simply
( 10 .24)
t
= -G(t, y, u)
t (not T). The derivative of
[ equation of motion for r ]
and its initial and terminal values are
( 10.25) .
f(O) = - ta(t,y, u ) dt
0
f(T)
=
-
=
(a(t,y, u) dt �
0
0
-k
(by ( 10 .23)]
r
is
CHAPTER
10:
283
OPTIMAL CONTROL WITH CONSTRAINTS
Using (10.24) and
(10.25), we can restate problem (10.23) as
Maximize
fTF( t, y, u ) dt
subject to
j = f( t, y , u )
0
t
= - G(t, y, u )
y(O) = Yo y( T ) free
f(T) � -k
f(O) = 0
( 10.26)
and
(y0, T given)
(k
given)
Like the problem in (10.19), this is an unconstrained problem with two state
variables. But, unlike (10.19), the new variable r in (10.26) has a truncated
vertical terminal line.
The Hamiltonian of problem (10.26) is simply
H=
( 10.27)
F(t,y, u) + A [(t,y, u ) - J.L G(t,y, u )
If the constraint qualification is satisfied, then the maximum principle
requires that
:. ' }'•
Max H
u
aH
.
aH
[ equation o f motion for A]
ay
aH
f=­
OJ.L
aH
= - ar
>.. ( T ) = 0
J.L ( T ) � 0
iL
. , ··
t E [0, T ]
[ equation of motion for y]
y=o>..
A=
for all
..
[ equation of motion for r]
[ equation of motion for J.L]
[ transversality condition for y]
f(T ) + k � 0 J.L( T ) [ f( T ) + k ] = 0
[transversality condition for r}
Note, again, that because the Hamiltonian i s independent o f f , we have
aH
P- = - - = 0
ar
J.L(t)
= constant
It becomes clear, therefore, that the multiplier associated with any integral
constraint, whether equality or inequality, is constant over time. In the
present problem, moreover, we can deduce from the transversality condition
284
PART 3, OPTIMAL CONTROL THEORY
that the constant value of JL is nonnegative. But, as in
fact omit from
(10.28)
the equations of motion for
r
( 10.21),
and
JL,
we can in
as long as we
bear in mind that JL is a nonnegative constant. In contrast, the transversal­
ity condition for
r -involving
complementary slackness-should be re­
tained to reflect the inequality nature of the constraint. In sum, the
conditions in
(10.28)
can be restated without reference to f as follows:
Max H
for all
u
y=
ii
.
E
[0, T ]
aH
aA
A = -
(10.28' )
t
,_ j
aH
ay
­
)
A( T = 0
:
·-.. .
T
k - J0 G(t,y, u) dt � O
JL [ k - �TG(t,y, u ) dt ] 0 [ ( 10 25 ) ]
JL = constant � O
by
=
and
.
In the preceding discussion, the four types of constraints have been
explained separately. But in case they appear simultaneously in a problem,
we can still accommodate them by combining the procedures appropriate to
each type of constraint present. For every constraint in the form of
g(t,y, u)
=
0
or
g(t,y, u)
�
c,
we append a new 8-multiplier term to the
Hamiltonian, which is thereby expanded into a Lagrangian. And for every
integral constraint, whether equality or inequality, we introduce a new
(suppressible) state variable r, whose costate variable JL -a constant-is
reflected in a
-
JLG(t, y, u)
term in the Hamiltonian. If a constraint is an
inequality, moreover, there will be a complementary-slackness condition on
the multiplier 8 or JL, as the case may be.
EXAMPLE 1
The political-business-cycle model
(7.61):
Maximize
( 10. 29)
subject to
p = <f>( U }
+ a7T
ir = b(p - 7T)
and
7T(O)
=
7To
7T( T )
contains an equality constraint p = <f>(U)
free
+ a7T.
( 7T0, T
given)
But we solved the problem
as an unconstrained problem by first substituting out the constraint equa­
tion. Now we show how to deal with it directly as a constrained problem.
CHAPTER
10: OPTIMAL CONTROL WITH CONSTRAINTS
285
In this problem, 7T is the state variable; it has an equation of motion.
Earlier, since p was substituted out, U was the only control variable. Now
that p is retained in the model, however, it ought to be taken as another
control variable. Thus the constraint equation
p
-
4>( U ) - a7T
=
0
is in line with the general format of g(t, y, u 1, u 2 )
there is no explicit t argument in it.
By (10.3), we can write the Lagrangian
( 10.30)
= c
in (10.1), although
./ = v( U, p ) e '1 + Ab( p - 7T) + O [ cf>( U ) + a 7T - p]
If the following specific functions are adopted:
v( U, p) = - U 2 - hp
cf>( U ) = j - kU
(h
> 0)
[from (7 .62)]
( j , k > O)
[from ( 7 .63)]
then the Lagrangian becomes
( 10.30' )
..;/' = ( - U 2 - hp )er1 + Ab(p - 7T) + O[j - kU + a 7T - p ]
Accordingly, the maximum principle calls for the conditions
a./
- = - 2 Ue'1 - Ok
au
a./
- = - he'1 >.b ap
( 10.31)
+
( 10.32)
=
8
0
=
a./
- = j - k U + a7T - p
ao
a./
7T = - = b( p - 7T)
a>.
a./
A = - - = >.b - Oa
a7T
( 10 .33)
( 10.34)
( 10.35)
[by ( 10. 4 ) ]
[ by ( 10.4) ]
0
=
0
[by ( 10.5)]
[by ( 10.6)]
[by ( 10 .7)]
plus a transversality condition. These should, of course, be equivalent to. the
conditions derived earlier in Sec. 7.6 by a different approach.
To verify the equivalence of the two approaches, let us reproduce here
for coml;larison the major conditions from Sec. 7.6:
( 10.36)
aH
au
= ( - 2U
+ hk )ert - >.bk = 0
[maximizing the Hamiltonian]
( 10.37)
7T = b[j - k U - (1 - a )7T]
[state equation of motion]
( 10.38)
A = haert + A b(1 - a )
[costate equation of motion]
PART 3o OPTIMAL CONTROL THEORY
286
We shall first show that (10.31) and (10.32)-conditions on the two control
variables U and p-together ate equivalent to (10.36). Solving (10.32) for 8,
we find
fJ
( 10.39 )
= - he rt + >.. b -
Substituting this into (10.31) results in
ad'
- = - 2 Ue rt
au
+
hkert - >.. bk = 0
which is identical with (10.36). Next, solving (10.33) for
p = j - kU + a rr
p,
we get
which, of course, merely restates the constraint. This enables u s to rewrite
(10.34) as
7T
=
b(j - k U + arr - rr) = b[ j - kU - ( 1 - a ) rr]
which is the same as (10.37). Finally, using the () expression in (10.39), we
may rewrite (10.35) as
A = >.. b + haert - Aba = Ab(1 - a ) + haert
which is identical with (10.38). Hence, the two approaches are equivalent.
EXAMPLE 2
In Sec. 7.4, Example 3, we encountered a time-optimal
problem
'(ltJ.40)
Maximize
t - 1 dt
subject to
y =y + u
o�· �- 1
0
y( T )
y( O) = 5
u(t) E ( - 1, 1 ]
and
=
11
T free
with a constrained control variable. We solved it by resorting to the signum
function (7.48) in choosing the optimal control. But it is also possible to
view the control set as made up of two inequality constraints
( 10.41 )
-1
:o;
u (t)
and
u ( t)
:$
1
and solve the problem accordingly.
Note that the constraint qualification is satisfied here since each
constraint function in (10.41) is linear in u .
First, w e augment the Hamiltonian o f this problem,
H = - 1 + A(y
+ u)
CHAPTER lOo OPTIMAL CONTROL WITH CONSTRAINTS
287
into a Lagrangian by taking into account the two constraints in (10.41). The
result is
J=
( 10 .42)
- 1 + A(y + u ) + 0 1( u + 1)
+
02 ( 1 - u )
[by ( 10.9)]
To satisfy the maximum principle, we must have:
for all t
( 10.43)
E
[ 0, T ]
( 10.44)
( 10.45)
( 10 .46)
( 10 .47)
{ · ··
a..I"
- = 1 - u >- O
ao 2
a..I"
:Y = - = y + u
aA
.
A
=
a..l"
- -
ay
=
-
02( 1 - u ) = O
A
plus a transversality condition. Even though the two equations of motion
(10.46) and (10.47) here are derived from ..1" instead of H, they are the
same as those derived from H in Sec. 7.4. So nothing more needs to be said
here about these. What needs to be verified is that conditions (10.43)
through (10.45) would lead us to the same choice of control as the signum
function:
( 10.48)
u* = 1 if A > 0
u* = - 1 if A < 0
We shall demonstrate here that A > 0 indeed implies u* = 1. Let A > 0.
Then, to satisfy (10.43), we must have 01 - 0 2 < 0 or 0 1 < 0 2 . Since both 0 1
and 0 2 are nonnegative, the condition 0 1 < 0 2 implies that 02 > 0. Thus, by
the complementary-slackness condition in (10.45), it follows that 1 - u = 0,
so the optimal control is u* = 1. This completes the intended demonstra­
tion. The other case (A < 0) is similar, and will be left to the reader as an
exercise.
Current-Value Hamiltonian and
Lagrangian
When the constrained problem involves a discount factor, it is possible to
use the current-value Hamiltonian He in lieu of H. In that case, the
Lagrangian 1 should be replaced by the current-value Lagrangian �-
288
PART 3, OPTIMAL CONTROL THEORY
Consider the inequality-constraint problem
(10�'l9 )
'�
(
Maximize
J T<t>(t, y, u ) e-P1 dt
subject to -
y = f( t, y, u )
0
g(t , y, u ) � c
boundary conditions
•. .
and
The regular Hamiltonian and Lagrangian are
( 10.50)
H = <P(t, y , u ) e -pt + A f( t , y, u )
.L' = <P(t, y, u )e-pt + A f( t , y , u ) + 9 ( c - g ( t, y , u ) ]
And the maximum principle calls for (assuming interior solution)2:
( 10.51)
( 10 .52)
( 10.53)
( 10 .54)
ad'
for all t E [0, T ]
au
ad'
9�0
- = c - g(t ' y u ) > 0
a9
ad'
:Y = ­
aA
ad'
.
A = -­
ay
-=0
'
ad'
9- = 0
a9
plus an appropriate transversality condition.
By introducing new multipliers
( implying A = me-P1)
( implying 9 ne-P1)
( 10.55)
=
we can introduce the current-value versions of H and
( 10.56)
2
He = HeP1
=
<P(t, y , u) + mf( t , y , u )
..zf
as
follows:
..£; = .L'eP1 = <P(t, y, u ) + mf( t, y, u ) + n ( c - g( t , y, u ) ]
If the control variable is constrained by a nonnegativity requirement,
to
u�O
ad'
u- = 0
au
[cf.
(10.51) must be changed
( 10.12)]
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
289
Tt can readily be verified that
a..£;,
au
--
=
a..z!'
Pt
e
au
-
an
and
ao
Therefore, conditions (10.51), (10.52), and (10.53) can be equivalently ex­
pressed with ..£;, and the hew multipliers m and n as follows:
a..£;,
0
au
a..£;,
>0
an a..£;,
j=am
( 10 .57)
-
( 10.58)
( 10.59)
=
for all t E
[0, T ]
n ;::: O
n
a..£;,
=0
an
-
The only major modification required when we use ..£;, i s found in the
equation of motion for the costate variable, (10.54). The equivalent new
statement, using the new variable m , is
m.
( 10.60)
=
-
a..£;,
ay
- + pm
To verify this, first differentiate the A expression in (10.55) with respect to
t, to obtain
Then differentiate
..z!'
with respect to y to get
iJ..z!'
- -
iJy
=
-
<1>y e
- pt
-
A {y + (Jgy
Equating these two expressions in line with (10.54), and multiplying through
by e P 1, we find that
=
- <I>Y - mfy + ngY
Since the right-hand-side expression is equal to - il.L';,jily [from (10.56)], the
rit -
pm
result in ( 10.60) immediately follows.
For problems with integral constraints only, no Lagrangian function is
needed, or, to put it differently, the Lagrangian-r-educes to the Hamiltonian.
This is because we absorb each integral constraint into the problem via a
new state variable-such as f in (10.19) and (10.26). The maximum-princi­
ple conditions can, accordingly, be stated in terms of the Hamiltonian. If we
decide to use He in place of H, the procedure outlined in Sec. 8.2 can be
applied in a straightforward manner. The major modification is, again, to be
found in the costate equation of motion, namely, replacing the condition
290
PART a, OPTIMAL CONTROL THEORY
rit
-aHcfiiy + p m . With a newly introduced state variable f, the problem will have a new costate variable J.L. Since
its equation of motion is jL -iJHjiJf 0, J.L is a constant.
A = -iJHjoy by the condition
=
=
r,
=
Sufficient Conditions
The Mangasarian and Arrow sufficient conditions, previously discussed in
the context of unconstrained problems, turn out to be valid also for con­
strained problems when the terminal time T is fixed. This would include
cases with a fixed terminal point, a vertical terminal line, or a truncated
vertical terminal line.
Let us use the symbol u to represent the vector of control variables.
For problems (10.1) and (10.8), for example, we have u = ( u 1, u 2). As
before, let H 0 denote the maximized Hamiltonian-the Hamiltonian evalu­
ated along the u*(t) path. But, in the present context, the. Hamiltonian is
understood to be maximized subject to all the constraints of the g(t,y, u) = c
form or the g(t, y, u) � c form present in the problem. Besides, since every
integral constraint present in the problem is reflected in H via the new
costate variable J.L, it must also be similarly reflected in H 0 •
For simplicity, we can consolidate the Mangasarian and Arrow suffi­
cient conditions into a single statement3: The maximum-principle condi­
tions are sufficient for the global maximization of the objective functional if
{ 10.61)
either _£' is concave in (y, u ) for all t E [0, T ] ;
or H 0 is concave in y for all t E [0, T ] , for given A
These conditions are also applicable to infinite-horizon problems, but in the
latter case, (10.61) is to be supplemented by a transversality condition
{ 10.62)
lim A { t) [y(t) - y* ( t)J
t-oo
�
0
[cf. (9.21)]
A few comments about (10.61) may be added here. First, as pointed out
before, the concavity of d' in (y, u) means concavity in the variables y and
u jointly, not separately in y and in u. Second, since H and ../ are
·�.I.�-� ._
For a more comprehensive statement of sufficient conditions, see Ngo Van Long and Neil
Vousden, "Optimal Control Theorems," Essay 1 in John D. Pitchford and Stephen J.
Turnovsky, eds., Applications of Control Theory to Economic Analysis, North-Holland. Ams­
terdam, 1977, pp. 1 1-34 (especially pp. 25-28, Theorems 6 and 7).
3
·:.:· ·
�
'
,i:l
i
291
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
composed of the
H
=
F,
{, g, and G functions as follows:
F+
A ( - J.LG
and
./ =
H
+ 0[ c
-
g]
it is clear that (10.61) will b e satisfied i f the following are simultaneously
true:
F is concave in (y, u )
A f is concave in (y, u )
J,LG is convex in (y, u )
and O g is convex in (y, u )
for all
tE
[0, T ]
In the case of an inequality integral constraint, however, where J.L is a
nonnegative constant [by ( 10.28)], the convexity of J,LG is ensured by the
convexity of G itself. Similarly, in the case of an inequality constraint,
where 0 2:: 0 [by (10. 1 1)], the convexity of Og is ensured by the convexity of
g itself. Finally, if the current-value Hamiltonian and Lagrangian are used,
0
( 10.61) can be easily adapted by replacing ./ by _£,:, and H 0 by Hc .
EXERCISE 10.1
1
In Example 1, p is taken to be an additional control variable. Why is it not
taken to be a new state variable?
In Example 2, it is demonstrated that if A > 0, then the optimal control is
*
1. By analogous reasoning, demonstrate that, if A < 0, then the
optimal control is u *
- 1.
3 An individual's productive hours per day (24 hours less sleeping and leisure
time-normalized to one) can be spent on either work or study. Work
results in immediate earnings; study contributes to human capital K(t)
(knowledge) and improves future earnings. Let the proportion of productive
hours spent on study be denoted by s. The rate of change of human capital,
K, is assumed to have a constant elasticity a (0 < a < 1) with respect to
sK. Current earnings are determined by the level of human capital
multiplied by the time spent on work.
(a ) Formulate the individual's problem of work-study decision for a
planning period [0, T ] with a discount rate p (0 < p < 1).
(b) Name the state and control variables.
(c) What boundary conditions are appropriate for the state variable?
(d) Is this problem a constrained control problem? Which type of
constraints does it have?
(e) How does this problem resemble the Dorfman model of Sec. 8.1? How
does it differ from the Dorfman model?
2
u
=
=
'. !
PART 3, OPTIMAL CONTROL THEORY
292
4
Consider the problem
Maximize
subject to
jrF(t , y , u ) dt
0
j = f( t , y , u )
g(t,y, u ) � c
y ( O) = Yo
and
l'
i. ... ·-·
0
�
y( T ) free
(y0 , T given)
u(t)
We can, as we did in (10. 12), apply the Kuhn-Tucker conditions to the
nonnegativity restriction on u(t) without introducing a specific multiplier.
Alternatively, we may treat the nonnegativity restriction as an additional
constraint of the g(t, y, u ) � c type. Write out the maximum-principle
conditions under both approaches, and compare the results. [Use 6' do
denote the new multiplier for the 0 � u(t) constraint and -t!'' to denote
the corresponding new Lagrangian.]
6 In a maximization problem let there be two state variables (y1, y2), two
control variables ( u 1 , u 2), one inequality constraint, and one inequality
integral constraint. The initial states are fixed, but the ternrinal states are
free at a fixed T.
(a) Write the problem statement.
(b) Define the Hamiltonian and the Lagrangian.
(c) List the maximum-principle conditions, assuming interior solutions.
'
10.2
THE DYNAMICS OF A REVENUE·
MAXIMIZING FIRM
In the Evans model of the dynamic monopolist (Sec. 2.4), the stated
objective is to maximize the total profit. This, of course, is nothing extraor­
dinary because profit maximization has long been the accepted hypothesis
among economists. A well-known model by Baumol takes the view, however,
that instead of maximizing profits, modern firms may actually try to
maximize sales revenue, subject to a minimum requirement on the rate of
return.4 The primary basis for this view is the separation of ownership and
management in the typical corporate form of the firm, where the managers
may, in their own interest, seek to maximize sales, despite the stockholders'
�
William J. Baumol, "On the Theory of Oligopoly,"
See also his
Business Behavior, Value and Growth,
Econometrica , August 1958,
pp. 187-198.
revised edition, Harcourt, Brace
&
World,
New York, 1967. This model is discussed as an example of nonlinear programming in Alpha
Chiang,
Fundamental Methods of Mathematical Economics,
1984, Sec. 21.6.
C.
3d ed., McGraw-Hill, New York,
CHAPTER
1 0 : OPTIMAL CONTROL WITH CONSTRAINTS
293
objective of maximizing profit instead. To placate the stockholders, however,
the managers must attempt to attain at least a minimum acceptable rate of
return.
Since the Baumol model is static, its validity has been questioned in
the dynamic context. In particular, since profits serve as the vehicle of
growth, and growth makes possible greater sales, it would seem that in the
dynamic context profit maximization may be a prerequisite for sales maxi­
mization. To address the issue of whether the Baumol proposition retains
its validity in a dynamic world, Hayne Leland has developed an optimal
control model.5
The Model
Consider a firm that produces a single good with a neoclassical production
function Q = Q(K, L), which is linearly homogeneous and strictly quasicon­
cave. The usual properties listed below are supposed to hold:
( 10 .63)
Q
- = 4> ( k )
L
4>(0) = 0
QK = cf>' ( k )
QK > O
QL = lf>( k ) - klf>' ( k )
All prices-including the wage rate W and the price of capital
goods-are assumed to be constant, with the price of the firm's product
normalized to one. Thus the revenue and (gross) profits of the firm are,
respectively,
R = Q( K, L )
1r
=R
-
·
1 = Q( K, L )
WL = Q ( K , L ) - WL
To satisfy the stockholders, the managers must keep in mind a minimum
acceptable rate of return on capital, r0 • This means that the managerial
behavior is constrained by the inequality
( 10.64)
or
Q(K, L ) - WL - r0 K ?:! 0
It remains to specify the decision on investment and capital accumula­
tion. For simplicity, it is postulated that the firm always reinvests a fixed
proportion of its profits 1r. Define a as the fraction of profits reinvested,
5
Hayne E. Leland, "The Dynamics of a Revenue Maximizing Firm,"
Review, June 1972, pp. 376-385.
International Economic
PART 3, OPTIMAL CONTIIOL �
294
divided by the constant price of capital goods. Then we have
K( = l ) = a'7T = a[Q( K, L ) - WL]
K(O) = K 0•
' ( 10.65)
. with initial capital
Drawing together the variou� considerations, we � state the problem
of this revenue-maximizing firm as one with an inequs,ijty 1:9nstraint:
·'
·.... .. .
· ·'
Maximize
( 10.66)
subject to
and
K = a[Q( K, L ) - WL ]
WL + r0 K - Q ( K, L) s; 0
K(O) K0
K( T ) free
=
( K0 , T given)
There are only two variables in this model, K and L. The presence of an
equation of motion for K identifies K as a state variable; this leaves L as
the sole control variable.
.
Note that the constraint function WL + r0K - Q(K, L) is convex in
the control variable L, and the constraint set does contain a point such that
the strict inequality holds (the rate of return exceeds the r0 level). Thus the
constraint qualification is satisfied.
·---� !
The Maximum Principle
The current-value Hamiltonian of this problem is
Hc = Q( K , L ) + m a [ Q(K, L ) - WL} ·
By augmenting the Hamiltonian with the information in the inequality
constraint, we can write the current-value Lagrangian [see (10.56)] as
..£;;
= Q(K, L)
+
ma [ Q( K, L) - WL ] + n [Q( K, L) - WL - r0K ]
From (10.57) and (10.58), we then have the following first-order conditions
for maxim�ing ..£;; :
( 10.67)
( 10.68)
il..£;;
iJL = ( 1 + ma + n ) QL - ( m a
+
n)W = 0
for all t E
[O, T )
il..£;;
- = Q( K , L) - WL - r0K ;;>: O n ;;>: O
iJn
and n [ Q(K, L ) - WL - r0K ] =
'·
0
To determine·ttte dynamics of the system, we rely on the two equations of
motion
.
K = iJ..£;;
iJ m = a(Q( K, L) - WL ]
CHAPTD
10: OPTIMAL CON'I'BOL WITH CONSTRAINTS
295
and
a.£;,
.
m = - - + p m = - ( 1 + m a + n ) QK -t nro f pm
K
a
[by ( 1 0 .60 )]
Finally, the transversality condition
m(T)
should be imposed since K ( T ) is free.
=
0
Qualitative Analysis
Rather than employ a specific production function to obtain a quantitative
solution, Leland carries out the analysis of the model qualitatively. Also,
instead of working with the original input variables K and L, he uses the
properties of the production function in ( 10.63) to convert the maximum­
principle conditions into terms of the capital-labor ratio, k. Accordingly,
condition (10.67) is restated as
( 10.69)
a.£;,
= ( 1 + m a + n ) [ �j� ( k ) - k ljl'( k ) ] - ( m a + n ) W = 0
aL
By dividing the first inequality in (10.68) by L
write a new version of the condition:
( 10 .70)
1/l ( k ) - W - r0k � 0
*
0 and using ( 10.63), we can
n [ 1/1( k ) - W - r0k ] = 0
n�O
And the two equations of motion become
' '
( 10.71)
K = a L [ I/I ( k ) - W ]
( 10 .72)
m = - ( 1 + rria + n )ljl'( k ) + nr0 + pm
The k variable now occupies the center stage except in (10.71), where the
original variables K imd L still appear.
Certain specific levels of the capital-labor ratio k have special roles to
play in the model. Two of these are:
k
k
0
=
the profit-maximizing level of k
=
the k that yields the rate of return r0
[ k satisfies QL = W or ljl(k)
[ k 0 satisfies ;
= r0
- k ljl' ( k ) = W-hy ( 10 .63) }
or 1/1( k) - W = r0k-by ( 10 .64)
)
In Fig. 10.1, we plot the APPL (QjL ) and the MPPL (QL) curves against k.
The APPL curve has been encountered previously in Figs. 9. 1 and 9.3a; the
MPPL curve should lie below the APPL curve by the vertical distance
296
PART 3: OPTIMAL CONTROL THEORY
!{
=
L
,P(k )
w
. '
'
�-,
'.-'
FIGURE 10.1
k cp'(k). The intersection of the QL curve and the W line detennines the ."
magnitude of k. While the position of k 0 can only be arbitrarily indicated, it
is reasonable to expect k 0 to be located to the left of k.
Since it is not permissible to choose a k < k0, Leland concentrates on
the situation of k > k0, the case where the rate of return exceeds the
minimum acceptable level r • The complementary-slackness condition in
0
(10.70) then leads to n = 0. This will simplify (10.69) to
( 10 .73)
a.£;,
a
J
L n-O
= (1
+ ma) [cp( k ) - k cp'( k )] - m aW = 0
This equation can be plotted as a curve in the km space i f we can find an
expression for its slope so that we can ascertain its general configuration.
Such a slope expression is available by invoking the implicit-function rule.
Call the expression in (10.73) between the two equals signs F( m k).
Then the slope of the curve is
,
( 10 .74)
(
)
dm
a.£;,
aF;ak
for
0
BL = = aF;am
dk
=
( 1 + m a ) k <P"( k )
a ( cp ( k ) - k cp' ( k ) - W ]
The following observations may be gathered from this result:
(1) The numerator is negative, because cp"(k) < 0, as shown by the curva­
ture of the APPL curve in Fig. 10. 1.
(2) For any k < k, the denominator is also negative. This can also be seen
in Fig. 10. 1 since the QL curve lies below the W line to the left of k.
Thus, for any k E (k0, k), the a.L;,;aL = 0 curve is positively sloped.
(3) As k approaches k from the left, the denominator tends to zero, so
dmjdk -+ oo.
(4) As k approaches zero, the numerator tends to zero, so dmjdk -+ 0.
CHAPTER
m
1 0:
297
OPTIMAL CONTROL WITH CONSTRAINTS
m. = 0
1 /.
m
I
I
I
I
I
I
I
I
>
0
. (·:
'
a;ec = 0
aL �
... �:·
I
1\
ks
FIGURE 10.2
k
These observations enable us to sketch the iJ..zf',jiJL = 0 curve the way it is
shown in Fig. 10.2.
By concentrating on k > k 0, which implies n = 0, we also get a
simpler version of m from (10. 72). If we set that m equal to zero, then we
have the equation
( 10.75)
m ln- 0 = - ( 1 + m a ) 4>' ( k ) + p m = 0
which can also be graphed in the km space. To use the implicit-function
rule again, let us call the expression in (10. 75) between the two equals signs
G(m, k). Then the slope of the m 0 curve is
=
( 10 .76)
dm
- ( for m = 0)
dk
=
iJGjiJk
( 1 + m a ) 4>" ( k )
- --- = -----iJGjiJm
- a4>' ( k ) + p
While the numerator is negative, the denominator can take either sign. Let
k + = the k value that satisfies p = a 4>' ( k )
Then we see, first, that since 4>'(k ) decreases with k (see Fig. 10.1), for any
> k + the denominator in (10.76) is positive, so that dmjdk is negative
and the m = 0 curve is downward sloping. Second, as k -> k + from the
right, the slope of the m = 0 curve tends to infinity. These explain the
general shape of the m = 0 curV-e in Fig. 10.2. The two curves intersect at
k k '. Thus,
k
=
k ' = the k level that satisfies iJ..zf,jiJL = m = 0, when
n =
0
Such an intersection exists if and only if k + occurs to the left of k.
The m = 0 curve is where the multiplier m is stationary. What
happens to m at points off the m = 0 curve? The answer is provided by the
derivative
a m.
ak = - ( 1 + m a ) 4>" ( k )
>
0
298
PART 3: OPTIMAL CONTROL THEORY
The message here is that as k increases, m will increase, too . Consequently,
we have m < 0 to the left of the m = 0 curve, and m > 0 to its right. This
is why we have made the arrowheads on the a.L;,;aL = 0 curve point
southward to the left, and northward to the right, of the m 0 curve.
From Fig. 10.2, it should be clear that the stationary point formed by
the intersection of the two curves-may not occur at k , the profit-maximizing
level of k . That is, placing the revenue-maximizing firm in the dynamic
context does not force it to abandon its revenue orientation in favor of the
profit orientation. In fact, Leland shows by further analysis that there is an
eventual tendency for the revenue-maximizing firm to gravitate toward k 0,
where the rate of return is at the minimum acceptable level r0• The only
circumstance under which the firm would settle at the profit-maximizing
level k is when k + exceeds k .
The reader may have noticed that condition (10.7 1 ) has played no role
in the preceding analysis. This is because the analysis has been couched in
terms of the capital-labor ratio k, whereas (10.71) involves K and L as well
as k . A simple transformation of (10.71) will reveal what function it can
perform in the model. Since k "" K/L, we have K = kL, so that
=
K = kL + ki
Equating this with the K expression in ( 10.71) and rearranging,
that
t
a [ cf> ( k ) k
L
we
find
j't .
W] - k
Once we have found an optimal time path for k , then this equation can yield
a corresponding optimal path for the control variable L.
10.3
STATE-SPACE CONSTRAINTS
The second category of constraints consists of those in which no control
variables appear. What such constraints do is to place restrictions on the
state space, and demarcate the permissible area of movement for the
variable y. A simple example of this type of constraints relevant to many
economic problems is the nonnegativity restriction
y(t)
�
0
or
-y( t )
�
0
for all t
E
[0, T J
But more generally, the constraint may take the form
h(t,y)
�c
for all t
E
[0, T ]
In either case, the control variable u is absent in the constraint function.
By coincidence, it may happen that when we ignore the state-space
constraint and solve the �ven problem as an unconstrained one, the optimal
path y*(t) lies entirely in the permissible area. In that event, the constraint
CHAPTER 10: OPTIMAL CONTROL W1'l'H C01IIliiTRAll'ITS
299
y
y
y*(t )
0
0 �-------��/��--/' ' ,
Unconstrained
solution
(a)
iT
1
'1
r
I
y•co
'\-�- /
'
' ' - - - _...
1 I
I I
7--..L---/"-. T
.
t
Unconstrained
solution
(b)
FIGURE 10.3
is trivial. With a meaningful constraint, however, we would expect the
unconstrained optimal path to violate the constraint, such as the broken
curve in Fig. 10.3a which fails the nonnegativity restriction. The true
optimal solution, exemplified by the solid curve, can contain one or more
segments lying on the constraint boundary itself. That is, there may be one
or more "constrained intervals" (or " blocked intervals") within [0, T ] when
the constraint is binding (effective), with h (t, y) c. Sometimes the true
optimal path may have some segment(s) in common with the unconstrained
solution path; but the two paths can also be altogether different, such as
illustrated in Fig. 10.3b.
Although we cannot in general expect the unconstrained solution to
work, it is not a bad idea to try it anyway. Should that solution turn out to
satisfy the constraint, then the problem would be solved. Even if not, useful
clues will usually emerge regarding the nature of the true solution.
=
Dealing with State-Space Constraints
Let the problem be
( 10. 77)
Maximize
JTF( t, y, u ) dt
subject to
y = f( t, y, u )
...
0
h(t, y )
_ ,_ _ _ ,_
and
$
c
.
·· s
.
boundary conditions
Instinctively, we may expect the method of Sec. 10. 1 to apply to this .
problem. If so, we have the Lagrangian
( 10 .78)
./
=
F(t, y , u) + A f( t , y , u)
+
8 [ c - h ( t , y)]
300
PART 3, 0PT� CONTROL THEORY
with maximum-principle conditions (assuming interior solution)
aJ
- = F + A {, = O
au
u
u
-
j)_/
-ao = c - h(t, y ) � o
j)_/
y = aA = f(t, y , u)
-,
j)_/
'1·
-:
''
aJ
6- = 0
a6
6�0
. ·· ·.-., .
ay
plus a transversality condition if needed
Why cannot these conditions be used as we did before?
For one thing, in previously discussed constrained problems, the solu­
tion is predicated upon the continuity of the y and A variables, so that only
the control variables are allowed to jump (e.g., bang-bang). But here, with
pure state constraints, the costate variable A can also experience jumps at
the junction points where the constraint h(t, y) $ c turns from inactive
(nonbinding) to active (binding) status, or vice versa. Specifically, if -r is a
junction point between an unconstrained interval and a constrained interval
(or the other way around), and if we denote by A -(-r) and A +(-r) the value of
A just before and just after the jump, respectively, then the jump condition
is6
( 10.80)
( b � 0)
Since the value of b is indeterminate, however, this condition can only help
in ascertaining the direction of the jump. Note that it is possible for b to be
zero, which means that A may not be discontinuous at a junction point.
This latter situation can occur when the constraint function h( t, y) has a
sharp point at -r, that is, when h is discontinuous at -r.
At any rate, the solution procedure requires modification. We now
need to watch out for junction points, and make sure not to step into the
forbidden part of the state space.
An Alternative Approach
While it is possible to proceed on the basis of the conditions in (10.79), it
would be easier to take an alternative approach in which the change in the
status of the constraint h(t, y) s c at junction points is taken into account
6
For a more detailed cliscussion of jump conditions, see Atle Seierstad and Knut Sydsreter,
Optimal Control Theory with Economic Applications, Elsevier, New York, 1987, Chap. 5,
especially pp. 317-319.
301
CHAPTER 10, OPTIMAL CONTROL WITH CONSTRAINTS
in a more explicit way.7 The major consideration of this latter approach is
that since h(t, y) is not allowed to exceed c, then whenever h(t, y) = c (the
constraint becomes binding) we must forbid h(t, y) to increase. This can be
accomplished simply by imposing the condition
dh
whenever h ( t, y)
dt Note that the derivative dhjdt (a total derivative), unlike h itself, is a
function of t and y, as well as u , because
d
ah ah dy
.
dt h(t, y ) = dt = h1 + h y f( t,y, u) = h ( t , y, u )
at + ay Therefore, the new constraint dhjdt ::; 0 whenever h(t,y)
or, more
-<0
= c
= c,
explicitly,
( 10.81)
h (t,y, u ) = h 1 + h J( t, y, u ) ::; 0 whenever h (t,y)
fits nicely into the g(t, y, u) ::; category discussed in Sec. 10.1. The only
difference is that (10.81) is not meant for all t E [0, T ], but needs to be put
in force only when h(t, y) =
= c
c
c.
With this new constraint, the problem statement now takes the form
Maximize
J TF(t, y, u ) dt
subject to
:Y = f( t , y, u )
h( t, y, u ) = h 1 + h y {( t, y, u ) ::; 0
whenever h(t, y)
( 10.82)
0
= c
and
boundary conditions
In order to make possible a comparison of the new maximum-principle
conditions with those in (10.79), we shall adopt distinct multiplier symbols
A. and 0 here. Let the Lagrangian be written as
( 10 .83)
..£''
= F(t,y, u ) + A {( t, y , u ) - 0h
Then, as part of the maximum-principle conditions, we require (assuming
that the' constraint qualification is satisfied) that
a.L"' = + - 0h �" = 0
yI
au F A f
u
7See Magnus R. Hestenes, Calculus
York, 1 966, Chap. 8, Theorem 2 . 1 .
u
u
of Variations and Optimal Control Theory, Wiley, New
302
PART 3: OPTIMAL CONTROL THEORY
and
a1''
ae
0- = 0
While the set of conditions on 0 seems to serve the purpose of restating the
constraint, it does not make clear that this set applies only when h(t, y) = c.
To remedy this, we append the complementary-slackness condition
h ( t , y)
�
c
e[c - h (t,y)] = 0
Then, h(t, y) < c (constraint not binding) would mean 0 = 0, which would
cause the last term in _/' to drop out, and thereby nullify the conditions
regarding a1'';ae. Conversely, when h(t, y) = 0 (constraint binding), we
intend the complementary-slackness condition to imply 0 > 0. Thus this is
a stronger form of complementary slackness. [In the normal interpretation
of complementary slackness, h(t, y) = 0 is consistent with 0 = 0 as well as
e > o .J
In addition to the preceding, the maximum principle for the present
approach also places a restriction on the way the 0 multiplier changes over
time: At points where 0 is differentiable, 0 must be nonpositive whenever
h(t, y) c . This restriction will be explained later.
Collecting all the results together, we have (assuming interior solution) the following set of conditions:
=
a1''
au = Fu + A fu - ® h y fu = 0
. .
a1''
.
= - h = - [ h 1 + h y {( t, y, u ) j :2: 0
ae
e[c - h (t,y)] = 0
h ( t, y ) � c
( 10.84)
"' .
\,
0
0
[ = 0 when h ( t, y) <
a1''
y = aA = f( t, y, u )
c]
r�'f· � t�
a1''
A = - ----ay = - FY - A {y + 8 [ h y1 + h y {y + h yy f j
.
: ..,_
�
a1''
ae
8 - == 0
plus a transversality condition if needed
Note that, after deriving the h expression from the constraint function
h(t, y ), we can directly apply the conditions in (10.84) without first trans­
forming problem (10. 77) into the form of ( 10.82).
Here, as in (10.79), we assume that the maximization of the La­
grangian with respect to· u yields an interior solution. If the control variable
is itself subject to a nonnegativity restriction u (t ) � 0, then the a../'jau = 0
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
303
condition should be replaced by the Kuhn-Tucker conditions
a../'
-- < 0
au -
u
2:
a../'
u --
0
=
au
0
These conditions allow for the possibility of a boundary solution. Boundary
solutions can also occur, of course, if there is a closed control region for u .
A distinguishing feature of this approach is that the f(t, y , u) function
enters into the Lagrangian ../' twice, with two different multipliers, A (a
costate variable) and e (a Lagrange multiplier for a constrained problem).
Accordingly ' the partial derivative f" also appears twice in the a..L" I au = 0
condition-the first line in (10.84)-once with A and once with 0. In
contrast, the o../j ou = 0 condition in ( 10.79) does not contain the (} multi­
plier. Thus, under the new approach, the behavior of 0 and its effects on
the system are more explicitly depicted. When the state-space constraint is
nonbinding, with h(t, y ) < c, e takes a zero value, and (10.84) reduces to
the regular maximum-principle conditions. But when the constraint changes
its status from nonbinding (inactive) to binding (active), the conditions in
(10 .84) will become fully operative, prescribing how the 0 multiplier affects
the system, and how 0 itself must change over time.
While the approach summarized in (10.84), with more explicit informa­
tion about junction points, is easier to use than the approach in (10. 79), the
two approaches are in fact equivalent. The reader is asked to demonstrate
their equivalence by following the set of steps outlined in Exercise 10.3,
Prob. 1. In the process, it will become clear why the 0 � 0 condition is
needed.
In another exercise problem, the reader is asked to write out the
maximum-principle conditions for the special case of nonnegativity con­
straint y(t) 2:: 0.
An Example
Let us consider the problem8
'
�:.-
Maximize
f\4 - t ) u � ..
subject to
j= u
( 10.85)
0
y -
t�1
y(O)
and
U E
=
0
( 0, 2 ]
8
This problem is given as an exercise in Atle �
provide and discuss here its complete solution. .
· .< . ,..
8114 :IC.mat�;'t'lp. eit. P- 329. We
304
PART a, OPTIMAL CONTROL THEORY
y
y=t+ 1
3
2
,, ,, 'i
<J,
.
,.
0
2
which contains a state-space constraint,
constraint and form the Hamiltonian
y
-
t ;s;
1. If
we disregard this
H = (4 - t) u + Au
we see that" H
is
linear in
u , with slope
aH
-=4-t+A
au
Given the closed control region [0, 2], we may expect a boundary solution to
arise. Specifically, from the integrand function we gather that in order to
maximize the objective functional, we should let u be as high as possible.
Thus a reasonable conjecture about the unconstrained solution is u(t) = 2.
This would imply that y = u = 2 and y(t) = 2t + k. After using the initial
condition y(O) = 0, we obtain the simple linear path
y(t)
=
2t
This path is the one that allows the fastest rise in y, because it corresponds
to the highest value of u , and hence the highest value of y . As Fig. 10.4
shows, however, this path, steadily rising from the point of origin, can stay
below the constraint boundary y = t + 1 only up to the point (1, 2), that is,
up to t = 1, at which time the constraint becomes binding. Then we have to
change course.
Let us now turn to (10.84) for guidance as to what new course to take.
First, the state-space constraint y t ;s; 1 implies that
-
h(t,y) =" y - t
c=1
h =y - 1 = u - 1
CHAPTER
1 0: OPTIMAL CONTROL WITH CONSTRAINTS
305
Since the constraint function is linear in u (it does not oontain
constraint qualification is satisfied. The Lagrangian is, by �10.13�
u ),
the
../' = ( 4 - t) u + Au - 0( u - 1)
which i s linear i n
u . The maximum principle requires, by (10.84), that
../' be maximized with respect to
a..I''
u
[corner solution]
-- = 1 - u >O
0( 1 - u ) 0
0�0
a0
y - t 5: 1
0(1 - y + t) = 0
0 5: 0
( = 0 when constraint nonbinding)
y=u
=
=
A = constant
No transversality condition is needed because the endpoints are fixed.
Taking the clue from the earlier-discussed unconstrained solution, we
initially adopt for the first time interval [0, 1) the control u = 2, which
implies the state path y = 2t. Thus, we have
u*[O, 1) = 2
( 10.86)
the
y*[0, 1 ) = 2t
As soon as we hit the constraint boundary,
condition tells us to set
a../'ja0
u=1
0
becomes positive, and
[by complementary slackness]
This means that the equation of motion becomes y = 1, and yields the path
To definitize the arbitrary constant kl> we reason from the
continuity of y that this second segment must start from the point (1, 2)
where the first segment ends. This fact enables us to definitize the path to
y(t) = t + k1•
y(t) = t + 1
[same as the constraint boundary]
Thus, as shown in Fig. 10.4, the y path now begins to crawl along the
boundary. But this path obviously cannot take us to the destination point
(3, 3), so we need at least one more new segment in order to complete the
journey. To determine the third segment, we first try to ascertain the
proper control, and then use the equation y = u.
For the first segment, we chose u*
2 to ensure the fastest rise in y,
and adhered to that u value for as long as we could, until the boundary
constraint forced us to change course. The constraint boundary later be­
comes the new best path, and we should again stay on that path for as long
=
306
PART 3, QPT�L CONTROL THEORY
as
feasible, until we are forced by the consideration of the destination
location to veer off toward the point (3, 3). Since u( y) cannot be negative,
it is not feasible to overshoot the y = 3 level and then back down. So the
second segment should aim at y = 3 as its target, terminating at the point
(2, 3). Thus we have
=
u* [ 1 , 2)
( 10.87)
slope
=
1
y* [ 1 , 2)
=
t+ 1
It then follows that the third segment is a horizontal str,right
line with
.
u = 0:
u*[2, 3]
( 10.88)
=
0
y* [2, 3 ]
=
3
By piecing together (10.86), (10. 88), and ( 10.89), we finally obtain the
complete picture of the optimal control and state paths.
In the present example, the first segment of the optimal path coincides
with that of the unconstrained problem. But this is not always the case. In
problems where nonlinearity exists, the true optimal path may have no
shared segment with the unconstrained path. This type of outcome is
illustrated in Fig. 10.3b.9
EXERCISE
1
2
10.3
Establish the equivalence of (10.79) and (10.84) by the following procedure:
0), and solve for A.
(b) Differentiate A totally with respect to t, to get an expression for A.
( c) Expand the expression obtained in part (b) by using the A expression
in (10.79).
(d) Substitute the result from part (a) into the A expression in (10.84).
(e) Equate the two A expressions in parts (c) and (d), then simplify, and
deduce that e � 0.
(a) Equate iJ.L'fiJu and iJ./'jiJu (both =
(a) Write out the maximum-principle conditions under each of the
two
approaches when the state-space constraint is y(t) � 0.
(b) In a problem with y(t) � 0, assume that the terminal time T is fixed.
Is there need for a transversality condition? If so, what kind of ·
transversality condition is appropriate?
I. Kamien and Nancy 'L� �i Dynamic �::n.e
Calculus of Variations and Optimal Control in ECOitOfrliu and Management, lkl ed., Elaeviar,
New York, 1991, pp. 231-234.
,· .
9For an example, see Morton
307
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
10.4
ECONOMIC EXAMPLES OF
STATE-SPACE CONSTRAINTS
In this section, we present two economic examples in which a nonnegativity
state-space constraint appears.
Inventory and Production
The first example is concerned with a firm's decision regarding inventory
and production.1 0 To begin with, let the demand for the firm's product be
given generally as D(t) > 0. To meet this demand, the firm can either draw
down its inventory, X, or produce the demanded quantity as current
output, Q, or use a combination thereof. Production cost per unit is c, and
storage cost for inventory is s per unit, both assumed to be constant over
time. The firm's objective is to minimize the total cost f0T(cQ + sX) dt over
a given period of time [0, T ].
It is obvious that output cannot be negative, and the same is true of
inventory. Thus, Q(t) ;;:: 0 and X(t) ;;:: 0. These two variables are related to
each other in that inventory accumulation (decumulation) occurs whenever
output exceeds (falls short oO the demanded quantity. Thus,
X( t) = Q(t) - D(t)
This suggests that Q can be taken a s a control variable that drives the state
variable X.
Assuming a given initial inventory X0 , we can express the problem as
Maximize
jT( - cQ - sX ) dt
subject to
X = Q - D(t)
( 10.89)
0
-X �
X(O)
and
0
= X0
Q E [0,
oo)
X(T)
<!: . o
free
( X0, T given)
Note that minimization of total cost has been tranaJated into maximization
10
,.,;_,� _ to Applied Optimal Control,
This problem is discussed in Greg Knowles, An l
.
' ,.
·,
Academic, New York, 1981, pp. 135-137.
308
PART 3: 0PTUMAL CONTROL THEORY
of the negative of total cost. Note also that the nonnegativity state-space
constraint automatically necessitates the truncation of the vertical terminal
line.
=
The unconstrained view of this problem shows that the Hamiltonian
H
- cQ :_ sX + A(Q - D)
i s linear i n the control variable
Q, with
- = -c
aH
aQ
The rule for the choice of
+A
Q is therefore
Q*
{=
0
is indeterminate
is unbounded
}
Q) is not feasible, whereas the second
Q) is not helpful. Observe, however, that even in
the case of A
we can still select Q*
0, as in the case where A
In
fact, Q*
0 would make a lot of sense since Q enters into the objective
functional negatively, so that choosing the minimum admissible value of Q
= c,
< c.
The third possibility (unbounded
possibility (indeterminate
=
=
serves the purpose of the problem best. However, then the equation of
motion becomes
X = -D(t)
[when
< 0
Q
=
0)
and the firm's inventory is bound to be exhausted sooner or later. When the
= 0, it must reorient the control.
firm runs into the constraint boundary
Before applying conditions (10.84), we first verify that the constraint
X
qualification is satisfied. This is indeed so, because the constraint function
-X
is linear in
Q
(it contains no
Q).
From the constraint
-X ;:<;; 0,
we
readily find that
h
= -X c = O
=c
and
h = -X = - (Q - D)
Thus, by (10.83), the Lagrangian is
d''
-
Q - sX + A(Q - D) + 0(Q - D)
which i s linear in the control variable
Q. The conditions i n (10.84) require
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
309
that
../''
be maximized with respect to
a../''
ae
Q
[comer solution]
8(Q - D ) = 0
-=Q-D>O
0X = 0
[ 0 when X > 0]
=
a../''
=Q -D
aA
B../''
A = - -- = s
ax
A( T ) ;;;: 0
X( T ) ;;;: 0
X=
A( T ) X( T ) = 0
Acting on the clue from the unconstrained view of the problem,
first choose
we
Q=O
which simplifies the state equation of motion to
X=
-D
This result indicates that the firm should produce nothing and meet the
demand exclusively by inventory decumulation. While following this rule,
the inventory existing at any time will be
But such a policy can continue only up to the time t = r, when the given
inventory X0 is exhausted. The value of -r, the junction point, can be found
from the equation
{0 n(t) dt
=
X0
[exhaustion of inventory]
If we assume that -r < T, then at t = r we need to revise the zero-produc­
tion policy.
The rest of the story is quite simple. Having no inventory left, the firm
must start producing. In terms of the maximum principle, the activation of
the constraint makes the 8 multiplier positive, so that a../'';ae 0, or
=
Q=D
for t E
[ -r, T ]
PART 3, 0PTU4AL CONTROLTHEORY
310
Under this new rule,. the firm should meet the entire demand from current
production. Inasmuch as
X = Q - D, it follows that
X= O
for
t E [ 'T, T ]
meaning that no change in invento-ry should be contemplated. Since
0,
inventory should remain at the zero level from the point
complete optimal control and state paths are therefore
( 10.90)
Q*(O, T)
=
0
Q* (T, T ] = D( t)
'T
X(T ) =
on. The
X*[O, T) = X0 - fD(t) dt
0
X* ( T, T ] = 0
Capital Accumulation under Financial
Constraint
William Schworm has analyzed a firm that has no borrowing facilities, and
whose investment must be financed wholly by its own retained earnings.U
This model can serve as another illustration of a state-space constrained
problem.
The firm is assumed to have gross profit
expenditures
J(t).
7T(t, K ) and investment
J(t) � 0. Besides, it
It cannot sell used capital. Hence,
cannot borrow funds. Thus its cash flow
c/l(t) is dependent only on the profit
proceeds and the investment outlay:
cJ!(t) = 7T(t, K )
I t is
the .,.I of the firm t o maximize
-
I(t)
its present
[0 'c/1( t)e-P1 dt
where the symbol
value .
p denotes the rate of return the stockholders can earn on
alternative investments as well as the rate of return the firm can earn on its
retained earnings.
R(t)
p) on R, and by
Starting from a given initial level, the firm's retained earnings
can be augmented only by any returns received (at the rate
any positive cash flow. Consequently, we have
R (t) = pR(t) + c/l (t)
= pR(t) + 7T(t, K)
-
l( t)
11William E. Schworm, " Financial Constraints and Capital Accumulation," International
Economic Reu�w, October 1980, pp. 643-660.
\
Assuming no d
�preciation, we also have
CHAPTER 10: OPTI
CONTROL W1TH CONSTRAINTS
K(t)
=
311
I(t)
These two differential equations suggest that R and K can serve as state
variables of the model, whereas I can play the role of the control variable.
R and K must, of course, be nonnegative. But whereas
automatically takes care of the nonnegativity of
the assumption of I(t)
K, the nonnegativity of retained earnings, R, needs to be explicitly pre­
scribed in the problem. That is why the problem features a state-space
Both state variables
�0
constraint.
Collecting all the elements mentioned above, we can state the problem
as follows:
Maximize
([ rr ( t, K ) - I( t)] e-P1 dt
subject to
R(t)
=
pR(t) + rr( t, K) - l(t)
K( t)
=
I( t)
( 10.91)
0
�
-R(t)
R(O)
and
I( t)
=
E
0
R0
[0,
K(O)
=
K0
oo)
Being given in the general-function form, this model can only be analyzed
qualitatively. Actually, Schworm first transforms the model into one where
R(t) is a constrained control (rather than state) variable. But we shall treat
it as one with a state-space constraint and derive some of Schworm's main
results by using the maximum-principle conditions in
(10.84).
Again, we can verify that the constraint qualification is satisfied,
because the constraint function, - R(t), is linear in the control variable I.
The constraint function also supplies the information that
h
=
-R
h
=
-R
=
- ( pR + 7r(t, K ) - I ]
Thus, by (10.83), we have the Lagrangian
...£''
=
[ rr(t , K ) - I ] e -pt + A n ( p R
"•'.•, : . "
+ rr(t , K ) - I ]
+ A K I + E> ( p R + rr(t, K ) - I]
• • ;
where AR and A K are the costate variables for R and x; reepective)y. The
312
PART 3: OPTIMAL CONTROL THEORY
conditions in
( 10.92)
( 10.93)
( 10.94)
{ 10.95)
(10.84) stipulate that
iJ.£"
-= - e -P1 - A. R + A. X - 0 -< 0
iJI
iJ./'
iJ0 = pR + 1r( t, K ) - I � 0
R(t) � 0
0R(t) = 0
.
( = 0 when R > 0)
iJ
./'
.
R = iJ
= pR + 1r(t, K ) - I
A. R
iJ./'
K= iJA.x = I
iJ./'
.
A. R = - iJ = -p( A. R + 0)
R
.
()..£''
I�O
iJ./'
IiJ I = 0
iJ./'
0- = 0
a0
·.· i i
Ax = - iJ = - TTx(e-P' + A. R + 8)
K
plus transversality conditions
Note that, by virtue of the nonnegativity restriction on the control variable
I, the Kuhn-Tucker conditions are applied in (10.92).
While Schwarm discusses both the cases of I(t) > 0 and I(t) = 0, we
shall consider here only the optimization behavior for the case of I(t) > 0.
With I positive, complementary slackness mandates that iJ./ 'jiJ I = 0, or
[ by ( 10.92)]
Differentiating this equation with respect to
Ax = -pe-p1 + A R + 0
t, we get
= -pe-P' - p( A. R + 0) + 0
[by ( 10.94)]
= -p ( e -pt + A. R + 0) + 0
Moreover,
by equating this Ax expression with (10.95), we find that
( 10.96 )
This result embodies the optimization rule for the firm when I > 0.
It is of interest that, except for the 0 term on the right, the two sides
of (10.96) are identical in structure. From (10.93) we know that 0 = 0 when
R > 0. We can thus conclude from (10.96) that whenever R > 0, the firm
313
CHAPTER 10: OPTIMAL CONTROL WITH CONSTRAINTS
should see to it that
( 10 .97)
7rx = P
[investment rule when R > OJ
That is, when retained earnings are positive (financial constraint nonbind­
ing), the firm should simply follow the usual myopic rule that the marginal
profitability of capital be equal to the market rate of return.
If, on the other hand, R = 0 (financial constraint binding) in some
time interval (t1, t 2 ), then it follows that R 0 and R 0 in that interval.
Hence, the R(t) equation in (10.91) will reduce to
=
( 10 .98) l(t)
=
1r(t, K )
[investment rule when R
=
=
R = 0]
This means that in a constrained interval, the firm should invest its current
profit. We see, therefore, that when the R constraint is binding, investment
becomes further constrained by the availability of current profit.
Rules (10.97) and (10.98), though separate, are to be used in combina­
tion, with the firm switching from one rule to the other as the status of the
R constraint changes. By so doing, the firm will be able to get as close as
possible to the unconstrained optimal capital path.
Schworm also derives other analytical conclusions. For those, the
reader is referred to the original paper.
EXERCISE 10.4
1
2
Draw appropriate graphs to depict the Q* and X* paths given in
In problem (10.89), the demand is generally expressed
different specifications of D(t) affect the following?
as
(10.90).
D(t). Would
(a) r
(b) Q*[O, r)
(c) Q*[r, TJ
(d) X*[O, r)
(e)
10.5
X*[r, T)
LIMITATIONS OF DYNAMIC
OPTIMIZATION
While by no means exhaustive, the present volume has attempted to intro­
duce-in a readable way, we hope-most of the basic topics in the calculus
of variations and optimal control theory. The reader has undoubtedly
noticed that even in fairly simple problems, the solution and analysis
procedure may be quite lengthy and tedious. It is for this reason that simple
specific functions are often invoked in economic models to render the
solution more tractable, even though such specific functions may not be
totally satisfactory from an economic point of view.
314
PART s, OPTIMAL CONTROL THEORY
It
is for the same reason that writers often assume that the parame­
ters in the problem, including the discount rate, remain constant through­
out the planning period. Although there do exist in real life some economic
parameters that are remarkably stable over time, certainly not all of them
are. The constancy assumption becomes especially problematic in infinite­
horizon problems, where the parameters are supposed to remain at the
same levels from here to eternity. Yet the cost-in terms of analytical
complexity-of relaxing this and other simplifying assumptions can be
extremely high. Thus we have here a real dilemma. Of course, in practical
applications involving specific parameter values, the simple way out of the
dilemma is to reformulate the problem whenever there occur significant
changes in parameter values. But the constancy assumption may have to
be
retained in the new formulation.
Owing to the complexity of multiple differential equations, economic
models also tend to limit the number of variables to be considered. With the
advent of powerful computer programs for solving mathematical problems,
though, this limitation may be largely overcome in due time.
After the reader has spent so much time and effort to master the
various facets of the dynamic-optimization tool, we really ought not to end
on a negative note. So by all means go ahead and have fun playing with
Euler equations·, Hamiltonians, transversality conditions, and phase dia­
grams to your heart's content. But do please bear in mind what they can
and cannot do for you.
·.t
ANSWERS TO
SELECTED
EXERCISE
PROBLEMS
_,'
,,
1.2
EXERCISE
3 The terminal curve should be upward sloping.
1.3
EXERCISE
1
F[t, y(t), y'(t)]
2 (b) and (d)
=
1
1.4
EXERCISE
V*(D) 8; optimal path DZ i s DGIZ.
V*(E) 8; optimal path EZ is EHJZ.
V*(F) 10; optimal path FZ is FHJZ.
V*(K)
4 V*(I) = 3; IZ
V*(J) 1; JZ
V*(A) 23; ACFHKZ
1
=
=
=
=
=
=
2;
�.
2.1
EXERCISE
2 dljdx
4x3(b - a)
4 dljdx 2e2x
6 y*(t) -l.t3 + �t + 1
=
,, · - .
=
8 y*(t)
=
=
e' + e-•
+
!te'
315
316
ANSWERS TO SELECTED EXERCISE PROBLEMS
2.2
EXERCISE
y*(t) 2t
3 y*(t) = !t 2 + it + 2
5 y*(t) e< I + t)/2 + e <I - t)/2
1
=
=
2.3
EXERCISE
1
y*(t) = t
3 y* (t) z* (t)
=
4
e ""/2 - e .-;2
----,
;;;-;;;­
2.4
EXERCISE
1
-
a + 2aab + {3b
P = P, = static monopoly price
O
The lowest point of the price curve occurs at
P. =
2b(l + ab)
t0
=
>
(In A 2 - In A 1 )j2r � 0
Only case (c) can possibly (but not necessarily) involve a price reversal
the time interval [0, T ]. The other cases have rising price paths.
2.5
EXERCISE
2
3
(a)
in
Not different.
(b) A1 � 0, A 2 > 0
3.2
EXERCISE
1 (a) y*(t) = 4
3 (a) y*(t) = t + 4, T*
EXERCISE
=1
3.3
1 y*(t)
!<t 2 - 3t + 1)
2" (a) No.
(b) y*(t) = l t2 - tt + 1
3 (a) Only one condition is needed, to determine T*.
=
EXERCISE
1
(a)
3.4
'fT'(Lr) 2p/bk > 0. Thus Lr* is located such that the slope of the
'fT(L) curve is 2p/bk, on the positively sloped segment.
=
' 1
317
ANSWERS TO SELECTED EXERCISE PROBLEMS
2
(b)
An increase in p (or b, or
m
(a) L/
=
n
- !!_ /bk
n
(b) 7T'( L) = 0 when L
k) pushes the location of L/
to the left.
m
=
n
-
( c) L T* is located to the left of mjn, such that the 7T( L) curve has a
positive slope.
EXERCISE 4.2
1
3
(a) There is no y term in
F, and F is strictly convex in y'.
(c ) Sufficient for a unique minimum.
(a) The determinantal test (4.9) fails because ID21 = 0.
(b) The determinantal test (4.12) is satisfied for positive semidefiniteness
because I D 1 1 = 8 and 2, and ID21 = 0. The characteristic roots are
r1
10 and r2 = 0.
(c) Sufficient for a minimum.
=
EXERCISE 4.3
1
Problem 6: Fy'y' = 4 for all t; satisfies the Legendre condition for a
minimum.
Problem 9: Fy'y' = 0 for all t; satisfies the Legendre condition for a
maximum as well as for a minimum.
3 F.,.. .,.• = 2
( ;2;: )
1
2
e -pt
> 0 for all t; satisfies the Legendre condition for a
minimum.
EXERCISE 5.2
1
K * (t )
3
Cg•g•K" - pCg• + 'TI'g
4
K * (t) =
=
( K0 - 25 + p) exp
(
=
(K0 - 25) exp
0
(
P -
P -
�)
t
�)
t
+ 25 - p
+ 25
EXERCISE 5.3
1 (a)
(b)
C * ( t)
K * ' (t
= A - 1/(b + 1 )e rt/(b+ 1)
)=
B -
U+
(C
=
C * 0 e rt/Cb+
( C * ) - bjb
* ) - <b + U
�
1
-C*
b
1)
[since B =
U)
3UJ
r
(c) K*( t) = __ K e r t /(b+ !)
b+ 1 °
rtf(b + IJ
(t)
=
K
d)
( K*
oe
EXERCISE
2
5.4
( a ) No, (5.40) is valid with or without saturation.
(b) Equation (5.41) is unaffected, but (5.42) should be changed to Q'(K) =
0 because now IL can never be zero.
3 The new equilibrium is still a saddle point, but it involves a positive (rather
5
than zero) marginal utility in equilibrium.
(a) Only the streamlines lying above the stable branch remain relevant;
the others cannot take us to the new level of K T·
(c) Yes.
EXERCISE 6.1
A 2
3 y*(t) = - 4 t
4 y*(t)
5
=
+
c1t
+
c2
A1e1 + A2e-1 + c1
The general solution is the same as in Example 2.
EXERCISE 6.2
1
5'= B - U(C) + D(L) + A[ - C + Q(K, L) - K']
- U'(C) - A = 0
A = - U'(C) = - J.L
2 (a)
.J;,:;i �-·,·;,_
Three variables and two constraints.
(b) 5'= (1T - C)e-pt + A1[ - 1T
(c) For 1r: e -pt - A1 = 0
For C: - e -P1 - A2 = 0
d
+ a K - {3K 2 ] + A 2[ - C + aK'2 + bK' ]
=
=
A1 = e -pt
A 2 = - e-P'
For K: A 1(a - 2{3K) - d [A2(2aK' + b)] -"" 0
t
dA 2
A1(a - 2{3K) - dt(2aK' + b) - A 2(2aK") = 0
=
EXERCISE 6.3
2
4
Similarly to (6.10), A may now be
condition will come into play.
rQ • = g -
Ep
rQm - rQ •
+
rE
E- 1
>
<-: 0. A complementary-slackness
rQ •
[ E > 1 for positive MR]
AHSWERS TO SELECTED EXERCISE PROBLEMS
6
(b)
319
{'N(q, q')e -P' dt
Maximize
0
f0ooq' dt = So
subject to
EXERCISE 7.2
2
A*(t) = 3 e4 - 1 - 3
u* ( t) = 2
4 u*(t) = A*(t) = A 1 e li•
y *(t)
=
7e1 - 2
+ A2e- fi•
y* (t) = ( /2 - 1)A 1 e .f2• - (/2 + 1)A2 e - fit
where A1 =
'
,.
A2 =
- e - 2/i
r.;
( 1 - /2 ) e - 2V2 - ( /2 + 1 )
(1 -
1
--:=
-
-
/2 ) e - 2 12 - ( /2 + 1 )
EXERCISE 7.4
A*(t) = 2
u*(t) = 1
y*(t) = t + 4
u* = - 1
A* = - !
T*
y *(t) = - 2t + 8
4 u* = Jl/38
y*(t) = Jl/38 t
A* = Jl/39
1
3
=
4
(
EXERCISE 7.6
( )
1
(a) The cyclical pattern will be eliminated.
3
-
a
or
dU*
--
dt
1
= - (T - t)-khbae8<T-o < 0 for all t (except t = T )
2
EXERCISE 7. 7
,
1
(a) Yes.
3
(a) H = U[C(E), P(E))e -P1 - A E
(b)
' ---\.
No, E* 2 is characterized by " marginal utility > ma.rgiDal di8utilit;y."
aH
··
:::
aE
= [ UcC'(E) + UpP'( E ))e -pt - A = 0
(b)
The A path is still a constant (zero) path.
E*(t) E * as in (7.81).
(d) The condition in part (a ) becomes
(c)
=
· · . · ;· .
ANSWERS TO SELECTED EXERCISE PROBLEMS
320
so
dE
-
dt
=
-a'lt ;at
---
a'lt;aE
=
peeP'
-;;--;;-<
-
-
UccC' 2
+
UcC" + UppP' 2
+
Up P"
0
EXERCISE 8.2
1 ( He - m <f>'J,_re-pT 0
T* :5: Tmax
(Hclt� T � 0
=
3
EXERCISE 8.3
1
The Mangasarian conditions are satisfied, based on F being strictly concave
in u (Fuu
- 2), f being linear in (y, u ), and (8.26) being irrelevant. The
Arrow condition is satisfied because H 0 Ay + !A2, which is linear in y
for given A.
4 The Mangasarian and the Arrow conditions are both satisfied. (Here, H 0 is
independent of y, and hence linear in y for given A.)
6 Both the Mangasarian and Arrow conditions are satisfied.
=
=
EXERCISE 8.4
2 For a positive y0 , the
y * value at first increases at a decreasing rate, to
reach a peak when arc CD crosses the y axis. It then decreases at a
decreasing rate until -r is reached, and then decreases at an increasing rate,
to reach zero at t T .
3 (a) The initial point cannot be on arc AO or BO; there must be a switch.
(b) The initial point lies on arc AO; no switch.
(c) The initial point lies on arc B O; no switch.
5 (a) The switching point, F, has to satisfy simultaneously
=
y=
and
y
=
-
!z2
!z2 + k
(z > 0)
[equation for arc B 0]
(k
[equation for parabola containing arc EF ]
<
0)
The solution yields y
inadmissible).
( b ) 7' Fk - zo
(c) The optimal total time
=
�k and
=
=
2Fk
-
z
=
Fk (the negative root is
z0•
EXERCISE 8.5
1
An unbounded solution
.can
occur when A*
*
0.
321
ANSWERS TO SELECTED EXERCISE PROBLEMS
3
It is not possible to have an interior solution for the control variable A.
[ Hint: An interior solution A* would mean that {3Ap + A s = 0 , which
would imply Ap = constant, and Upe-p1 = SA p from the costate equation
of motion. The existence of a tolerably low level of P would imply A p = 0
for all t, which would lead to a contradiction.]
EXERCISE 9.3
1
(a) tf>(k)
(c) He =
=
.
A
AK"
1
k
=
U '( c ) =
c -< 1 +bl
be- + m[Ak " - c -
U-
c -
(d) k = AP -
b
(n +
fi)k
c=
( n + /3 + r ) 1/(a - 1)
(
(n
c
c =A
Aa
B)k]
1+b
--
Aa
( n + {i + r ) " /( a - 1 )
+
- (n + [i )
[ A a k " - 1 - (n + {i +
.
r)]
( n + {i + r ) 1/(a - 1)
Aa
3 ( b) E will give way t o a new steady state, E', o n the k = 0 curve to the
right of E, with a higher k.
(c) No.
EXERCISE 9.4
as
1
(b) To write it
5
The marginal productivity of human capital in research activities could not
grow in proportion to A.
Y = K"[ A(t)Lil] will conform to the format of (9.48).
EXERCISE 10.1
3 (a) Maximize
subject to
JT(l - s)Ke -p1 dt
0
K = A(sK)"
K(O)
=
K0
(A > 0)
K(T ) free
O�s � 1
and
EXERCISE 10.3
1
(a) A
=
A +
<b> A - ;. +
Shy
my + 9(h,y + hyy n
(K0, T given)
ANSWERS TO SELECTED EXEBCIM PROJILBIIIII!I
S22
(c) A =
(e) ( ()
+
-Fy - A {y + ()hy + ehy + ®( h,y + hyy f)
e )hy = 0
e = - () :;;; 0 (since () ;;:: 0]
=
EXERCISE 10.4
2
(a) Higher Oower) demand would lower (raise) -r. But it is also conceivable
that a mere rearrangement of the time profile of D(t) may leave -r
unchanged.
(c) Yes, because Q*[-r, T] is identical with D(t).
· (e) X*[-r, T] 0 regardless of D(t), but the length of the' )leriod (-r, T ]
would be affected by any change in T.
·:�
=
'\
' ; �· '
IND EX
. ,.
Adaptive expectations,
Antipollution policy,
Arc,
55, 194
234-239
4-5
· �"'
Arc length,
41
5, 13
Arrow, K. J., 217, 244n., 245-247, 268, 278
Arrow sufficient condition, 217-218, 252
applied, 219-221
for constrained problems, 290-291
Autonomous problem, 103, 212
Arc value,
constancy of optimal Hamiltonian
examples of,
in, 190
106, 113, 248-249, 257
Auxiliary variable, 167
164, 1 76, • *
J., 292-293
Bellman, R. E., 20, 22
·"-'""
Benveniste, L. M., 251
Bernoulli, J., 17
Beyer, W. H., 42n.
Bliss, 1 13, 123, 132, 248
Blocked interval, 299
Boltyanskii, V. G., 19n., 167
Bolza, problem of, 15-16, 76
Boundary solution, 164, 173, 237-239
Burness, H. S., 152-153, 155
Bang-bang solution,
Baumol, W.
Capital accumulation:
of a firm
(see Jorgenson model;
Eisner-Strotz model)
under financial constraint,
golden rule of,
modified golden rule of,
of society
310-313
248, 259
259
(see Ramsey model; Neoclassical
theory of optimal growth)
Capital saturation,
121, 125-126
104
Cass, D., 253, 263-264
Catenary, 43
Catenoid, 43
Chakravarty, S., 99n., lOOn.
Characteristic-root test, 85-86
Chiang, A. C., 38n., 40n., 50n., 65n., 86n.,
88n., 1 17n., 153n., 174n., 183n.,
185n., 189n., 197n., 245n., 262n.,
292n .
Cobb-Douglas production function, 105, 270
Complementary slackness, 138, 183, 236, 238,
279, 287, 302, 312
Concavity, checking, 83-86, 88-90
Conservation, 151-156
Constrained interval, 299
Constraint qualification, 278
applied, 294, 305, 308, 311
Capital stock,
Constraints in calculus o f variations:
Calculus of variations,
differential-equation,
17-18
fundamental (simplest) problem of,
link with optimal control theory,
191-193
Canonical system,
170-171
18, 27
166-167,
137
equality,
134-136
inequality, 137
integral, 138-141
isoperimetric problem,
138-141
323
J24
INDEX
Embedding of problem, 20-22
Endogenous technological progress, 267-274
�onstraints in optimal control theory:
equality, 276-277
inequality, 277-280
inequality integral, 282-284
integral, 280, 282
isoperimetric problem, 280-282
Energy conservation, 151-156
Environmental quality, 200
Equation of motion, 18, 165-166, 169, 181
with second-order derivative, 227
state-space, 298-303
�onsumption saturation, 121, 125-126
�antral set, 19-20, 164
�antral variable, 161-162
�onvergence of functional, 99-101, 1 06,
�onvexity, checking, 83-86, 88-90
�orner solution (see Boundary solution)
�ostate equation, 181
�ostate variable, 167, 280
constancy of, 280
as shadow price,
Euler equation, 28, 33, 61, 192
113
development of, 32-34
expanded form of, 34
for higher derivatives, 4 7-48
integral form of, 33
for several variables, 45-46
special cases of, 3 7-45
Euler-Lagrange equation, 135-138, 141-142
Euler-Poisson equation, 48
Evans, G. C., 49, 69, 70 n .
Exhaustible resource, 148-156, 200
206-208
�ounterexamples against infinite-horizon
transversality condition:
Halkin, 244-247
Expectations-augmented Phillips relation, 55,
194
Extremal, 27
Pitchford, 250-251
Shell, 247-251
�ourant, R., lOOn.
�urrent-value Hamiltonian, 210, 213, 257,
First variation, 95-97
Fixed-endpoint problem (see Horizontal­
terminal-line problem)
�urrent-value Lagrangian, 287-289, 291, 294
Fixed-time (-horizon) problem
terminal-line problem)
Flexible accelerator, 105, 1 1 0
)antzig, G. B., 217n .
Fulks, W., lOOn.
)eterminantal test:
Functional, 7-8
287-290, 291, 294
::urrent-value Lagrange multiplier, 210
for sign definiteness, 84-85
(see Vertical-
Forster, B . A., 200, 204, 234
Gamkrelidze, R. V., 19n., 167
for sign semidefiniteness, 85
)ifferential equation:
with constant coefficient and constant term,
50, 189
with constant coefficient and variable term,
197
with separable variables, 143
with variable coefficient and variable term,
185, 244-245
>ifferentiation of definite integral, 29-31
>orfman, R., 205, 243, 291
)ynamic monopolist, 49-53, 69-70, 87-88,
94
)ynamic optimization, 3-6
Golden rule, 248, 259
modified, 259
Gould, J. P., 105n.
H (see Hamiltonian function)
He (see Current-value Hamiltonian)
H 0 (see Maximized Hamiltonian)
Hadley, G., lOOn., 130n.
Halkin, H., 244-247
Hamermesh, D. S., 75
Hamiltonian function, 167-168
current-value, 210, 213, 257, 287-291, 294
economic interpretation of, 207
linear
)ynamic production function, 264-265
(in
u ),
170
maximized, 218, 252, 290
)ynamic programming, 20-22
optimal, 190
Hamiltonian system, 170-1 7 1
�fficiency labor, 266
�isner (see Eisner-Strotz model)
Harrod neutrality, 265, 2 7 1 , 273
Hestenes, M. R., 19, 167, 301n.
Hicks neutrality, 265
•:isner-Strotz model, 106-110, 124, 148
as a control problem, 212, 220
Horizontal-terminal-line problem, 111-11, 63,
phase-diagram analysis of, 1 1 7-120
sufficient conditions applied to, 130, 220
182, 187-190
,· ,
INDEX
Hotelling, H.,
Lewis, T. R.,
Human capital,
L'Hopital's rule,
Hurwicz,
148, 149n., 152, 162
269
L., 278
Implicit-function rule,
296-297
Infinite horizon:
·
,,,;,. · · . .._, .:
····
convergence of functional,
99-101, 106, 113
98-103
transversality conditions, 101-103, 115,
240-243, 261-262
counterexamples against, 244-251
Inflation-unemployment tradeoff, 54-58,
70-72
as a constrained problem, 145-14 7
Integration by parts, 32, 82, 216
Intertemporal resource allocation, 111
Inventory problem, 307-310
methodological issues of,
Investment
Isochrone,
( see
Capital accumulation)
139-143, 149, 150,
262
244n., 290 n.
Mangasarian,
0.
L.,
214
Mangasarian sufficient conditions,
214-217,
252
applied,
219-221
for constrained problems,
290-291
152-153, 155
Maximized Hamiltonian, 218, 252, 290
Maximum area under a curve, 141-143
Maximum principle, 19, 167-171
Matthews, S. A.,
(see
Constraints
21 1-212
205-210
economic interpretation of,
rationale of,
177-181
for several variables,
134, 137n .
Jacobian matrix, 262-263
Jorgenson model, 103-105
Jump discontinuity, 164, 300
Junction point, 300, 309
223-226
15-16, 76
lOOn.
Mayer, problem of,
McShane, E. J.,
Michel, P.,
1., 168n., 217n., 221n., 306n.
Kemp, M. C.,
lOOn., l30 n .,
307n.
Kuhn, H. W., 247n.
Kuhn-Tucker conditions, 65, 102, 183, 236,
278-279, 303
Kurz, M., 217n., 244n., 245-247
Knowles, G.,
.f (see Lagrangian function)
(see Current-value Lagrangian)
distance problem)
Minimum principle,
265
75-78
Lagrange multipliers, 135, 178
constancy of, 139-140
current-value, 210
Lagrangian function (augmented
276, 279
current-value, 287-289, 291, 294
Lagrangian integrand function, 134-135, 138,
140
Learning by doing, 268
Legendre necessary condition, 91-94, 192
Leibniz's rule, 29-30
applied, 31, 32, 80
Leland, H. E., 293, 295, 296
19n.
Minimum surface o f revolution,
40-41, 93
Mishchenko, E. F.,
Modified golden
19n., 167
rule, 259
Monopoly:
consequences of,
Multistage
Labor-demand adjustment,
243
Minimum-distance problem (see Shortest-
dynamic,
.L',
Hamiltonian),
equations,
Long, N. V.,
for current-value Hamiltonian,
Jacobian determinant,
Labor-augmenting technological progress,
Linearization of nonlinear differe�ial
in optimal control theory)
280-282
Isovote curve, 194
Kamien, M.
152-153, 155
249 n.
of F function, 35-36
for constrained problems
127-128
Isoperimetric problem,
Linearity
325
150-156
49-53, 69-70, 87-88, 94
decision making, 4-6
Natural boundary condition,
Negative definiteness
(see
Negative semidefiniteness
63
Quadratic form)
( see
Quadratic
form)
Neoclassical production function,
103, 247,
253-254, 293
Neoclassical theory of optimal growth,
253-262
with Harrod-neutral technological progress,
265-267
Neutral technological progress,
Newton,
1., 17
Nonnegativity restriction,
Nordhaus,
265
298, 308
W. D., 193, 196, 199
J26
INDEX
)bjective functional, 12-16
Research success coefficient, 268, 270
)ptimal control theory, 18-20
Revenue-maximizing firm, 292-298
features of, 163-164
Rival good, 269
link with calculus of variations, 166-167,
Romer, P.
M., 269-274
191-193
with several state and control variables,
221-226
example from Pontryagin, 226-233
simplest problem of, 162, 164-166
)ptimal policy functional, 21
)ptimal value function, 20-21
Saddlepoint equilibrium, 120, 123-124, 262
Samuelson, P.
A., 126 n .
Sato, K., 269
Scheinkman, J. A., 2 5 1
Schwartz, N.
Schworm,
W.
L.,
1 6 8 n . , 2 1 7 n . , 2 2 1 n . , 306n.
E., 310-313
Second-order conditions, 79-81, 91-94
Second variation, 95-97
'ath:
admissible, 6-7
Seierstad,
optimal, 6
sgn
(see
A., 99n . , 221n., 300n., 303n.,
Signum function)
'ath value, 6-7
Shadow price, 206-209
'erturbing curve, 28, 180-182
Shell, K., 247-251, 255, 268
'hase diagram:
Shortest-distance problem:
one-variable, 38-39, 229-231
two-variable, 1 1 7-129, 260-262
in calculus of variations:
between a point and a line, 66-67
'helps, E. S., 248n.
between two given points, 44
'iecewise continuity, 163
between two points on a surface, 136-137
'iecewise differentiability, 163
'itchford, J. D., 244n, 250, 251, 290 n .
'olitical business cycle, 193-199, 284-286
second-order condition, 86-87, 93
in optimal control theory:
between a point and a line, 1 7 1 - 1 73
sufficient conditions, 218-219
'ollution, 200
as a flow, 200
Signum function, 188, 227, 286
as a stock, 234-235
Social loss function, 54-55
'ontryagin,
241n.
L.
S., 19, 167, 177, 226, 232, 240,
'ositive definiteness (see Quadratic form)
'ositive semidefiniteness
Social utility function, 255
Solow, R., 248
( see Quadratic form)
Solow neutrality, 265
Stable branches, 1 19, 120, 124, 260-261
'rinciple of optimality, 22
Stage variable, 4
'rinciple of reciprocity, 143
State equation, 18, 165
'roduction function:
State-space constraints, 298-303
Cobb-Douglas, 105, 270
economic examples of, 307-313
dynamic, 264-265
State variable, 4
neoclassical, 103, 247, 253-254, 293
Steady state, 1 18, 248, 260
'ure competition and social optimum,
150-151
'ythagoras' theorem, 40
and Harrod neutrality, 265-266, 273
Stiglitz, J. E., 151-155
Strotz, R. H.
(see
Eisner-Strotz model)
Sufficient conditions:
!uadratic form, 83-86, 88-90
in calculus of variations, 81-83
'.amsey, F. P., 1 1 1 , 128
in optimal control theory, 214-218, 252
:amsey device, 1 13, 248, 250
:amsey model, 1 1 1-116
as a constrained problem, 144-145
phase-diagram analysis of, 1 18-120,
122-126
simplified, 120-126
sufficient condition applied to, 131-132
for infinite horizon, 130-132
applied, 219-221
for constrained problems, 290-291
Switching of control, 174-176, 230-233
Sydsreter, K., 99n., 221n., 300n., 303n.
Szegii, G. P., 247n.
Takayama,
A., 81n., 168 n ., 244n . , 278n.
:amsey rule, 115, 1 1 6
Taylor, D., 54
ID1 k condition, 278
Taylor series, 96
INDEX
Technological progress, 264
endogenous, 267-274
neutrru exogenous, 265
labor-augmenting, 265
Terminru-control problem, 15, 244
Terminal-curve problem, 11, 64, 182
Time-optimal problem, 11, 187-189, 227,
286-287
Tintner, G., 52
Tradeoff between inflation and unemployment
(see Inflation-unemployment tradeoff)
Trajectory, 27
Transition equation, 18
Transversality conditions:
in calculus of variations, 11-12, 60-63,
73-74
with horizontal terminal line, 63-64, 193
with infinite horizon, 101-103, 115
with terminal curve, 64-65
with truncated horizontal terminal line,
66
with truncated vertical terminal line,
65-66
with vertical terminal line, 63, 192
in optimal control theory:
economic interpretation of, 209-210
with horizontru terminal line, 182, 190,
193
with infinite horizon, 240-243, 247-250
327
with terminal curve, 182
with truncated horizontru terminal line,
184
with truncated vertical terminal line,
182-183
with vertical terminal line, 169, 181, 192
Trirogolf, K. N., 19n., 167n.
Truncated-horizontal-terminal-line problem,
66, 72
Truncated-vertical-terminru-line problem,
9-10, 65-66, 182-183, 186-187,
200-204, 235-236, 308
Turnovsky, S. J., 244n., 250n., 290 n .
Turnpike behavior, 126-129
Uzawa, H., 278
Variable endpoint, types of, 9-11
Variations, first and second, 95-97
Veinott, A. F., Jr., 217n.
Vertical-terminal-line problem, 9, 63, 164,
171, 181, 192, 195
Vote function, 193-194
Vousden, N., 244n., 290 n .
Weierstrass-Erdmann corner condition,
163n.
Weitzman, M . L., 251
Yellow brick road, 1 20, 124, 261