Probabilis/c Graphical Models Ac/ng Decision Making Maximum Expected U/lity Daphne Koller
Simple
mp Decision Making
g
A simple
p decision making
g situation D:
• A set of possible actions Val(A)={a1,…,aK}
• A sett of
f states
t t V
Val(X)
l(X) = {{x1,…,xN}
• A distribution P(X
( | A))
• A utility function U(X, A)
Daphne Koller
Expected
p
Utility
y
• Want to choose action a that maximizes
the expected utility
Daphne Koller
Simple
mp Influence
f
Diagram
g m
m0
m1
m2
0.5
0.3
0.2
Market
Found
U
m0
m1
m2
f0
0
0
0
f1
-7
5
20
Daphne Koller
More Complex
p
Influence Diagram
g
Difficulty
Intelligence
Study
Grade
VG
L tt
Letter
Job
VQ
VS
Daphne Koller
Information
f m
Edges
g
m0
m1
m2
0.5
0.3
0.2
Market
s0
Survey
Decision rule δ at
action node A is a CPD:
P(A | Parents(A))
m0
m1
m2
s1
0.6 0.3
04
0 3 0.4
0.3
0.1 0.4
f0
0
0
0
f1
-7
5
20
s2
0.1
03
0.3
0.5
Found
U
m0
m1
m2
Daphne Koller
Expected
p
Utility
y with Information
• Want to choose the decision rule δA
that maximizes the expected utility
Daphne Koller
Finding
F
g MEU Decision Rules
Market
Survey
Found
U
Daphne Koller
Finding
F
g MEU Decision Rules
Market
Survey
s0
m0
m1
m2
0.5
0.3
0.2
m0
m1
m2
s1
0.6 0.3
0.3 0.4
0.1 0.4
s0
s1
s2
f0
0
0
0
s2
0.1
0.3
0.5
f1
-1.25
1 25
1.15
2.1
m0
m1
m2
f0
0
0
0
f1
-7
5
20
Found
U
Daphne Koller
More Generally
y
Daphne Koller
MEU Algorithm
g
m Summary
mm y
• To compute MEU & optimize decision at A:
– Treat A as random variable with arbitrary CPD
– Introduce utility
y factor with scope
p PaU
– Eliminate all variables except A, Z (A’s
parents) to produce factor μ(A,
μ(A Z)
– For each z, set:
Daphne Koller
Decision Making
g under Uncertainty
y
• MEU principle provides rigorous foundation
• PGMs
PGM provide
id structured
t
t
d representation
t ti f
for
probabilities, actions, and utilities
• PGM inference methods (VE) can be used for
– Finding the optimal strategy
– Determining overall value of the decision situation
• Efficient methods also exist for:
– Multiple
p utility
y components
p
– Multiple decisions
Daphne Koller
Probabilis"c Graphical Models Ac"ng Decision Making U"lity Func"ons Daphne Koller
Utilities and Preferences
f
0.2
$4mil
p
0.25
$3mil
0.75
$0
f
0.8
$0
≈
Daphne Koller
Utility
y = Payoff?
y ff
0.8
$4mil
p
1
$3mil
0
$0
f
0.2
$0
≈
Daphne Koller
St. Petersburg
g Paradox
• Fair coin is tossed repeatedly
p
y until it
comes up heads, say on the nth toss
• Payoff = $2n
Daphne Koller
U
U($500)
D=
UD
0
400 500
$0
with prob. 1-p
$1000 with prob
prob. p
1000 $ reward
Certainty
C
t i t equivalent
i l t
insurance/risk premium
Daphne Koller
Typical
yp
Utility
y Curve
U
$
Daphne Koller
Multi-Attribute Utilities
• All attributes affecting
g preferences
p
must
be integrated into one utility function
• Human
H
llife
f
– Micromorts
– QALY (quality-adjusted life year)
Daphne Koller
Example:
mp Prenatal diagnosis
g
U1(T) + U2(K) + U3(D,L) + U4(L,F)
Testing
g
Down’s
’
syndrome
Knowledge
g
Loss of
f
fetus
Future
pregnancy
Daphne Koller
Summary
mm y
• Our utility function determines our preferences
about
b t decisions
d isi s that
th t involve
i
l uncertainty
t i t
• Utility generally depends on multiple factors
– Money,
M
time,
ti
chances
h
of
f death,
d th …
• Relationship is usually nonlinear
– Shape
Sh
of
f utility
tilit curve d
determines
t
i
attitude
ttit d tto risk
i k
• Multi-attribute utilities can help decompose
high dimensional function into tractable pieces
high-dimensional
Daphne Koller
Probabilis0c Graphical Models Ac0ng Decision Making Value of Perfect Informa0on Daphne Koller
Value of
f Information
f m
• VPI(A | X) is the value of observing X
before choosing an action at A
• D = original influence diagram
• DX → A = influence diagram with edge X → A
Daphne Koller
Finding
F
g MEU Decision Rules
Market
Survey
Found
U
Daphne Koller
Value of
f Information
f m
• Theorem:
– VPI(A | X) ≥ 0
– VPI(A | X) = 0 if and only if the optimal decision
rule for D is still optimal for DX → A
Daphne Koller
Value of
f Information
f m
Example
mp
s1
s2
s3
0.1
0.2
0.7
f0
s1
s2
s3
f1
.
0.9 0.1
0.6 0.4
0.1 0.9
State1
State2
s1
s2
s3
0.4
0.5
0.1
Funding1 Company
p y Funding2
V
δ*(C | S2) =
P(c2))=11 if S2 = s3
P(c1)=1 otherwise
EU(D[c
E
(D[ 1]) = 0.72
.
EU(D[c2]) = 0.33
MEU(DS1 → C) = 0.743
Daphne Koller
Value of
f Information
f m
Example
mp
s1
s2
s3
0.5
0.3
0.2
f0
s1
s2
s3
f1
.
0.9 0.1
0.6 0.4
0.1 0.9
State1
State2
s1
s2
s3
0.4
0.5
0.1
Funding1 Company
p y Funding2
V
δ*(C | S2) =
P(c2))=11 if S2 = s2,s3
P(c1)=1 otherwise
EU(D[c
E
(D[ 1]) = 0.35
.
EU(D[c2]) = 0.33
MEU(DS2 → C) = 0.43
Daphne Koller
Value of
f Information
f m
Example
mp
s1
s2
s3
0.5
0.3
0.2
State1
State2
s1
s2
s3
0.4
0.5
0.1
f0
f1
Funding1 Company
Funding2
1
p
y
.7
s 0.3 0.7
s2 0.2 0.8
s3 0.01 0.99
V
δ*(C | S2) =
P(c2))=11 if S2 = s2,s3
P(c1)=1 otherwise
EU(D[c
E
(D[ 1]) = 0.788
.
EU(D[c2]) = 0.779
MEU(DS1 → C) = 0.8142
Daphne Koller
Summary
mm y
• Influence diagrams provide clear and
coherent
h
t semantics
ti f
for th
the value
l of
f
making an observation
– Difference between values of two IDs
• Information is valuable if and only if it
induces a change in action in at least one
context
t t
Daphne Koller