Clase 3: Object detection

Aprendiendo y reconociendo categorias de objetos
Clase 3: Object detection
Juan Wachs
Computer Science Department & MOVES Institute
Naval Postgraduate School, Monterey, CA
Courtesy of Antonio Torralba
Aprendiendo y reconociendo categorias de objetos
Overview of section
• Object detection with classifiers
• Boosting
–
–
–
–
Gentle boosting
Weak detectors
Object model
Object detection
• Multiclass object detection
Aprendiendo y reconociendo categorias de objetos
Discriminative methods
Object detection and recognition is formulated as a classification problem.
The image is partitioned into a set of overlapping windows
… and a decision is taken at each window about if it contains a target object or not.
Background
Decision
boundary
Where are the screens?
Bag of image patches
Computer screen
In some feature space
Aprendiendo y reconociendo categorias de objetos
Formulation
• Formulation: binary classification
…
Features x =
x1
x2
x3 … xN
y=
-1
+1
-1
Labels
-1
xN+1 xN+2 … xN+M
?
Training data: each image patch is labeled
as containing the object or background
?
?
Test data
• Classification function
Where
belongs to some family of functions
• Minimize misclassification error
(Not that simple: we need some guarantees that there will be generalization)
Aprendiendo y reconociendo categorias de objetos
Boosting
• Defines a classifier using an additive model:
Strong
classifier
Weak classifier
Weight
Features
vector
• We need to define a family of weak classifiers
from a family of weak classifiers
Aprendiendo y reconociendo categorias de objetos
Overview of section
• Object detection with classifiers
• Boosting
–
–
–
–
Gentle boosting
Weak detectors
Object model
Object detection
• Multiclass object detection
Aprendiendo y reconociendo categorias de objetos
The GentleBoost approach
•
Our cost function is an empirical
average over the training data
N
J wse = ∑ w (zi − hm ( vi ) )
2
m
i
i =1
where
f
f
f
hm ( vi ) = aδ (vi > θ ) + bδ ( vi ≤ θ )
and
wim = e− zi H m −1 ( vi )
Minimizing J requires taking the derivative wrt a and then b, and
equating to zero, we get:
w z δ (v
∑
a=
∑ w δ (v
i i
i
f
i
f
i
i
i
>θ)
>θ)
and
b=
f
δ
(
w
z
v
∑i i i i ≤ θ )
f
w
δ
(
v
∑i i i ≤ θ )
Aprendiendo y reconociendo categorias de objetos
Regression “stumps”
•
Lets describe the weak classifier’s parameters by a decision tree:
hm ( v if )
f2
f1
θ1
a1 ,b1
θ2
a2 , b2
θ3
a3 ,b3
f3
θ4
f4
• • •
• • •
fK
θp
a4 , b4
Pick the node that minimizes the cost function Jwse
a p , bp
Aprendiendo y reconociendo categorias de objetos
GentleBoost Pseudocode
Aprendiendo y reconociendo categorias de objetos
From images to features:
Weak detectors
We will now define a family of visual features that can be
used as weak classifiers (“weak detectors”)
Takes image as input and the output is binary response.
The output is a weak detector.
Aprendiendo y reconociendo categorias de objetos
Weak detectors
Textures of textures
Tieu and Viola, CVPR 2000
Every combination of three filters
generates a different feature
This gives thousands of features. Boosting selects a sparse subset, so computations
on test time are very efficient. Boosting also avoids overfitting to some extend.
Aprendiendo y reconociendo categorias de objetos
Weak detectors
Haar filters and integral image
Viola and Jones, ICCV 2001
The average intensity in the
block is computed with four
sums independently of the
block size.
Aprendiendo y reconociendo categorias de objetos
Weak detectors
Other weak detectors:
• Carmichael, Hebert 2004
• Yuille, Snow, Nitzbert, 1998
• Amit, Geman 1998
• Papageorgiou, Poggio, 2000
• Heisele, Serre, Poggio, 2001
• Agarwal, Awan, Roth, 2004
• Schneiderman, Kanade 2004
• …
Aprendiendo y reconociendo categorias de objetos
Weak detectors
Part based: similar to part-based generative models. We
create weak detectors by using parts and voting for the
object center location
Car model
Screen model
These features are used for the detector on our course.
Aprendiendo y reconociendo categorias de objetos
Weak detectors
First we collect a set of part templates from a set of training
objects.
Vidal-Naquet, Ullman (2003)
…
Aprendiendo y reconociendo categorias de objetos
Weak detectors
We now define a family of “weak detectors” as:
=
*
=
Better than chance
Aprendiendo y reconociendo categorias de objetos
Weak detectors
We can do a better job using filtered images
*
=
=
*
=
Still a weak detector
but better than before
Aprendiendo y reconociendo categorias de objetos
Training
First we evaluate all the N features on all the training images.
Then, we sample the feature outputs on the object center and at random
locations in the background:
Aprendiendo y reconociendo categorias de objetos
Representation and object model
Selected features for the screen detector
…
…
1
2
3
4
Lousy painter
10
100
Aprendiendo y reconociendo categorias de objetos
Representation and object model
Selected features for the car detector
…
…
1
2
3
4
10
100
Aprendiendo y reconociendo categorias de objetos
Overview of section
• Object detection with classifiers
• Boosting
–
–
–
–
Gentle boosting
Weak detectors
Object model
Object detection
• Multiclass object detection
Aprendiendo y reconociendo categorias de objetos
Feature
output
Example: screen detection
Aprendiendo y reconociendo categorias de objetos
Feature
output
Thresholded
output
Example: screen detection
Weak ‘detector’
Produces many false alarms.
Aprendiendo y reconociendo categorias de objetos
Example: screen detection
Feature
output
Thresholded
output
Strong classifier
at iteration 1
Aprendiendo y reconociendo categorias de objetos
Example: screen detection
Feature
output
Thresholded
output
Second weak ‘detector’
Produces a different set of
false alarms.
Strong
classifier
Aprendiendo y reconociendo categorias de objetos
Example: screen detection
Feature
output
Thresholded
output
Strong
classifier
+
Strong classifier
at iteration 2
Aprendiendo y reconociendo categorias de objetos
Example: screen detection
Thresholded
output
Strong
classifier
+
…
Feature
output
Strong classifier
at iteration 10
Aprendiendo y reconociendo categorias de objetos
Example: screen detection
Thresholded
output
Strong
classifier
+
…
Feature
output
Adding
features
Final
classification
Strong classifier
at iteration 200
Aprendiendo y reconociendo categorias de objetos
Demo
Demo of screen and car detectors using parts, Gentle boost, and stumps:
> runDetector.m
Aprendiendo y reconociendo categorias de objetos
Probabilistic interpretation
• Generative model
• Discriminative (Boosting) model.
Boosting is fitting an additive logistic regression model:
It can be a set of arbitrary functions of the image
This provides a great flexibility, difficult to beat by current generative
models. But also there is the danger of not understanding what are they
really doing.
Aprendiendo y reconociendo categorias de objetos
Weak detectors
• Generative model
• Discriminative (Boosting) model.
Boosting is fitting an additive logistic regression model:
gi
fi, Pi
Image
Feature
Part template
Relative position
wrt object center
Aprendiendo y reconociendo categorias de objetos
Object models
• Invariance: search strategy
• Part based
gi
fi, Pi
Here, invariance in translation and scale is achieved by the search strategy: the
classifier is evaluated at all locations (by translating the image) and at all scales
(by scaling the image in small steps).
The search cost can be reduced using a cascade.
Aprendiendo y reconociendo categorias de objetos
Cascade of classifiers
Fleuret and Geman 2001, Viola and Jones 2001
100%
Precision
100 features
30 features
3 features
0%
Recall
100%
We want the complexity of the 3 features classifier with the performance of the 100
features classifier:
Select a threshold with high
recall for each stage.
We increase precision using
the cascade
Aprendiendo y reconociendo categorias de objetos
Summary of Class 3
• Object detection with classifiers
• Boosting
–
–
–
–
Gentle boosting
Weak detectors
Object model
Object detection
• Implementation and use