Epilogue: what we covered and didn`t cover in this course

12/5/13
What I hope you got out of this course
Epilogue: what we covered and didn’t
cover in this course
The machine learning toolbox
 
 
 
ʻViagraʼ
=0
ʻlotteryʼ
=0
ĉ(x) = ham
 
=1
ĉ(x) = spam
Formulating a problem as an ML problem
Understanding a variety of ML algorithms
Running and interpreting ML experiments
Understanding what makes ML work – theory and
practice
=1
ĉ(x) = spam
16
14
12
10
8
6
4
2
0
2
4
6
8
10
12
14
1
Learning scenarios we covered
Types of models
Classification: discrete/categorical labels
Geometric
q 
Ridge-regression, SVM, perceptron
w
wTx + b > 0
wTx + b < 0
Distance-based
Regression: continuous labels
q 
Clustering: no labels
Probabilistic
16
q 
K-nearest-neighbors
Naïve-bayes
14
12
P (Y = spam|Viagara, lottery)
Logical models: Tree/Rule based
10
q 
8
Decision trees
ʻViagraʼ
=0
ʻlotteryʼ
=0
6
4
2
0
2
4
6
8
10
12
14
3
Ensembles
ĉ(x) = ham
=1
ĉ(x) = spam
=1
ĉ(x) = spam
4
1
12/5/13
Structured Prediction
Types of learning tasks
Handle prediction problems with complex output
spaces
Supervised learning
Unsupervised learning
Semi-supervised learning
Access to a lot of unlabeled data
Reinforcement learning
Learn action to maximize payoff
 
Structured outputs: multivariate, correlated,
constrained
 
 
–  Payoff
is often delayed
Multi-label classification
Structured output learning
Novel, general way to solve many learning problems
Examples taken from Ben Taskar’s 07 NIPS tutorial
Handwriting Recognition
x
Natural Language Parsing
y
brace
Sequential structure
x
x
y
The dog chased the cat
y
S
NP
VP
NP
Det
N
V
Det
N
Recursive structure
2
12/5/13
Bilingual Word Alignment
x
y
What is the anticipated
cost of collecting fees
under the new proposal?
En vertu des nouvelles
propositions, quel est le
coût prévu de perception
des droits?
What
is
the
anticipated
cost
of
collecting
fees
under
the
new
proposal
?
Combinatorial structure
Local vs. Global
En
vertu
de
les
nouvelles
propositions
,
quel
est
le
coût
prévu
de
perception
de
les
droits
?
Global classification takes advantage
of correlations and satisfies the constraints
in the problem
brace
GO term prediction
Types of learning tasks
Supervised learning
Unsupervised learning
Semi-supervised learning
Access to a lot of unlabeled data
Reinforcement learning
Learn action to maximize payoff
 
 
–  Payoff
Hierarchical structure
is often delayed
Multi-label classification
Structured output learning
Multi-task learning
3
12/5/13
Other techniques
Probabilistic modeling
Bayesian learning
13
4