Probabilistic Graphical Models for Image Analysis
Fall Semester 2014
This course will focus on inference with statistical models for image analysis. We use a framework called probabilistic graphical models which include Bayesian Networks and Markov Random Fields. We apply the approach to traditional vision problems such as image denoising, as well as recent problems such as object recognition. The course covers amongst others the following topics:
Traditional Supervised Learning:  Logistic and Linear Regression, Naive Bayes 
Directed and undirected Graphical Models:  Markov Random Fields and Bayesian networks 
Inference in Graphical Models:  SumProduct Algorithm, GraphCut, Belief Propagation, Variational Methods and Sampling 
Applications of Graphical Models in vision:  Conditional Random Fields and Structured Output Learning 
News
12/18/2014  Added solutions for SSVM exercises. 
12/17/2014  Added solutions for LBP exercises. 
12/11/2104  Added homework for SSVM lecture. 
12/10/2014  Added SSVM slides and solutions for SVM and CRF lectures. 
12/04/2014  Added LBP exercises and solutions for factored Gaussians example. 
12/03/2014  Added SVM slides and homework for SVM lecture. 
11/26/2014  Added CRF slides and homework for CRF lecture. 
11/17/2014  Added Sampling slides. 
11/05/2014  Added solutions for belief nets and belief prop and reading for Loopy BP. 
10/27/2014  Added homework for belief prop and Variational slides. 
10/22/2014  Added solutions for Holmes/Watson network and another inference exercise. 
10/22/2014  Added solutions for homework 5 and 6. 
10/13/2014  Added solutions for homework 4. 
10/02/2014  Added lecture 3 slides and additional exercises for lecture 1. 
09/23/2014  Added more reading for lecture 1. 
Contact
Lecturer  Dr. Brian McWilliams, Dr. Aurelien Lucchi 
Time and Place
PLEASE NOTE THE NEW TIMES AND LOCATION:
day  time  room 

Monday  15:00  16:00  CAB G 51 
Thursday  10:00  12:00  CLA E 4 
Exam
30 Minute oral exam in English.
Syllabus
Date  Topic  Slides  Additional Exercises  Reading  Background Material 

09/18/14  Introduction/Learning from Data  Lecture 
hw solutions 1 hw solutions 2 hw solutions 3 hw solutions 4 
Barber Ch. 1 
notes on machine learning probability background 
09/22/14  Introduction/Learning from Data (cont.) 
Learning from data basics (solutions) 
Barber Ch. 1 , 8, 13.2 , 17.1, 18.1.1  
09/25/14  Probabilistic models  Lecture 
hw solutions 1 hw solutions 2 
Barber Ch. 8, 10 
Ghahramani on Bayesian modeling Nice example of a generative model 
09/29/14  Probabilistic models  Barber Ch. 17.4, 29.35  
10/02/14  Belief Networks  Lecture 
worked example solutions Inference in Belief nets (solutions) 
Barber Ch. 2, 3 

10/09/14  Markov Random Fields  Lecture 
hw4 solutions 
Barber Ch. 4  
10/16/14  Learning as Inference  Lecture 
hw5 solutions 
Barber Ch. 9  
10/16/14  MAP inference 
Lecture 
hw6 solutions 
Barber Ch. 9, 28.9 
1. energy minimization via graphcuts 2. texture synthesis 3. photomontage 
10/23/14  Belief Propagation  Lecture  Barber Ch. 5  
10/27/14  Belief Propagation (cont.) 
Beliefprop homework (solution) 

10/27/14  Variational Approximation 
Lecture 
Barber Ch. 18.2.2, 28  
11/06/14  Variational Approximation (cont.) 
Lecture 
Additional exercises Solution to factored Gaussians 
Barber Ch. 28  
11/06/14  Loopy Belief Propagation 
Lecture 
LBP exercises (solutions) 
Barber 28.7 Wainwright and Jordan 34.1.6 
Challis and Barber. Gaussian KullbackLeibler Approximate Inference 
11/17/14  Sampling 
Lecture 
Barber Ch. 27  
11/27/14  Conditional Random Fields 
Lecture 
series11.pdf solutions11.pdf hw11 solutions 
Barber 9.6.5 and 23.4.3 
Intro to CRFs Application to image segmentation Learning CRFs with graph cut 
12/01/14  No class  
12/04/14  SVMs 
Lecture 
series12.pdf solutions12.pdf 
SVM tutorial 
Learning the kernel Discriminative MRFs 
12/11/14  Structured SVMs 
Lecture 
series13.pdf solutions13.pdf 

12/15/14  No class 
Schedule
Day  Time  Room  

Lecture  Wednesday  9  11 h  ML F 34 
Exercise  Wednesday  11  12 h  ML F 34 
Resources
Author  Title  Published  Description 

D. Barber  Bayesian Reasoning and Machine Learning  Cambridge University Press 2012  The main course text. Brand new book which covers many topics in graphical models and machine learning. 
M. Wainwright and M.I. Jordan  Graphical models, exponential families and variational inference  Foundations and Trends in Machine Learning 2008  Advanced treatment of graphical models and variational inference 
David J.C. Mackay  Information Theory, Inference and Learning Algorithms  Cambridge University Press, 2003  
C. Bishop  Pattern Recognition and Machine Learning  Springer 2007  This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. 
D. Koller and N. Friedman  Probabilistic Graphical Models: Principles and Techniques  The MIT Press 2009  Covers Bayesian networks and undirected graphical models in great detail. 
Frequently Asked Questions
Question  Answer 

What is a good reference for probability theory required for the course?  See Barber Ch. 1. and MacKay: Ch. 2, 3. Make sure you are comfortable with the exercises in the first week's slides too. 
What is the scope of the course?  We cover material from Part I (all), II and III (some) and V (all) of Barber. We look briefly at the first four sections of Wainwright & Jordan. 