16-824: Visual Learning and Recognition

Fall 2023
[ Home | Schedule | Assignments and Resources | Piazza | Previous Offerings]

Date Topics Course Materials Instructor Deadlines
Week 1
Lecture 1
Monday 08/28/23		 Introduction Slides [CMU only]
No Readings
 Jun-Yan
Lecture 2
Wednesday 08/30/23		 Introduction to Data Slides [CMU only]
Readings:
#1: Alon Halevy, Peter Norvig, and Fernando Pereira
#2: Antonio Torralba & Alexei A. Efros
#3: Chen Sun et al.
#4: Timnit Gebru et al. 		Jun-Yan
Week 2
Lecture 3
Wednesday 09/06/23		 Convolutional Neural Network Slides [CMU only]
Readings:
#1: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
#2: Karen Simonyan, Andrew Zisserman
#3: Kaiming He et.al.
#4: Gao Huang*, Zhuang Liu*, et al.
#5: Jie Hu*, Li Shen* et al. 		Jun-Yan
Week 3
Lecture 4
Monday 09/11/23		 Visualizing and Understanding Neural Networks Slides [CMU only]
Readings:
#1:Aravindh Mahendran, Andrea Vedaldi
#2: David Bau, Bolei Zhou, et al.
#3: Ramprasaath R. Selvaraju et al.
#4: Olah et al.		 Jun-Yan
Bonus Lecture
Tuesday 09/12/23		 AWS and PyTorch Tutorial AWS Tutorial[CMU only]
Pytorch Tutorial[CMU only]		 TAs
Lecture 5
Wednesday 09/13/23		 Attention and Transformers Slides [CMU only]
Readings:
#1: Ashish Vaswani et al.
#2: Ilya Sutskever, Oriol Vinyals, Quoc V. Le
#3: Dzmitry Bahdanau, KyungHyun Cho, Yoshua Bengio
#4: Sepp Hochreiter, et al		 Jun-Yan
Week 4
Lecture 6
Monday 09/18/23		 Vision Transformers Slides [CMU only]
Readings:
#1: Alexey Dosovitskiy et al.
#2: Ze Liu et al.
Optional Readings:
#3: Ilya Tolstikhin et al.
#4: Zhuang Liu et al.
#5: Antoni Buades, Bartomeu Coll, and Jean-Michel Morel 		Jun-Yan HW1 is out
Lecture 7
Wednesday 09/20/23		 Image Segmentation Slides [CMU only]
Readings:
#1: Jonathan Long, Evan Shelhamer, Trevor Darrell
#2: Kaiming He et al.
#3: Olaf Ronneberger et al.
Optional Readings:
#4: Sixiao Zheng et al.
#5: Enze Xie et al.
#6: René Ranftl, Alexey Bochkovskiy, Vladlen Koltun 		Jun-Yan
Week 5
Lecture 8
Monday 09/25/23		 Object Detection (Part I) Slides [CMU only]
Readings:
#1: Pedro F Felzenszwalb et.al.
#2: Ross Girshick et.al.
#3: Joseph Redmon et al.
#4: Nicolas Carion et al.
Optional Readings:
#5: Xingyi Zhou et al. 		Jun-Yan
Lecture 9
Wednesday 09/27/23		 Object Detection (Part II) Slides [CMU only]
No Readings. 		Jun-Yan
Week 6
Lecture 10
Monday 10/02/23		 Guest Lecture: Spatially-aware Robot Learning
No Readings.		 Prof. David Held HW1 due
HW2 out
Lecture 11
Wednesday 10/04/23		 Guest Lecture: Deep Tracking
No Readings.		 Prof. David Held
Week 7
Lecture 12
Monday 10/09/23		 Generative Models (Part I) Slides [CMU only]
Readings:
#1: Alec Radford, Luke Metz, Soumith Chintala
#2: Diederik P Kingma, Max Welling
#3: Aaron van den Oord, et al.
#4: Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio
Optional Readings:
#5: Tero Karras, et al.
#6: Ali Razavi, Aaron van den Oord, Oriol Vinyals 		Jun-Yan
Lecture 13
Wednesday 10/11/23		 Generative Models (Part II) Slides [CMU only]
Readings:
#1: Jonathan Ho, Ajay Jain, Pieter Abbeel
#2: Yang Song et al.
#3: Phillip Isola et al.
Optional Readings:
#4: Jun-Yan Zhu*, Taesung Park* et al.
#5: Yang Song, Diederik P. Kingma 		Jun-Yan Project Proposal Due
Week 8 - Fall Break; No Classes
Week 9
Lecture 14
Monday 10/23/23		 3D Image Understanding (Part I) Slides [CMU only]
Readings:
#1: Derek Hoiem, Alexei A. Efros, Martial Hebert
#2: David Eigen, Rob Fergus
#3: Georgia Gkioxari, Jitendra Malik, Justin Johnson 		Jun-Yan
Lecture 15
Wednesday 10/25/23		 3D Image Understanding (Part II) Slides [CMU only]
Readings:
#1: Charles R. Qi*, Hao Su* et al.
#2: Yue Wang et al.
#3: Jeong Joon Park et al. 		Jun-Yan HW3 out
Week 10
Lecture 16
Monday 10/30/23		 View Synthesis and 3D Generative Models Slides [CMU only]
Readings:
#1: Shenchang Eric Chen
#2: Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, et al.
#3: Katja Schwarz et al.
Optional Readings:
#4: Thomas Müller et al.
#5: Jiatao Gu et al. 		Jun-Yan HW2 due
Lecture 17
Wednesday 11/01/23		 Self-supervised Learning (Part I) Slides [CMU only]
Readings:
#1: Deepak Pathak et al.
#2: Richard Zhang et al.
#3: Kaiming He et al.
Optional Readings:
#4: Jeff Donahue, Karen Simonyan
#5: Mark Chen et al. 		Jun-Yan
Week 11
Lecture 18
Monday 11/06/23		 Self-supervised Learning (Part II) Slides [CMU only]
Readings:
#1: Ting Chen et al.
#2: Kaiming He et al.
#3: Aaron van den Oord, Yazhe Li, Oriol Vinyals
Optional Readings:
#4: Jean-Bastien Grill et al.
#5: Mathilde Caron et al.
#6: Xinlei Chen and Kaiming He 		Jun-Yan
Lecture 19
Wednesday 11/08/23		 Language and Vision (Part I) Slides [CMU only]
Readings:
#1: Kelvin Xu et al.
#2: Stanislaw Antol*, Aishwarya Agrawal*, et al.
#3: Alec Radford et al.
Optional Readings:
#4: Ethan Perez et al.
#5: Junnan Li et al. 		Jun-Yan
Week 12
Lecture 20
Monday 11/13/23		 Language and Vision (Part II) Slides [CMU only]
Readings:
#1: Elman Mansimov et al.
#2: Robin Rombach et al.
#3: Aditya Ramesh et al.
Optional Readings:
#4: Chitwan Saharia et al.
#5: Jiahui Yu et al.
#6: Kang et al. 		Jun-Yan
Lecture 21
Wednesday 11/15/23		 Transfer Learning Slides [CMU only]
Readings:
#1: Jake Snell, Kevin Swersky, Richard S. Zemel
#2: Eric Tzeng et al.
#3: Chelsea Finn et al.
Optional Readings:
#4: Menglin Jia et al.
#5: Amir Bar*, Yossi Gandelsman* et al. 		Jun-Yan
Week 13
Lecture 22
Monday 11/20/23		 Action Recognition and Videos (Part I) Slides [CMU only]
Readings:
#1: Karen Simonyan, Andrew Zisserman
#2: Du Tran et al.
#3: Joao Carreira, Andrew Zisserman
Optional Readings:
#4: Anurag Arnab et al.
#5: Ze Liu et al. 		Jun-Yan HW3 due
Week 14
Lecture 23
Monday 11/27/23		 Action Recognition and Videos (Part II) Slides [CMU only]
No Readings. 		Jun-Yan
Lecture 24
Wednesday 11/29/23		 Efficient Deep Learning Slides [CMU only]
Readings:
#1: Song Han, Huizi Mao, William J Dally
#2: Geoffrey Hinton, Oriol Vinyals, Jeff Dean
#3: Andrew G. Howard, et al.
Optional Readings:
#4: Han Cai et al.
#5: Tri Dao et al.
#6: Daniel Bolya et al. 		Jun-Yan
Week 15
Lecture 25
Wednesday 12/04/23		 Course Project Presentation I Students
Lecture 26
Wednesday 12/06/23		 Course Project Presentation II Students Project presentations are due
Report Week

Friday 12/15/23		 Project final reports are due