16-824: Visual Learning and Recognition

Fall 2022
[ Home | Schedule | Assignments and Resources | Piazza]

Date Topics Course Materials Instructor Deadlines
Week 1
Lecture 1
Mon 08/29/22		 Introduction Slides [CMU only]
No Readings		 Jun-Yan
Lecture 2
Wed 08/31/22		 Introduction to Data Slides [CMU only]
Readings:
#1: Alon Halevy, Peter Norvig, and Fernando Pereira
#2: Antonio Torralba & Alexei A. Efros
#3: Chen Sun et al.
#4: Timnit Gebru et al. 		Jun-Yan
Week 2
Lecture 3
Wed 09/07/22		 Convolutional Neural Networks Slides [CMU only]
Readings:
#1: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
#2: Karen Simonyan, Andrew Zisserman
#3: Kaiming He et al.
#4: Gao Huang*, Zhuang Liu*, et al.
#5: Jie Hu*, Li Shen* et al.		 Jun-Yan
Week 3
Lecture 4
Mon 09/12/22		 Visualizing and Understanding Neural Networks Slides [CMU only]
Readings:
#1: Aravindh Mahendran, Andrea Vedaldi
#2: David Bau, Bolei Zhou et al.
#3: Ramprasaath R. Selvaraju et al.
#4: Olah et al.		 Jun-Yan
Bonus Lecture
Tue 09/13/22		 PyTorch and AWS Tutorial Slides [See Piazza]
No Readings
TAs
Lecture 5
Wed 09/14/22		 Attention and Transformers Slides [CMU only]
Readings:
#1: Ashish Vaswani et al.
#2: Ilya Sutskever, Oriol Vinyals, Quoc V. Le
Optional Readings:
#3: Sepp Hochreiter, et al 		Jun-Yan
Week 4
Lecture 6
Mon 09/19/22		 Vision Transformers Slides [CMU only]
Readings:
#1: Alexey Dosovitskiy et al.
#2: Ze Liu et al.
Optional Readings:
#3: Ilya Tolstikhin et al.
#4: Zhuang Liu et al. 		Jun-Yan HW1 is out
Lecture 7
Wed 09/21/22		 Image Segmentation Slides [CMU only]
Readings:
#1: Jonathan Long, Evan Shelhamer, Trevor Darrell
#2: Kaiming He et al.
#3: Olaf Ronneberger et al.
Optional Readings:
#4: René Ranftl, Alexey Bochkovskiy, Vladlen Koltun 		Jun-Yan
Week 5
Lecture 8
Mon 09/26/22		 Object Detection Slides [CMU only]
Readings:
#1: Pedro F Felzenszwalb et al.
#2: Ross Girshick et al.
#3: Joseph Redmon et al.
#4: Nicolas Carion et al.
Optional Readings:
#5: Xingyi Zhou et al. 		Jun-Yan
Lecture 9
Wed 09/28/22		 3D Image Understanding I Slides [CMU only]
Readings:
#1: Derek Hoiem, Alexei A. Efros, Martial Hebert
#2: David Eigen, Rob Fergus
#3: Georgia Gkioxari, Jitendra Malik, Justin Johnson 		Jun-Yan
Week 6
Lecture 10
Mon 10/03/22		 3D Image Understanding II Slides [CMU only]
Readings:
#1: Charles R. Qi*, Hao Su* et al.
#2: Yue Wang et al.
#3: Jeong Joon Park et al. 		Jun-Yan HW1 Due
Lecture 11
Wed 10/05/22		 Generative Models I Slides [CMU only]
Readings:
#1: Alec Radford, Luke Metz, Soumith Chintala
#2: Diederik P Kingma, Max Welling
#3: Aaron van den Oord, et al.
#4: Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio
Optional Readings:
#5: Tero Karras, et al.
#6: Mark Chen, et al. 		Jun-Yan HW2 is out
Week 7
Lecture 12
Mon 10/10/22		 Generative Models II Slides [CMU only]
Readings:
#1: Joshua B. Tenenbaum, William T. Freeman
#2: Jonathan Ho, Ajay Jain, Pieter Abbeel
#3: Phillip Isola et al.
#4: Jun-Yan Zhu*, Taesung Park* et al.
Optional Readings:
#5: Yang Song
#6: Chenlin Meng et al. 		Jun-Yan
Lecture 13
Wed 10/12/22		 View Synthesis - 3D Generative Models Slides [CMU only]
Readings:
#1: Shenchang Eric Chen
#2: Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, et al.
#3: Katja Schwarz et al.
Optional Readings:
#4: Jonathan T. Barron et al.
#5: Jiatao Gu et al. 		Jun-Yan
Week 8 (Fall break)
Week 9
Lecture 14
Mon 10/24/22		 Efficient Deep Learning Slides [CMU only]
Readings:
#1: Song Han, Huizi Mao, William J Dally
#2: Geoffrey Hinton, Oriol Vinyals, Jeff Dean
#3: Andrew G. Howard, et al.
Optional Readings:
#4: Han Cai et al. 		Jun-Yan
Lecture 15
Wed 10/26/22		 Guest lecture: From 2D to 3D Visual Recognition: New Benchmarks, Models and Methods for 3D in the Wild Slides [CMU only]
No Readings 		Georgia Gkioxari
Fri 10/28/22
HW2 Due
Week 10
Lecture 16
Mon 10/31/22		 Action Recognition and Videos Slides [CMU only]
Readings:
#1: Karen Simonyan, Andrew Zisserman
#2: Du Tran et al.
#3: Joao Carreira, Andrew Zisserman
Optional Readings:
#4: Rohit Girdhar et al.
#5: Chao-Yuan Wu et al. 		Jun-Yan HW3 is out
Lecture 17
Wed 11/02/22		 Few-Shot and Zero-Shot Learning Slides [CMU only]
Readings:
#1: Jake Snell, Kevin Swersky, Richard S. Zemel
#2: Eric Tzeng et al.
#3: Chelsea Finn et al.
Optional Readings:
#4: Amir Bar*, Yossi Gandelsman* et al. 		Jun-Yan
Week 11
Lecture 18
Mon 11/07/22		 Self-supervised Learning I Slides [CMU only]
Readings:
#1: Deepak Pathak et al.
#2: Richard Zhang et al.
#3: Jeff Donahue, Karen Simonyan
Optional Readings:
#4: Kaiming He et al.
#5: Mark Chen et al. 		Jun-Yan
Lecture 19
Wed 11/09/22		 Self-supervised Learning II Slides [CMU only]
Readings:
#1: Ting Chen et al.
#2: Kaiming He et al.
#3: Aaron van den Oord, Yazhe Li, Oriol Vinyals
Optional Readings:
#4: Yonglong Tian et al.
#5: Xinlei Chen et al. 		Jun-Yan
Week 12
Lecture 20
Mon 11/14/22		 Language and Vision I Slides [CMU only]
Readings:
#1: Kelvin Xu et al.
#2: Stanislaw Antol*, Aishwarya Agrawal*, et al.
#3: Alec Radford et al.
Optional Readings:
#4: Ethan Perez et al. 		Jun-Yan
Lecture 21
Wed 11/16/22		 Guest lecture: Transformer-based reasoning across space and time Slides [CMU only]
No Readings 		Philipp Krähenbühl HW3 Due
Week 13
Lecture 22
Mon 11/21/22		 Language and Vision II Slides [CMU only]
Readings:
#1: Elman Mansimov et al.
#2: Robin Rombach et al.
#3: Aditya Ramesh et al.
Optional Readings:
#4: Han Zhang et al.
#5: Chitwan Saharia et al.
#6: Jiahui Yu et al. 		Jun-Yan
Week 14
Lecture 23
Mon 11/28/22		 Multimodal Perception (Sound/Touch/Action) Slides [CMU only]
Readings:
#1: Andrew Owens et al.
#2: Hang Zhao et al.
#3: Roberto Calandra et al. 		Jun-Yan
Lecture 24
Wed 11/30/22		 Towards Generalist Machines and Conclusion Slides [CMU only]
No Readings 		Jun-Yan
Week 15
Lecture 25
Mon 12/05/22		 Course Project Presentation I Students Project presentations are due
Lecture 26
Wed 12/07/22		 Course Project Presentation II Students Project presentations are due
Report Week

Fri 12/16/22		 Project final reports are due