16-824: Visual Learning and Recognition

Spring 2024
[ Home | Schedule | Assignments and Resources | Piazza | Previous Offerings]

Date Topics Course Materials Instructor Deadlines
Week 1
Lecture 1
Wednesday 01/17/24		 Introduction No Readings Deepak
Week 2
Lecture 2
Monday 01/22/24		 Theories and History I Slides [CMU only]
Readings:
#1： Lecture 1, 2 (Svetlana Lazebnik's course)
#2: William T. Freeman et.al.		 Deepak
Lecture 3
Wednesday 01/24/24		 Theories and History II Slides [CMU only]
Readings:
Lecture 3, 4 (Svetlana Lazebnik's course)		 Deepak
Week 3
Lecture 4
Monday 01/29/24		 Introduction to Data Slides [CMU only]
Readings:
#1: Alon Halevy, Peter Norvig, Fernando Pereira
#2: Antonio Torralba, Alexei A. Efros
#3: Timnit Gebru et.al.
#4: Sorscher et.al.
#5: Xu et.al.		 Deepak
Lecture 5
Wednesday 01/31/24		 Deep Learning and Convolutional Neural Networks Slides [CMU only]
Readings:
#1: Backpropagation: Olah and 231n
#2: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
#3: Karen Simonyan, Andrew Zisserman
#4: Kaiming He et.al.
#5: Diederik P. Kingma, Jimmy Ba
#6: Gao Huang, Zhuang Liu, et al.
#7: Ioffe et al.
#8: Cubuk et al.
#9: Srivastava et al. 		Deepak
Week 4
Lecture 6
Monday 02/05/24		 Visualizing and Understanding Neural Networks Slides [CMU only]
Readings:
#1:Aravindh Mahendran, Andrea Vedaldi
#2: David Bau, Bolei Zhou, et al.
#3: Ramprasaath R. Selvaraju et al.
#4: Olah et al.
#5: Olah et al. 		Deepak HW1 is out
Bonus Lecture
Tuesday 02/06/24		 AWS and PyTorch Tutorial AWS Tutorial[CMU only]
Pytorch Tutorial[CMU only]		 TAs
Lecture 7
Wednesday 02/07/24		 Attention and Transformers Slides [CMU only]
Readings:
#1: Ashish Vaswani et al.
#2: Ilya Sutskever, Oriol Vinyals, Quoc V. Le
#3: Sepp Hochreiter, et al
#4: Bahdanau, et al
#5: Gu, et al
#6: Peng, et al		 Deepak
Week 5
Lecture 8
Monday 02/12/24		 Vision Transformers Slides [CMU only]
Readings:
#1: Alexey Dosovitskiy et al.
#2: Ze Liu et al.
Optional Readings:
#3: Ilya Tolstikhin et al.
#4: Zhuang Liu et al. 		Deepak
Lecture 9
Wednesday 02/14/24		 Object Detection Slides [CMU only]
Readings:
#1: Pedro F Felzenszwalb et.al.
#2: Ross Girshick et.al.
#3: Joseph Redmon et al.
#4: Nicolas Carion et al.
#5: Dai et al.
#6: Zhou et al.		 Deepak
Week 6
Lecture 10
Monday 02/19/24		 Image Segmentation Slides [CMU only]
Readings:
#1: Jonathan Long, Evan Shelhamer, Trevor Darrell
#2: Kaiming He et al.
#3: Olaf Ronneberger et al.
#4: Cheng et al.
#5: Kirillov et al.		 Deepak
Lecture 11
Wednesday 02/21/24		 Generative Models I Slides [CMU only]
Readings:
#1: Alec Radford, Luke Metz, Soumith Chintala
#2: Diederik P Kingma, Max Welling
#3: Aaron van den Oord, et al.
#4: Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio
Optional Readings:
#5: Tero Karras, et al. 		Deepak HW1 due
HW2 out
Week 7
Lecture 12
Monday 02/26/24		 Generative Models II Slides [CMU only]
Readings:
#1: Jun-Yan Zhu*, Taesung Park* et al.
#2: Aditya Ramesh et al.
#3: Phillip Isola et al.		 Deepak
Lecture 13
Wednesday 02/28/24		 Generative Models III Slides [CMU only]
Readings:
#1: Ho et al.
#2: Nichol et al.
#3: Yang Song et al. 		Deepak

Friday 03/01/24
Week 8 - Spring Break; No Classes
Week 9
Lecture 14
Monday 03/11/24		 Efficient Deep Learning Slides [CMU only]
Readings:
#1: Song Han, Huizi Mao, William J Dally
#2: Geoffrey Hinton, Oriol Vinyals, Jeff Dean
#3: Andrew G. Howard, et al.
#4: Noam Shazeer, et al.
#5: Edward J. Hu, et al.
#6: Dettmers, et al.
Optional Readings:
#7: Han Cai et al. 		Deepak Project Proposal Due
Lecture 15
Wednesday 03/13/24		 Few-Shot and Transfer Learning Slides [CMU only]
Readings:
#1: Jeff Donahue et al.
#2: Eric Tzeng et al.
#3: Chelsea Finn et al.
#4: Yutong Bai et al. 		Deepak
Week 10
Lecture 16
Monday 03/18/24		 Self-supervised Learning I Slides [CMU only]
Readings:
#1: Deepak Pathak et al.
#2: Richard Zhang et al.
#3: Jeff Donahue, Karen Simonyan		 Deepak
Lecture 17
Wednesday 03/20/24		 Self-supervised Learning II Slides [CMU only]
Readings:
#1: Ting Chen et al.
#2: Kaiming He et al.
#3: Aaron van den Oord, Yazhe Li, Oriol Vinyals		 Deepak

Friday 03/22/24		 HW2 due
HW3 out
Week 11
Lecture 18
Monday 03/25/24		 3D Image Understanding I Slides [CMU only]
Readings:
#1: Derek Hoiem, Alexei A. Efros, Martial Hebert
#2: David Eigen, Rob Fergus
#3: Georgia Gkioxari, Jitendra Malik, Justin Johnson		 Deepak
Lecture 19
Wednesday 03/27/24		 3D Image Understanding II Slides [CMU only]
Readings:
#1: Charles R. Qi*, Hao Su* et al.
#2: Yue Wang et al.
#3: Jeong Joon Park et al.		 Deepak
Week 12
Lecture 20
Monday 04/01/24		 3D Image Understanding III Slides [CMU only]
Readings:
#1: Kanazawa et al.
#2: Hu et al.
#3: Mildenhall et al.
#4: Poole et al.
#5: Bernhard Kerbl et al. 		Deepak
Lecture 21
Wednesday 04/03/24		 Action Recognition and Videos Slides [CMU only]
Readings:
#1: Karen Simonyan, Andrew Zisserman
#2: Du Tran et al.
#3: Joao Carreira, Andrew Zisserman		 Deepak

Friday 04/05/24		 Mid-term project update due
Week 13
Lecture 22
Monday 04/08/24		 Mulitmodal Perception (Language/Sound/Touch/Action) Slides [CMU only]
Readings:
#1: Kelvin Xu et al.
#2: Stanislaw Antol*, Aishwarya Agrawal*, et al.
#3: Andrew Owens et al.
#4: Roberto Calandra et al. 		Deepak
Lecture 23
Wednesday 04/10/24		 Towards Generalist Machines and Conclusion Slides [CMU only] Deepak HW3 due
Week 14
Lecture 24
Monday 04/15/24		 Guest Lecture: Future of Action and Vision Slides [CMU only] Shikhar Bahl and Alex Li
Lecture 25
Wednesday 04/17/24		 Guest Lecture: Mamba, State Space Models Slides [CMU only] Albert Gu
Week 15
Lecture 26
Monday 04/22/24		 Course Project Presentation I Students Project presentations are due
Lecture 27
Wednesday 04/24/24		 Course Project Presentation II Students Project presentations are due
Report Week

Friday 05/03/24		 Project final reports are due