16-824: Visual Learning and Recognition

Fall 2024
[ Home | Schedule | Assignments and Resources | Piazza | Previous Offerings]

Date Topics Course Materials Instructor Deadlines
Week 1
Lecture 1
Monday 08/26/24		 Introduction Slides [CMU only]
No Readings
 Jun-Yan
Lecture 2
Wednesday 08/28/24		 Introduction to Data Slides [CMU only]
Readings:
#1: Alon Halevy, Peter Norvig, and Fernando Pereira
#2: Antonio Torralba & Alexei A. Efros
#3: Chen Sun et al.
#4: Timnit Gebru et al. 		Jun-Yan
Week 2
Bonus Lecture
Wednesday 09/04/24		 AWS and PyTorch Tutorial AWS Tutorial[CMU only]
Pytorch Tutorial[CMU only]		 TAs
Week 3
Lecture 3
Monday 09/09/24		 Convolutional Neural Network Slides [CMU only]
Readings:
#1: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
#2: Karen Simonyan, Andrew Zisserman
#3: Kaiming He et.al.
#4: Gao Huang*, Zhuang Liu*, et al.
#5: Zhuang Liu et al. 		Jun-Yan
Lecture 4
Wednesday 09/11/24		 Visualizing and Understanding Neural Networks Slides [CMU only]
Readings:
#1:Aravindh Mahendran, Andrea Vedaldi
#2: David Bau, Bolei Zhou, et al.
#3: Ramprasaath R. Selvaraju et al.
#4: Olah et al.		 Jun-Yan
Week 4
Lecture 5
Monday 09/16/24		 Attention and Transformers Slides [CMU only]
Readings:
#1: Ashish Vaswani et al.
#2: Ilya Sutskever, Oriol Vinyals, Quoc V. Le
#3: Dzmitry Bahdanau, KyungHyun Cho, Yoshua Bengio
Optional Readings:
#4: Sepp Hochreiter, et al		 Jun-Yan
Lecture 6
Wednesday 09/18/24		 Vision Transformers Slides [CMU only]
Readings:
#1: Alexey Dosovitskiy et al.
#2: Ze Liu et al.
#3: Andrew Jaegle et al.
Optional Readings:
#4: Ilya Tolstikhin et al. 		Jun-Yan HW1 is out
Week 5
Lecture 7
Monday 09/23/24		 Segmentation and Detection I Slides [CMU only]
Readings:
#1: Jonathan Long, Evan Shelhamer, Trevor Darrell
#2: Olaf Ronneberger et al.
Optional Readings:
#3: Sixiao Zheng et al.
#4: Enze Xie et al. 		Jun-Yan
Lecture 8
Wednesday 09/25/24		 Segmentation and Detection II Slides [CMU only]
Readings:
#1: Pedro F Felzenszwalb et.al.
#2: Ross Girshick et.al.
#3: Joseph Redmon et al. 		Jun-Yan
Week 6
Lecture 9
Monday 09/30/24		 Segmentation and Detection III Slides [CMU only]
Readings:
#1: Nicolas Carion et al.
#2: Kaiming He et al.
#3: Alexander Kirillov et al.
Optional Readings:
#4: Bowen Cheng et al. 		Jun-Yan
Lecture 10
Wednesday 10/02/24		 Generative Models I Slides [CMU only]
Readings:
#1: Ian J. Goodfellow et al.
#2: Alec Radford, Luke Metz, Soumith Chintala
#3: Phillip Isola et al.
Optional Readings:
#4: Tero Karras, et al.
#5: Jun-Yan Zhu*, Taesung Park* et al. 		Jun-Yan HW1 due
HW2 out
Week 7
Lecture 11
Monday 10/07/24		 Generative Models II Slides [CMU only]
Readings:
#1: Diederik P Kingma, Max Welling
#2: Aaron van den Oord et al.
#3: Patrick Esser et al.
Optional Readings:
#4: Diederik P. Kingma, Max Welling
#5: Aaron van den Oord et al. 		Jun-Yan
Lecture 12
Wednesday 10/09/24		 Generative Models III Slides [CMU only]
Readings:
#1: Jonathan Ho, Ajay Jain, Pieter Abbeel
#2: Yang Song et al.
Optional Readings:
#3: Yang Song, Diederik P. Kingma
#4: J Sohl-Dickstein et al. 		Jun-Yan
Week 8 - Fall Break; No Classes
Week 9
Lecture 13
Monday 10/21/24		 Self-Supervised Learning I Slides [CMU only]
Readings:
#1: Deepak Pathak et al.
#2: Richard Zhang et al.
#3: Kaiming He et al.
Optional Readings:
#4: Jeff Donahue, Karen Simonyan
#5: Mark Chen et al. 		Jun-Yan
Lecture 14
Wednesday 10/23/24		 Self-Supervised Learning II Slides [CMU only]
Readings:
#1: Ting Chen et al.
#2: Kaiming He et al.
#3: Aaron van den Oord, Yazhe Li, Oriol Vinyals
Optional Readings:
#4: Jean-Bastien Grill et al.
#5: Xinlei Chen and Kaiming He 		Jun-Yan HW2 due Oct 26
Week 10
Lecture 15
Monday 10/28/24		 Language and Vision I Slides [CMU only]
Readings:
#1: Stanislaw Antol*, Aishwarya Agrawal*, et al.
#2: Alec Radford et al.
#3: Haotian Liu et al.
Optional Readings:
#4: Junnan Li et al. 		Jun-Yan
Lecture 16
Wednesday 10/30/24		 Language and Vision II Slides [CMU only]
Readings:
#1: Elman Mansimov et al.
#2: Robin Rombach et al.
#3: Aditya Ramesh et al.
Optional Readings:
#4: Chitwan Saharia et al.
#5: Jiahui Yu et al.
#6: Kang et al. 		Jun-Yan
Week 11
Lecture 17
Monday 11/04/24		 Guest Lecture: 3D Image Understanding I
No Readings.		 Prof. Shubham Tulsiani
Lecture 18
Wednesday 11/06/24		 Guest Lecture: 3D Image Understanding II
No Readings.		 Prof. Shubham Tulsiani
Week 12
Lecture 19
Monday 11/11/24		 Transfer Learning, Visual Prompting I Slides [CMU only]
Readings:
#1: Jake Snell, Kevin Swersky, Richard S. Zemel
#2: Eric Tzeng et al.
#3: Chelsea Finn et al. 		Jun-Yan
Lecture 20
Wednesday 11/13/24		 Transfer Learning, Visual Prompting II Slides [CMU only]
Readings:
#1: Menglin Jia et al.
#2: Amir Bar*, Yossi Gandelsman* et al.
#3: Hyojin Bahng et al. 		Jun-Yan HW3 due Nov 15
Week 13
Lecture 21
Monday 11/18/24		 Action Recognition and Videos I Slides [CMU only]
Readings:
#1: Karen Simonyan, Andrew Zisserman
#2: Du Tran et al.
#3: Joao Carreira, Andrew Zisserman 		Jun-Yan
Lecture 22
Wednesday 11/20/24		 Action Recognition and Videos II Slides [CMU only]
Readings:
#1: Anurag Arnab et al.
#2: Ze Liu et al.
#3: Chen Sun et al.
Optional Readings:
#4: Zhan Tong et al. 		Jun-Yan
Week 14
Lecture 23
Monday 11/25/24		 Efficient Deep Learning Slides [CMU only]
Readings:
#1: Song Han, Huizi Mao, William J Dally
#2: Geoffrey Hinton, Oriol Vinyals, Jeff Dean
#3: Andrew G. Howard, et al.
Optional Readings:
#4: Han Cai et al.
#5: Tri Dao et al.
#6: Daniel Bolya et al. 		Jun-Yan
Week 15
Lecture 24
Monday 12/02/24		 Course Project Presentation I Students Project presentation slide due
Lecture 25
Wednesday 12/04/24		 Course Project Presentation II Students Project presentation slide due
Report Week

Wednesday 12/11/24		 Project final report due