Course Information

Time and Location:

Instructor: Chandan Reddy

Teaching Assistant: Tian Shi

Course Email: vtreddycourse@gmail.com

Course Description:

Deep Learning has gained a lot of popularity due to its recent breakthrough results in many real-world applications such as speech recognition, machine translation, image understanding, and robotics. The primary idea of deep learning is to build high-level abstractions of the data through multi-layered architectures. This course introduces the fundamental principles, algorithms and applications of deep learning. It will provide an in-depth understanding of various concepts and popular techniques in deep learning. This course is mainly designed for graduate students who are interested in studying deep learning techniques and their practical applications. Basic knowledge and understanding of machine learning and data mining algorithms is required.

The course begins with a thorough treatment of deep feedforward networks along with various regularization and optimization techniques used for efficiently learning these models. Different forms of the network architectures such as convolutional networks, recurrent neural networks and autoencoders will be discussed in detail. Other advanced concepts such as deep generative models and deep reinforcement learning will also be covered. Finally, the course will conclude with a discussion on few real-world application domains where deep learning techniques have produced astonishing results.

Prerequisites:

• CS5525 Data Analytics I (or) CS5824 Advanced Machine Learning.

Books: The material for this course will be adapted from a wide range of sources. While there is no single textbook that will be used in this course, the students might find the following books to be useful.

• Charu C. Aggarwal, "Neural Networks and Deep Learning: A Textbook", Springer, September 2018.
• Ian Goodfellow, Yoshua Bengio, and Aaron Courville, "Deep learning", MIT Press, 2016.
• Simon O. Haykin, "Neural Networks and Learning Machines", Pearson Education, 3rd Edition, 2008.

Homework Assignments:

There will be four written homework assignments. Homework problems will constitute some programming exercises that are designed to understand the working of deep learning algorithms. Students are encouraged to talk and discuss with other students to improve their conceptual understanding, but the final submission must be their own work. If any help is taken from others, please acknowledge the people from whom you received some help. Any homework turned in late will incur some penalty for each late day.

Quizzes:

There will be five in-class quizzes. The best four out of the five will be considered for the final grade in the course. Each quiz will be held in-class and will be held for 15-20 minutes. Each quiz will contain some basic questions about the course topics. The questions can be in the form of fill-in the blanks, multiple choice or simple calculations. Detailed algorithms or coding questions will not be present in the quiz.

Paper Readings:

The students will need to write rigorous critiques for two research papers from the list provided under the "Readings" section on the course website. Each critique must be around 2 pages long (single-spaced) and should clearly highlight the problem motivation, background, the main intuition and algorithmic insights along with the main results. The critique should provide own thoughts as opposed to merely reproducing the paper's content. One paper should be selected from the methodologies section and the other one from the applications section. The papers related to the student's course project are strongly recommended.

Final Project:

One of the major components of this course is the final project. In this project, students will investigate some interesting aspect of deep learning and apply it to a real-world problem. The main purpose of this project is to enable the students to get some hands-on experience in the design and implementation of a practical deep learning system. In addition to the core algorithmic aspects, the performance of the system significantly depends on some specific domain-dependent expert knowledge in the application field (such as business intelligence, imaging, robotics, text analysis, e-commerce, etc.). More details about the project proposal and project submission will be provided in early October.

Grading Policies:

Final grades are based on the performance in homeworks, final exam and the course project. Here is the distribution.

Accommodations Statement:

Honor Code Statement: