Machine Learning is concerned with building computer programs that automatically improve through experience. This 3-credit course covers introductory-level topics about the theory and practical algorithms for machine learning from a variety of perspectives. Topics include supervised learning (especially modern deep learning), unsupervised learning, learning theory, and RL. (My version will be taught and organized independently from the other session). This Spring, I will focus on deep learning and add many examples of the real-world applications.
Assignments include multiple short programming and writing assignments for hands-on experiments of various machine learning algorithms and multiple in-class quizzess.
You should build up a solid mathematical background. From probability to statistics / From multivariate calculus, to matrix algebra, e.g. VERY comfortable on gradients.
You should build up a general knowledge of machine learning. You don’t need to know every single special algorithms and architecture, but the basics help. You should get comfortable with the main concepts and terminology.
You should become familiar with at least one machine learning and one deep learning library. You should feel pretty confident in implementing some simple do supervised learning algorithms at least.
You should write own implementations. Limited by the time scope of the course, we only have a few implementation finished by the end. But you should implement as many of the learning algorithms from scratch as you can after the course. This is the best way to deepen your understanding of how they work, as well as to develop intuitions for specific performance characteristics.
Required courses as prerequisites: Calculus, Matrix algebra, Probability and Algorithm. Statistics is recommended.
If you are unsure of your math background, please check out the following two review lectures I made:
| Fields | Topics |
|---|---|
| Multivariate Calculus: | - Derivatives (including partial) |
| - gradient, Jacobian, Hessian | |
| Matrix Algebra: | - Rank, Trace, Determinant, Orthonormal, symmetrict, … |
| - Positive Semidefinite, Positive Definite | |
| - Eigen Decomposition, Singular Value Decomposition | |
| Probability: | - Bernoulli, Gaussian, Multinomial |
| - Conditional, Joint, Marginal | |
| - Maximum Likelihood Estimation | |
| Algorithms: | - O(), asymptotic run time / memory complexity |
| - Matrix Computation, Strassen’s | |
| - P / NP … | |
| - Vectorization, Memory Hierarchy |
Course Slack Space: We use Course Slack for office hour assistances and QA discussions on lectures and Quizzes. Please ask all technical questions about the course content, Quiz and homeworks on Course Slack.
We will also use this slack space for QAs on Assignments. This is the place where you can seek help, offer help, share your thoughts and discoveries, or discuss technical difficulties and potential troubleshooting on the assignments.
The grade will be calculated as follows:
Programming solutions should be placed in each student’s appropriate Collab directory.
Deep Learning with Python by Francois Chollet
Deep Learning, An MIT Press book in preparation, Ian Goodfellow, Yoshua Bengio and Aaron Courville
Yaser Abu-Mostafa : Caltech course: Learning from data+ book
The School of Engineering and Applied Science relies upon and cherishes its community of trust. We firmly endorse, uphold, and embrace the University’s Honor principle that students will not lie, cheat, or steal, nor shall they tolerate those who do. We recognize that even one honor infraction can destroy an exemplary reputation that has taken years to build. Acting in a manner consistent with the principles of honor will benefit every member of the community both while enrolled in the Engineering School and in the future.
Students are expected to be familiar with the university honor code, including the section on academic fraud. Each assignment will describe allowed collaborations, and deviations from these will be considered Honor violations. If you are in doubt regarding the requirements, please consult with me before you complete any requirement of this course. Unless otherwise noted, exams and individual assignments will be considered pledged that you have neither given nor received help. (Among other things, this means that you are not allowed to describe problems on an exam to a student who has not taken it yet. You are not allowed to show exam papers to another student or view another student’s exam papers while working on an exam.) Send, receiving or otherwise copying electronic files that are part of course assignments are not allowed collaborations (except for those explicitly allowed in assignment instructions).
Assignments or exams where honor infractions or prohibited collaborations occur will receive a zero grade for that entire assignment or exam, as well as a full letter-grade penalty on the course grade. Such infractions will also be submitted to the Honor Committee if that is appropriate. Students who have had prohibited collaborations may not be allowed to work with partners on remaining homeworks.
The University of Virginia strives to provide accessibility to all students. If you require an accommodation to fully access this course, please contact the Student Disability Access Center (SDAC) at 434-243-5180 or sdac@virginia.edu. If you are unsure if you require an accommodation, or to learn more about their services, you may contact the SDAC at the number above or by visiting their website at URL.
If you have been identified as an SDAC student, please let the Center know you are taking this class. If you suspect you should be an SDAC student, please schedule an appointment with them for an evaluation. Students who need academic accommodations should see me and contact the SDAC. All academic accommodations must be arranged through the SDAC.
If you have other special circumstances (athletics, other university-related activities, etc.) please contact instructor and/or TA as soon as you know these may affect you in class.
It is the University’s long-standing policy and practice to reasonably accommodate students so that they do not experience an adverse academic consequence when sincerely held religious beliefs or observances conflict with academic requirements. Students who wish to request academic accommodation for a religious observance should submit their request in writing directly to me by email as far in advance as possible. Students and instructors who have questions or concerns about academic accommodations for religious observance or religious beliefs may contact the University’s Office for Equal Opportunity and Civil Rights (EOCR) at UVAEOCR@virginia.edu or 434-924-3200.
Accommodations do not relieve you of the responsibility for completion of any part of the coursework missed as the result of a religious observance.
The University of Virginia is dedicated to providing a safe and equitable learning environment for all students. To that end, it is vital that you know two values that I and the University hold as critically important:
Power-based personal violence will not be tolerated. Everyone has a responsibility to do their part to maintain a safe community on Grounds. If you or someone you know has been affected by power-based personal violence, more information can be found on the UVA Sexual Violence website that describes reporting options and resources available - www.virginia.edu/sexualviolence.
As your professor and as a person, know that I care about you and your well-being and stand ready to provide support and resources as I can. As a faculty member, I am a responsible employee, which means that I am required by University policy and federal law to report what you tell me to the University’s Title IX Coordinator. The Title IX Coordinator’s job is to ensure that the reporting student receives the resources and support that they need, while also reviewing the information presented to determine whether further action is necessary to ensure survivor safety and the safety of the University community. If you would rather keep this information confidential, there are Confidential Employees you can talk to on Grounds (See http://www.virginia.edu/justreportit/confidential_resources.pdf). The worst possible situation would be for you or your friend to remain silent when there are so many here willing and able to help.
This syllabus is to be considered a reference document that can and will be adjusted through the course of the semester to address changing needs. This syllabus can be changed at any time without notification. It is up to the student to monitor this page for any changes. Final authority on any decision in this course rests with the professor, not with this document.