Large Scale Data Analysis Using Deep Learning
Course Introduction U Kang
Seoul National University
U Kang 2
In This Lecture
Motivation to study deep learning
Administrative information for this course
Outline
Deep Learning
Course Information
U Kang 4
Deep Learning as a Machine Learning
Machine Learning (ML)
Given x (predictor) and y (response), ML learns a function f() from data, such that y = f(x)
E.g., x = image, y = category
This learned function f() can be used to classify a new example x’
This is different from a typical programming where you want to compute y, given x and f()
Deep learning provides good performances in learning f() for many problems
Learns non-linear functions
Deep Learning as a Machine Learning
Data Size
Accuracy Deep Learning
Other machine learning
U Kang 6
Learning Tasks
Image classification
Speech recognition
Text classification
…
“Taxi”
Hello, dear
International politics
Main Idea
Most perception (input processing) in the brain may use one learning algorithm
Design learning methods that mimic the
brain
U Kang 8
Neurons In the Brain
Neural Network
[LeCun et al., Nature 2015]
U Kang 10
Convolutional Neural Net (CNN)
Representation Learning
Typical machine learning
Deep learning
Input Extract Output
Features
x y
Input Output
x y
Extract Features
Extract Features
Extract Features
Classifier
… Classifier
U Kang 12
Human Level Object Recognition
ImageNet
~ 15M labeled images, ~ 22K categories
Top-5 Error rates (1.2 million images, 1k categories)
Non-CNN based method (~2012): 26.2 %
Alexnet (2012): 15.3 %
GoogLeNet (2014): 6.66 %
Resnet (2015): 3.57%
Human level:
5.1% error
Human-Level Face Recognition
DeepFace
97% accuracy
~ Human-level
[Taigman et al.
CVPR 2014]
U Kang 14
Computer Game
Deepmind
Computer Game
Deepmind
[Nature 2015]https://www.youtube.com/watch?v=V1eYniJ0Rnk
U Kang 16
AlphaGo
[Silver et al., Mastering the game of Go with
deep neural networks and tree search, Nature 2016]
Neural Artist
[Gatys et al., Image Style Transfer Using Neural Networks, CVPR 2016]
U Kang 18
Machine Translation
Topics in This Course (tentative)
(ch. 1) Introduction
(ch. 2) Linear Algebra
(ch. 3) Probability and Information Theory
(ch. 4) Numerical Computation
(ch. 5) Machine Learning Basics
(ch. 6) Deep Feedforward Networks
(ch. 7) Regularization for Deep Learning
(ch. 8) Optimization for Training Deep Models
(ch. 9) Convolutional Networks
(ch. 10) Sequence Modeling: Recurrent and Recursive Nets
(ch. 11) Practical Methodology
(ch. 12) Applications
U Kang 20
Outline
Deep Learning
Course Information
M1522.001600, Spring 2017
http://datalab.snu.ac.kr/~ukang/courses/17S-DL/
Lecture slide: at least 1 hour before the lecture
TA
Beunguk Ahn (beunguk.ahn@gmail.com, 301-519)
Office hour
(Prof) Mon. 11:00 – 12:00
(TA) Tue. 14:00 – 15:00
Class meets: Mon, Wed 14:00 – 15:15, 301-101
U Kang 22
Textbook
Deep Learning (Ian Goodfellow, Yoshua Bengio, and Aaron Courville)
Available at http://www.deeplearningbook.org
Prerequisites
Basic probability
Average, std. deviation, typical distributions, MLE, …
Basic linear algebra
Rank, singular value decomposition
Programming language
Python, (C++, Java)..
Machine Learning or Artificial Intelligence
Basic understandings of machine learning
U Kang 24
Grading
10% Attendance and Quiz (random)
30% Project
30% Midterm
30% Final
(+5% Participation)
Late Policy
For all deliverables (homework, code, …)
No delay penalties, for medical etc. emergencies (bring doctor's note)
Each person has 4 'slip days' total, for the whole
semester. 10% per day of delay, after that
U Kang 26
Project
A good opportunity to solve real world problems using deep learning
Team
A group of 3-4 persons
If you cannot find your team mate, discuss with
the TA and/or instructor
Project
Topic
Improve the current status-of-the-art in deep learning
Deep learning applications
E.g., Novel applications
Deep learning implementations
E.g., Fast implementations using GPU
Method of deep learning
E.g., New regularization method
E.g., Achieve the best score in object recognition competition
We will provide some candidates, but feel free to propose your own topic
Feel free to discuss with the TA and/or the instructor
before the proposal
U Kang 28
Project
Advice
Start your project from day 1: today!
Think of the data you are interested in (e.g. image, audio, text, graph, etc.), and how to get it
Read related papers
Find your team mates
Data should be ready very soon
In the worst case, it should be ready until the end of March
If you plan to collect the data later, you might not get it until the semester ends
Project
Schedule and grading
Project proposal (due April 3) : 10%
Progress report (due May 3): 20%
Final report and presentation (due June 7) : 70%
U Kang 30
Advice
Read each chapter before class
It is ok to encounter something you don’t understand. Just mark it, and later you will understand it when you come back.
“Understand” intuitions of main ideas
Do not memorize without understanding
Improve your problem solving skills
Active participation encouraged
All questions are right; ask many questions
Use office hours (instructor and/or TA)
