O'Reilly logo
live online training icon Live Online training

Deep Learning from Scratch

Understanding deep learning foundations for data science

Bruno Gonçalves

Over the past few years we have seen a convergence of two large-scale trends: Big Data and Big Compute. The resulting combination of large amounts of data and abundant CPU (and GPU) cycles has brought to the forefront and highlighted the power of neural network techniques and approaches that were once thought to be too impractical.

Deep Learning, as this new wave of interest has come to be known, has made impressive and unprecedented progress on applications as diverse as Natural Language Processing, Machine Translation, Computer Vision, Robotics, etc. In this lecture, students will learn, in a hands-on way, the theoretical foundations and principal ideas underlying this burgeoning field. The code structure of the implementations provided is meant to closely resemble the way the state of the art deep learning libraries Keras is structured so that by the end of the course, students will be prepared to dive deeper into the deep learning applications of their choice.

What you'll learn-and how you can apply it

  • Linear regression
  • Logistic regression
  • Perceptrons and neurons
  • Back propagation
  • Auto-encoders
  • Training deep networks

This training course is for you because...

  • You are a data scientist who is interested in mastering the concepts and ideas behind neural networks and deep learning.
  • You may have either no previous experience in neural networks or machine learning, and want to take the first grounded steps, or you may have previous experience in using neural network libraries such as Keras or Tensorflow, and wish to get a greater understanding of what’s going on “under the hood.”

Prerequisites

Attendees should have experience with: - Basic Python - Numpy - Matplotlib - Jupyter

Course Set-up

  • Scientific Python distribution like Anaconda

Recommended Preparation

If you need to brush up on Python: - (video) Python Programming Language LiveLessons by David Beazley: - (video) Modern Python LiveLessons: Big Ideas and Little Code in Python by Raymond Hettinger

Recommended Follow-up

After you complete the Deep Learning from Scratch Live Online Training class, you may find the following resources helpful: - (live online training) Deep Learning for NLP by Jon Krohn (search the O'Reilly Learning Platform for an upcoming class) - (video) Deep Reinforcement Learning and GANs: Advanced Topics in Deep Learning - (video) Understanding Convolutional Neural Networks - (book) Learn Keras for Deep Neural Networks: A Fast-Track Approach to Modern Deep Learning with Python - (video) Natural Language Processing LiveLessons by Bruno Gonçalves

About your instructor

  • Bruno Gonçalves is currently a Senior Data Scientist working at the intersection of Data Science and Finance. Previously, he was a Data Science fellow at NYU's Center for Data Science while on leave from a tenured faculty position at Aix-Marseille Université. Since completing his PhD in the Physics of Complex Systems in 2008 he has been pursuing the use of Data Science and Machine Learning to study Human Behavior. Using large datasets from Twitter, Wikipedia, web access logs, and Yahoo! Meme he studied how we can observe both large scale and individual human behavior in an obtrusive and widespread manner. The main applications have been to the study of Computational Linguistics, Information Diffusion, Behavioral Change and Epidemic Spreading. In 2015 he was awarded the Complex Systems Society's 2015 Junior Scientific Award for "outstanding contributions in Complex Systems Science" and in 2018 is was named a Science Fellow of the Institute for Scientific Interchange in Turin, Italy.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Segment 1 - Regression (30 min)

  • Understand the different types of Machine Learning
  • Define Supervised Learning and Regression
  • Visualize Linear Regression
  • Implement Gradient Descent
  • Use Linear Regression

Break (10 min)

Segment 2 - Classification (40 min)

  • Define Classification
  • Visualize Logistic Regression
  • Learning Procedure
  • Implement Logistic Regression
  • Understand Classification evaluation

Break (10min)

Segment 3 – Data Preparation (20 min)

  • Visualize decision boundaries
  • Use Data Normalization
  • Understand Overfitting
  • Define the Bias Variance Tradeoff

Break (10min)

Segment 4 - Neural Networks (30 min)

  • Understand how the brain works
  • Understand Perceptrons and Forward Propagation
  • Define Activation Functions
  • Understand Back Propagation

Break (10 min)

Segment 5 – Recognizing Numbers (50 min)

  • Understand the MNIST Dataset
  • Understand Data Preparation
  • Implement Back Propagation
  • Understand the effect of the Learning Rate
  • Generalize the Code

Break (10 min)

Segment 6 – Advanced network applications (50 min)

  • Understand unsupervised Learning
  • Implement an Auto-Encoder
  • Understand Recurrent Neural Networks
  • Understand Convolutional Neural Networks
  • Explore the Learning vs Memorization Tradeoff