DEEP BLUEBERRY BOOK

🐳 β˜•οΈ 🧧

This is a tiny and very focused collection of links about deep learning. If you've always wanted to learn deep learning stuff but don't know where to start, you might have stumbled upon the right place!

This self-learning plan is split into five modules and designed to be completed in five weekends. Per module, you might want to take about four hours to digest the theory plus an additional six or more hours to experiment with available code.

Prerequisites

  1. Familiarity with basic multivariate calculus, linear algebra, probability, statistics
  2. Coding experience with Python or any other programming language
  3. (Recommended) S Keshav: How to Read a Paper

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp

Deep Learning Foundations

🐳 β˜•οΈ 🧧

  • Learn the principles behind deep learning and artificial neural networks
  • Implement simple feedforward neural networks with Tensorflow Keras and PyTorch

Videos + Readings

  1. 65 min 2017-Oct 3Blue1Brown: Neural Networks Playlist
    • 19 min But what is a Neural Network?
    • 21 min Gradient descent, how neural networks learn
    • 14 min What is backpropagation really doing?
    • 11 min Backpropagation calculus
  2. 24 min 2017-Mar Brandon Rohrer: How Deep Neural Networks Work
  3. 45 min 2019-Jan Alexander Amini: Introduction to Deep Learning
  4. 2015-Jun Michael Nielsen: A visual proof that NNs can compute any function
  5. 2017-May Jonathan Uesato: What is an ablation study? (Quora)

Coding Exercises

  1. Andrej Karpathy | Denny Britz - Build Neural Networks from Scratch
  2. Soumith Chintala: PyTorch 60 Minute Blitz
  3. Tensorflow Keras | PyTorch Tutorials | FChollet Tutorials

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp

Deep Computer Vision

🐳 β˜•οΈ 🧧

  • Learn what a convolutional neural network is and the mathematics behind recognizing images
  • Be familiarized with widely-used techniques like transfer learning, feature preprocessing, data augmentation, and layer visualization

Videos + Readings

  1. 18 min 2015-Mar Fei Fei Li: How we teach computers to understand pictures
  2. 16 min 2017-May Caryk Huang: Teaching my computer to give me friends
  3. 27 min 2016-Aug Brandon Rohrer: How CNNs work
  4. 2018-Jun Irhum Shafkat: Intuitively understanding convolutions for deep learning
  5. 33 min 2017-Mar Luis Serrano: A friendly introduction to CNNs and image recognition
  6. Andrej Karpathy: Visualizing what CovNets learn
  7. Adam Harley: Interactive Visualization of CNN

Coding Exercises

  1. 2017-Jan Alex Staravoitau: Traffic Sign Classification
  2. Sasank Chilamkurthy: Classifying ants and bees (Transfer Learning)
  3. 2016-Jan Francois Chollet: How convolutional neural networks see the world
  4. Alexis Jacq: Neural Transfer Using PyTorch

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp

Deep Sequence Models

🐳 β˜•οΈ 🧧

  • Learn how RNNs (LSTMs, GRUs) work and why they are used to process sequences

Videos + Readings

  1. 40 min 2015-Sep Andrej Karpathy: Visualizing and Understanding Recurrent Networks
  2. 2015-Aug Christopher Olah: Understanding LSTM Networks
  3. 24 min 2018-Sep Michael Nguyen: An illustrated Guide to RNN and LSTM
  4. 27 min 2017-Jun Brandon Rohrer: RNN and LSTM
  5. 2019-Jan Ava Soleimany: Deep Sequence Modeling

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp

Deep Generative Models

🐳 β˜•οΈ 🧧

  • Learn how Generative Adversarial Networks and Variational Autoencoders can produce realistic, never-before-seen data

Videos + Readings

  1. 2019-Jan Alexander Amini: Generative Models Slides
VARIATIONAL AUTOENCODERS
  1. 2018-Feb Irhum Shafkat: Intuitively Understanding Variational Autoencoders
  2. 2018-Mar Jeremy Jordan: Variational Autoencoders
  3. 2017-Dec Keita Kurita: An Intuitive Explanation of Variational Autoencoders
  4. 2017-May Will Kurt: Kullback-Leibler Divergence Explained
GENERATIVE ADVERSARIAL NETWORKS
  1. 2016-Jun Kevin Frans: Generative Adversarial Networks Explained
  2. 22 min 2017-Oct Rob Miles: Generative Adversarial Networks (GANs)
  3. 2019-Jan Minsuk Kahng et al: Play with Generative Adversarial Network in Your Browser
  4. 2019-Jan Madhu Sanjeevi: Generative Adversarial Networks with Math

Coding Exercises

  1. 21 min 2018-Jul John Fisher: Autoencoder Tutorial in Keras
  2. MIT Bootcamp: Intro to Deep Learning Debiasing Faces
  3. 17 min 2018-Jan Dominic Mon: Make a Face Adversarial Network in 15 minutes
  4. Erik Linder-noren: PyTorch-GAN

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp

Deep Reinforcement Learning

🐳 β˜•οΈ 🧧

  • Understand the core ideas and terminologies used in the field
  • Understand a few reinforcement learning algorithms
  • Apply these algorithms to videogame-like environments such as OpenAIGym and MuJoCo

Readings + Coding Exercises

  1. 2017-Nov Josh Greaves: Understanding Reinforcement Learning
  2. JaromΓ­r Janisch: Let's Make DQN and A3C series
  3. 2018-Mar Thomas Simonini: Deep Reinforcement Learning (free course)

Advanced Material

  1. Dulat Yerzat: RL Adventure on DQN and Policy Gradients
  2. 2018-Nov Josh Achiam: OpenAI Spinning Up (free course)

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp

Deeper: What's next?

🐳 β˜•οΈ 🧧

If you've finished all five modules, congratulations! πŸŽ‰πŸŽ‰ You are now familiar with some of the hottest topics in deep learning today. You might want to continue your deep learning journey with the links listed below. πŸš€ are must-clicks!

  1. 2018-Mar Tess Ferrandez: Notes from Andrew Ng's courses
  2. 2016-Jan Sebastian Ruder: Gradient Descent Optimization Algorithms
  3. 2016-Sep Fjodor Van Veen: The Neural Network Zoo
  4. πŸš€ Distill.pub: A modern medium presenting research
  5. 18 min 2015-Feb Ian Goodfellow: Adversarial Examples
  6. πŸš€ 37 min 2019-Jan Ava Soleimany: Limitations and New Frontiers
  7. πŸ’°πŸ’° Brandon Rohrer: Neural Network Visualization
  8. πŸš€ Fast.AI: Practical and Cutting-Edge Deep-learning for Coders (free course)
  9. 2018-Nov Lilian Weng: Meta-Learning: Learning to Learn Fast
  10. 52 min 2017-Dec Pieter Abbeel: Deep Learning for Robotics
  11. 43 min 2018-Jan David Silver: Mastering games without Human Knowledge
  12. Papers with Code | Zaur Fataliyev: PWC

🐳 🐳 🐳

33Mudy961bUk9zz35p68g9fE3uuHLRduRp