Deep Learning Basics

🧠 Free Recall ❓ Quiz 🔍 Cues 🔮 Simulation 🥷 ShadowBox 📐 Encode Cards

Artificial Intelligence - The effort to automate intellectual tasks normally performed by humans

Symbolic AI is the idea of setting a large set of explicit rules for manipulating knowledge

Symbolic v. Machine Learning

With symbolic AI, humans input the rules and data to be processed according to rules and answers come out

With machine learning, humans input data as well as answers expected from the data and out come the rules (model). These rules can be applied to new data to produce original answers. A machine learning system is trained rather than programmed

Machine learning has dissociated with statistics and become more empirical and hands on.

Requisite Machine Learning Material
Input data points to be processed
Examples of expected output
A way to measure whether the algorithm is doing a good job

The central problem in machine learning and deep learning is to meaningfully transform data, to learn representations of the input data at hand, representations to get us closer to expected output.

What is Deep Learning

The deep learning stands for the idea of successive layers of representations.

Modern deep learning often involves tens or hundreds of successive layers of representations and they're all learned automatically from exposure to training data.

Layered representations are learned via models called neural networks, structured in literal layers stacked on top of each other.

Deep learning is a mathematical framework for learning representations from data. They are not models of how the brain works and there is no evidence brain implements anything like the learning mechanisms used in modern deep learning models.

Inputs are turned into many layers of representations all distilling different bits of information.

How the input is encoded into a representation is through its weights which are basically a bunch of numbers. Weights begin as random numbers and after an input is encoded through these weights from each layer, the representation is then compared to the desired output through a loss function. The loss function takes the prediction of the network and the true target and computes a score of the distance between the two which also represents how well the network has done on the specific example. The sore then allows the network to adjust it's weight using its Backpropagation algorithm. This is called the training loop.

The two most used techniques used to solve kaggle problems are

XGBoost - Gradient Boosting Machines
Keras - Deep Learning

Things that held back Deep Learning

The main ideas of deep learning, convolutional neural networks and backpropagation, were already well understood in 1989. Deep Learning was held back due to

Hardware - Better chips in computers
Datasets and Benchmarks - Better access to data thanks to the internet
Algorithmic advances

Unstarred

Dunk (exclude from study)

Deep Learning Basics

Review Questions

Todos

Sensemaker