Intro-ML-Arts-IMA-Summer24

Convolutional Neural Network

[ Slides | Sketches ]

Objectives

• Understand when and why you might train your own model from scratch versus use a pre-trained model or transfer learning.
• Understand how to work with image data for training your own model.
• Learn about different datasets and their formats.
• Learn to feed the input of a graphics canvas into a machine learning model.
• Learn to train an image classifier (with and without convolutional layers) with ml5.js.
• Learn the distinction between different types of layers of a neural network, especially a convolutional layer and a pooling layer.

Lecture Notes

Tools

• ml5.neuralNetwork Reference
  • ml5.neuralNetwork Source Code
  • TensorFlow.js Source Code

Datasets

• The Quick, Draw! Dataset from Google Creative Lab.
• The MNIST Dataset by Yann LeCun el al.

Related Projects

• MegaPixels curated by Tactical Tech, design and development by Adam Harvey.
• What Neural Networks See by Gene Kogan.

Quick Draw! Dataset

• Letter Collages by Deborah Schmidt.
• Face Tracking Experiment by Neil Mendoza.
• Faces of Humanity by La Boite à Tortue.
• Scribbling Speech by Xinyue Yang.
• How do you draw a circle? by Thu-Huong Ha and Nikhil Sonnad.
• Machine Learning for Visualization by Ian Johnson.

Code Examples

Working with Datasets

• Loading and Displaying Quick, Draw! Dataset
• Loading and Displaying MNIST Dataset

Convolution Neural Network Layers

• Convolutional Neural Network - Filter Demo
• Convolutional Neural Network - Max Pooling Demo

Training Image Classifiers

• Training a Doodle Classifier without Convolutional Layers
• Training a Doodle Classifier with Convolutional Layers
• Training a Handwritten Digit Classifier with Convolutional Layers
• Training a Webcam Image Classifier with Convolutional Layers
• Doodle Classification with DoodleNet (model trained by @yining1023)

Supplemental Materials

• An Intuitive Explanation of Convolutional Neural Networks by Ujjwal Karn.
• “Gradient-Based Learning Applied to Document Recognition” by Y. LeCun, L. Bottou, Y. Bengio, P. Haffner.
• How computers got shockingly good at recognizing images by Timothy B. Lee.
• A visual and intuitive understanding of deep learning, CNNs (0:00–9:40) by Octavio Good.
• Recognizing Human Facial Expressions With Machine Learning by Angelica Perez.
• Convolutional Neural Networks - The Math of Intelligence by Siraj Raval.

Visualizations and Demos

• Image Kernels Explained Visually by Victor Powell.
• Interactive Node-Link Visualizations of Convolutional Neural Networks by Adam W. Harley.
• Convolution Operation Demo by Deeplizard.
• Convolutional Neural Network demos from Yining Shi. (Note: these demos were created with an older version of ml5.js, refer to the ml5.js Resources Wiki page for more information.)

Video Tutorials

Note: ml5.js tutorials below were taught using an older version of ml5.js, refer to the ml5.js Resources Wiki page for more information.

Working with Quick, Draw! Dataset

• Replaying Drawings with Node Server - video tutorial by Daniel Shiffman.
• Replaying Drawings with Google Web API - video tutorial by Daniel Shiffman.

Convolution Neural Network

• ml5.js: What is Convolutional Neural Network Part 1 - Filters - video tutorial by Daniel Shiffman.
• ml5.js: What is Convolutional Neural Network Part 2 - Max Pooling - video tutorial by Daniel Shiffman.

Training Neural Networks

• Doodle Classifier: Prepping Quick Draw! Data in p5.js - video tutorial by Daniel Shiffman.
• ml5.js: Training a Neural Network with Pixels as Input - video tutorial by Daniel Shiffman.
• ml5.js: Training a Convolutional Neural Network for Image Classification - video tutorial by Daniel Shiffman.

Assignment 6

1. Read Exploring and Visualizing an Open Global Dataset by Google Research.
2. Choose a neural network we trained in class or one from the code examples above. Experiment with the hyperparameters (learningRate, batchSize, epochs, etc) and observe how it affects the training performance.
3. Document your response to the readings as well as your experiments and observations in a blog post and add a link to the post and your p5.js sketch on the Assignment 6 Wiki page. In your blog post, include visual documentation such as a recorded screen capture / video / GIFs of your sketch.
4. Lastly, be prepared to share preliminary ideas on your final project in the next class.

This site is open source. Improve this page.