Face and Hand Models
[ Slides | Sketches ]
Objectives
- Learn about TensorFlow Face Landmarks Detection model and how it works.
- Learn about TensorFlow Hand Pose Detection model and how it works.
- Learn to work with ml5.faceMesh and ml5.handPose.
- Learn to work with 3D graphics in p5.js WEBGL mode.
Lecture Notes
Music
Playful
Code Examples
Core Functionalities
Creative Applications
Supplemental Materials
Technical Blog Posts
Video Tutorials
Assignment 4
- Research and find a project (experiments, websites, art installations, games, etc) that utilizes machine learning in a creative way. Consider the following:
- What type of machine learning models did the creator use?
- What data might have been used to train the machine learning model?
- Why did the creator of the project choose to use this machine learning model?
- Pick one of the models above (FaceMesh or HandPose), following the examples above and the ml5.js documentation, experiment with controlling elements of a p5.js sketch (color, geometry, sound, text, etc) with the output of the model. Try to create an interaction that is surprising or one that is inspired by the project you find.
- Document your research, your response, and your p5.js sketch in a blog post. Add a link to the post and your p5.js sketch on the Assignment 4 Wiki page. In your blog post, include visual documentation such as a recorded screen capture / video / GIFs of your sketch.