Awesome
Mediapipe Handpose and Facemesh Demos
Table of Contnets
Handpose Demos
Collection of minimalistic demos/templates using mediapipe handpose with tensorflow.js. Created for and supported by the Frank-Ratchye STUDIO for Creative Inquiry.
The handpose package detects hands in an input image or video stream, and returns twenty-one 3-dimensional landmarks locating features within each hand. Such landmarks include the locations of each finger joint and the palm. -- blog.tensorflow.org
There are 4 templates included in this repo, 2 of them with three.js and 2 with p5.js, 2 are multiplyer via socket.io + express, and 2 are singleplayer static sites. They're all well commented and explained for beginners.
All demos are hosted on glitch.com 🎏
Online multiplayer 3D
Live | Remix | Source
- tensorflow.js + mediapipe handpose
- three.js
- socket.io + express
Online multiplayer 2D
Live | Remix | Source
- tensorflow.js + mediapipe handpose
- p5.js
- socket.io + express
Singleplayer 3D
Live | Remix | Source
- tensorflow.js + mediapipe handpose
- three.js
Singleplayer 2D
Live | Remix | Source
- tensorflow.js + mediapipe handpose
- p5.js
Also available on editor.p5js.org: https://editor.p5js.org/lingdong/sketches/1viPqbRMv
4 in 1
Glued together with iframes: https://handpose-demos.glitch.me/
Facemesh Demos
Collection of minimalistic demos/templates using mediapipe facemesh with tensorflow.js. Created for and supported by the Frank-Ratchye STUDIO for Creative Inquiry.
The facemesh package infers approximate 3D facial surface geometry from an image or video stream, requiring only a single camera input without the need for a depth sensor. -- blog.tensorflow.org
There are 4 templates included in this repo, 2 of them with three.js and 2 with p5.js, 2 are multiplyer via socket.io + express, and 2 are singleplayer static sites. They're all well commented and explained for beginners.
All demos are hosted on glitch.com 🎏
Keypoint Subsets
There're 3 additional keypoint subsets available in each demo to select from, as alternatives to the facemesh 468 vertices. See image below for indices and triangulation for each of them:
From left to right:
- Minimal 7 points
- Important features 33 points
- Standard facial landmark 68 points
- Full facemesh 468 points. A larger resolution and flattened version of the illustration is available in tfjs facemesh repo.
Online multiplayer 3D
Live | Remix | Source
- tensorflow.js + mediapipe facemseh
- three.js
- socket.io + express
Online multiplayer 2D
Live | Remix | Source
- tensorflow.js + mediapipe facemseh
- p5.js
- socket.io + express
Singleplayer 3D
Live | Remix | Source
- tensorflow.js + mediapipe facemseh
- three.js
Singleplayer 2D
Live | Remix | Source
- tensorflow.js + mediapipe facemseh
- p5.js
Also available on editor.p5js.org: https://editor.p5js.org/lingdong/sketches/ef6FB-uNq
Made at the Frank-Ratchye STUDIO for Creative Inquiry at Carnegie Mellon University.