Handsfree.js

# About Handsfree.js.org

# Master Plan

  1. Create a library that makes face, hand, eye, and pose tracking and voice and mind control easy to use
  2. Use the library to build a repository of custom handsfree plugins, components, and gestures
  3. Use the repository to seed a "user script manager" to power a handsfree browser extension
  4. Grow a community of handsfree users and developers around the library and repository
  5. Start "The Handsfree Foundation" to promote creative expression

# Special Thanks

This project couldn't have been possible without:

# Support Handsfree.js.org

Please consider supporting this project 💜 There are several ways to help:

# Changelog

Date Version Description
2020-08-27 8.5.1 MediaPipe Hands now works on mobile 🎉
2020-03-13 8.4.4 Handsfree.js is now transpiled with Babel and works with React, Vue, and other frameworks out of the box
2020-03-07 8.4.3 Fixes filename casing issue that prevented Handsfree.js from building on Linux
2020-02-26 8.4.2 Adds ability to use a video or stream instead of a webcam
2020-02-18 8.4.0 Removes the MediaPipe Holistic model. The holistic model is essentially the same as running Hands, Facemesh, and Pose together, although because the holistic data is structured differently (despite using the same models) it led to a lot of confusion and was incompatible with plugins.
2020-02-06 8.3.0 Adds basic gesture support through the Gesture Creator. This is still mostly undocumented
2020-01-28 8.2.6 Adds handsfree.data.hands.landmarks and handsfree.data.hands.landmarksVisible to MediaPipe Hands, which contain the landmarks for each hand in specific indexes:
.landmarks[0] === Left Hand, Person #1
.landmarks[1] === Right Hand, Person #1
.landmarks[2] === Left Hand, Person #2
.landmarks[3] === Right Hand, Person #2
2020-01-27 8.2.4 Improved the pinchScroll plugin so that it continuously scrolls (instead of having to release and pinch again); .handsfree-started is added to body of a client install (opens new window) (useful in Browser Extensions); merges the Handsfree Browser Extension to improve iteration feedback cycle (not documented yet); disabling the last active model turns off the camera; fixes a few Weboji bugs that prevented it from working with other models
2020-01-25 8.2.3 palmPointers plugin and faceClick now emit mousedown, mousemove, mousedrag, and mouseup instead of just click
2020-01-18 8.2.2 Adds new palmPointers plugin and updates the pinchScroll plugin to support all hands. Also adds a new /boilerplate/cdn/pinch-scrolling.html
2020-01-13 8.2.1 Adds new normalized helper: handsfree.data.handpose.normalized. These [x, y, z] are similar to .handpose.landmarks but normalized between [0, 1]
2020-01-06 8.2.0 Adds TensorFlow Handpose including a basic THREE boilerplate remixed from @LingDong-'s handpose-facemesh-demos (opens new window), and a fingertip raycaster
2020-01-01 8.1.2 Fixes Face Pointer offsets, which was causing the pointer to assume you were facing straight up 😆
2020-12-31 8.1.1 Adds pinchers Plugin for easily working with finger pinches through events, new properties, and styles
2020-12-30 8.1.0 Adds .showDebugger(), .hideDebugger(), and helper classes. Renames feedback to debugger
2020-12-29 8.0.10 Fixes a bug where handsfree.debug.$video had 0 width and height; Fixed issue where the webcam feed was not showing behind model wireframes when handsfree.config.showDebug === true
2020-12-28 8.0.9 Adds isClient Mode for remote inference. .runPlugins() .TweenMax(), .throttle(), reduced filesize by 80kb+
2020-12-22 8.0.7 Adds handsfree.model.weboji.isDetected

←

Debugger