Handsfree.js

# Model: FaceMesh

A 3D model of a hand projected above a person's hand

Overview and basic demo
  • 🙂 468 2D face landmarks
  • 😁😜 Track up to 4 faces at once
  • 📅 Extra helpers and plugins coming soon

This model doesn't come with any bonuses or plugins yet but they'll come soon. The API will remain exactly the same, so feel free to started with this model today!

# Usage

# With defaults

const handsfree = new Handsfree({facemesh: true})
handsfree.start()

# With config

const handsfree = new Handsfree({
  facemesh: {
    enabled: true,
    // The maximum number of faces to detect [1 - 4]
    maxNumFaces: 1,

    // Minimum confidence [0 - 1] for a face to be considered detected
    minDetectionConfidence: 0.5,
    
    // Minimum confidence [0 - 1] for the landmark tracker to be considered detected
    // Higher values are more robust at the expense of higher latency
    minTrackingConfidence: 0.5
  }
})

handsfree.start()

# Data

// faceIndex [0 - 3] An array of landmark points for each detected face
handsfree.data.facemesh.multiFaceLandmarks[faceIndex] == [
  // Landmark 0
  {x, y},
  // Landmark 1
  {x, y},
  // ...
  // Landmark 467
  {x, y}
]

// face 0, landmark 0
handsfree.data.facemesh.multiFaceLandmarks[0][0].x
handsfree.data.facemesh.multiFaceLandmarks[0][0].y

# Examples of accessing the data

handsfree = new Handsfree({facemesh: true})
handsfree.start()

// From anywhere
handsfree.data.facemesh.multiFaceLandmarks[0]

// From inside a plugin
handsfree.use('logger', data => {
  if (!data.facemesh) return

  console.log(data.facemesh.multiFaceLandmarks[0][0])
})

// From an event
document.addEventListener('handsfree-data', event => {
  const data = event.detail
  if (!data.facemesh) return

  console.log(data.facemesh.multiFaceLandmarks)
})

# Inspiration

The following projects where built with TensorFlow's Face Mesh. They weren't made with Handsfree.js, but they're listed here as they use the same model and data ✨

AR Face Filter
Various full digital face masks mimicking a persons head movements

In this project, Samarth Gulati and Praveen Sinha demonstrate how to use Face Mesh together with Three.js to create customizable face filters!

Add your project
If you've made something with this model I'd love to showcase it here! DM me on Twitter, make a pull request, or find us on Discord.

← 🤸‍♀️ Pose →

Debugger