Handsfree.js

# ðŸĪļ‍♀ïļ Pose

A person striking various poses, with a wireframe overlaid on top of them

Overview and basic demo
  • ðŸĪļ‍♀ïļ Full body mode with 33 2D pose landmarks
  • ðŸĪš Upper body mode with 25 2D upper pose landmarks
  • 📅 Extra helpers and plugins coming soon

This model doesn't come with any bonuses or plugins yet but they'll come soon. The API will remain exactly the same, so feel free to started with this model today!

# Usage

# With defaults

const handsfree = new Handsfree({pose: true})
handsfree.start()

# With config

const handsfree = new Handsfree({
  pose: {
    enabled: false,
    
    // Outputs only the top 25 pose landmarks if true,
    // otherwise shows all 33 full body pose landmarks
    // - Note: Setting this to true may result in better accuracy 
    upperBodyOnly: false,

    // Helps reduce jitter over multiple frames if true
    smoothLandmarks: true,

    // Minimum confidence [0 - 1] for a person detection to be considered detected
    minDetectionConfidence: 0.5,

    // Minimum confidence [0 - 1] for the pose tracker to be considered detected
    // Higher values are more robust at the expense of higher latency
    minTrackingConfidence: 0.5
  }
})

handsfree.start()

# Data

# Pose Landmarks


Image source, MediaPipe: https://google.github.io/mediapipe/solutions/pose#pose-landmark-model-blazepose-tracker (opens new window)

// An array of landmark points for the face
handsfree.data.pose.poseLandmarks == [
  // Landmark 0
  {x, y, visibility},
  // Landmark 1
  {x, y, visibility},
  // ...
  // Landmark 32
  {x, y, visibility}
]

// landmark 0
handsfree.data.pose.poseLandmarks[0].x
handsfree.data.pose.poseLandmarks[0].y
// The confidence in this pose landmark
handsfree.data.pose.poseLandmarks[0].visibility

# Examples of accessing the data

handsfree = new Handsfree({pose: true})
handsfree.start()

// From anywhere
handsfree.data.pose.poseLandmarks

// From inside a plugin
handsfree.use('logger', data => {
  if (!data.pose) return

  console.log(data.pose.poseLandmarks)
})

// From an event
document.addEventListener('handsfree-data', event => {
  const data = event.detail
  if (!data.pose) return

  console.log(data.pose.poseLandmarks)
})

# Examples

Flappy Pose
Person playing Flappy Bird by flapping their arms. Flappy Bird is a game where you must flap the birds wings to fly or dodge barriers

In this game, the goal is to flap your arms to get the bird to fly around dodging obstacles. Made with an older version of Handsfree, but the API is very similar!

Add your project
If you've made something with this model I'd love to showcase it here! DM me on Twitter, make a pull request, or find us on Discord.

← 😉 Weboji →

Debugger