What are people using for hand tracking gesture detection?

Questions
1

I’m currently working on adding hand tracking to Digital Bacon, and want to capture some gesture based equivalents for the trigger and grip buttons. Using a pinch for the trigger is simple enough as I can check the distance between thumb tip and index tip, but for the grip button I’m not sure what to do. Ideally I check if the hand is in a fist, but before I start trying to think of a solution for that, I wanted to see if there are any existing solutions I can use?

2

You may have a look at this one:

1 Like
3

Last year I spent a couple of days building a basic gesture recognition system. Its hosted in Angular project components, but the basic algorithm is written in Typescript. It basically measures the distance of each finger tip from the wrist. It compares that against registered gestures. If all 5 fingers are the best match for a gesture, it assumes that gesture. It works surprisingly well. Its independent of the orientation of the the hand.
There’s also a method to capture a gesture. One hand makes the pose, the other hand indicates when to take a snapshot. It compares against a single previous snapshot to indicate if the distances are better or worse.
There’s definitely room for improvement, but its potentially a place to start
You can find demo here. Left hand to monitor or capture gestures. I think text appears on the far wall when a gesture matches. Right hand finger pinch to capture gesture and dump results to browser console. Sorry it isn’t any better than that. See this file for some of the gestures recognized.

In the research notes I left myself,

  • I don’t expect the gestures I captured of my hands to match anyone else. Each person would need to provide their own capture gesture distances
  • When I reverse and guessed gestures using right hand, some where hard to get a match. This means matching/training must be for both hands

Hope this helps

1 Like
4

OpenCV was shipping 1-factor hooks in 1999.

5

Thank you! I will definitely be using this once I start needing additional gestures! Right now I just need to figure out “grabbing” and the other solution ended up being quite simple for that

1 Like
6

Thanks! I ended up going off of your solution and checking the distance of the fingertips to the wrist against the distance of the knuckles to the wrist