Animation of Indian Sign language - looking for three.js developer

Jobs
1

I would like to animate an avatar for Indian sign gestures from Indian sign language -
https://indiansignlanguage.org/

We would like to animate the Avatars from here

and it’s running version is on at

I wanted to have

  1. A way to automate the gesture generation from words from uploaded video
    or
  2. Create a JS file attached for each word, I have attached for Time and Home

I would also like to be suggested a way forward to animate using any avatars.

I am looking for the Three.js developer who can build this animation for us

Satyendra
HOME.js (1.9 KB)
TIME.js (5.4 KB)

3

If you don’t have a motion capture gloves, my advise is to look at Google’s Mediapipe.
Use it to save hands positions, then apply this data to any IK-rigged avatar.

Mediapipe is more or less stable, here is my project, based on it:
https://shaderpipe.yesbird.online/
https://github.com/syanenko/ShaderPipe

4

Thanks, we indeed use Mediapipe to capture Gesture for the animating part we are still evaluating a way to create the Hand Gesture using some technique on the fly.

5

Maybe I am missing something, but if you can create a gesture “vocabulary” with Mediapipe,
then you can apply it to an avatar after speech recognition, yes ?

6

Yes that’s one of the way we thought of and in pipeline and applying it to avatars.
At the moment we are looking for three.js developer who can do this animation for us

1 Like
7

Just dropping in the idea that gestures can be done without sophisticated tools. Here is my attempt to model from scratch a fingerspelling in the Japanese Sign Language. You may click to see it live (but you need hiragana characters in your fonts to see the moras):

https://boytchev.github.io/virtual-robotics/examples/fingerspelling.html

image

I’m not applying for this position.

8

Thanks you so much for the shared article, it will definitely be helpful to us.

9

I have a suggestion for your situation. You can apply animations to the model in Autodesk or Blender and then run it in Three.js. This will allow you to control various aspects, such as timing, repetition, and more.