Exporting Skeletal Animations as USD(A)

I have been trying to export GLB and FBX imported animations (and generated ones from Google MediaPipe) as USDA, but cannot get the bindTransforms and animation quaternions right. It plays in the web browser, the exported file does not look right in Ominiverse.

I have tried various tool chains to generate a reference to work from, but failed so far. E.g. The Omniverse version of Blender has a USD export which has been the most successful, but they are Z-up and I would like Y-up. (Maybe I should give up on that goal.)

Current code for the model

  • I export bone.matrixWorld as the USD bindTransforms for the model
  • I export bone.matrix as the USD restTransform for the model

In the SkelAnimation

  • I export the rotation quaternions from the GLB animation array of .values

Initially I thought this was enough - head rotations come out correctly. But then I used the retargeting example I successfully used to retarget a mixamo animation to my own model in the browser. When I export that, the head moves correctly but the rest of the arms and legs do not.

You can see the head turn the same as the “Sitting Talking” animation from Mixamo.

But clearly the rest of the character is wrong. Just wondering if someone knowledgeable with the required rotations etc knew how the maths to apply to the quaternions before export. I have tried a range of tool chains to try and get a full body reference model to work from, but after 5 different tool chains I am running out of options. (The generated animation comes out wrong, or with Blender everything is converted to Z-up which I was hoping to avoid.) I don’t know if it is the bindTransforms, restTransforms, or time coded rotation quaternions that are wrong (or all three!).

The Blender GLTF exporter converts Z up to Y up.

I’m less familiar with USD but from what you’re saying it sounds like they default to Z up… same as Blender.

I’ve been down this road before and it’s a rough one.

No good answer, but you have my sympathy.

Maybe @donmccurdy has some insight? :smiley:

Have you tried just importing and then nesting in a Group object and rotating that? I would assume the animations keys are in objects local space.

I would make 3d mesh versions of axis’s xyz and export those along with the model in both, parented and unparented tests

And then try a custom armature rig in your 3d app that looks the same starter pose and basic movements to try to gauge whats different about them

It might be something about USD is just more proper in Maya vs Blender since Pixar does not use Blender internally or whatever app.
One of the bugbears of rigs is making sure the Roll property is correct but theres no good way to truely know it beyond tooling in your editor app and testing things

I finally solved the problem. USD required me to add position animation data for all joints. If I wanted to rotate the bone, I had to adjust the position as well. It turned out I had filled the position data with zeros (there was nothing in the animation clip - it only animated the root position). I had assumed (0,0,0) offset meant no movement. Wrong! The image shown is the character with all bones starting at position 0,0,0. Once I dropped in the default rest position for all the joint offsets it started working.

In case of any use to anyone stumbling across this thread in the future, I wrote it up as a blog post: Retargeting Skeletal Animation to USD – Extra Ordinary, the Series

2 Likes

AWESOME sauce!! usd can be funky and pixars docs are wordy

1 Like