How to modify meshes and material that are grouped under skinned meshes from a model created with gltfjsx (see description)

Questions
, ,
1

Hi.

I have a gltf model that I am importing using gltfjsx (see code below). With this, I am cloning the material, as well as rendering the skinnedMeshes twic - first with their original material and then with the clone material. For the cloned material, I set the color to black, and depthTest to false. For the other - original and untouched - set of skinnedMeshes, I set renderOrder to 5.

The result, as you can see is that for each skinnedMesh, the black model is hidden behind the other model, which has a higher renderOrder, but the other parts of the models, that are standard meshes, and - as you can see in the code - are not directly (in the jsx) rendered, but are rather children of one of the skinnedMeshes, the copied material is unmodified and the mesh that is a child of one of the original and untouched skinned meshes has a standard renderOrder, and so it bleeds through.

see images 1 and 2 for the result of the above.

Does anyhow have a tip on how I can

  1. for the original model, make sure all meshes (skinned and not) have a higher renderOrder, and 2
  2. for the copied material, make sure that it’s applied to the children of the skinnedMeshes of which it is applied.

I have also tried to remove all non skinned meshes in blender, and the result is as it should be (see images 3 and 4)

Hope this is somewhat clear, I leave images and code below, please ask if you need clarifications.

image image image image

export function Model(props: Props) {
  const group = useRef<THREE.Group>(null!)

  const { scene, materials, animations } = useGLTF("/NPCs/knight.glb") as GLTFResult
  const clone = useMemo(() => SkeletonUtils.clone(scene), [scene])
  const graph = useGraph(clone)
  const nodes = graph.nodes as GLTFResult["nodes"]
  const { actions } = useAnimations(animations, group)

  useNpcActions(actions, props.action)

  const material = useMemo(() => {
    const material = materials.knight_texture.clone()
    material.color = new THREE.Color(0, 0, 0)
    material.depthTest = false

    return material
  }, [materials.knight_texture])

  return (
    <group ref={group} {...props} dispose={null}>
      <group name="Scene">
        <group name="Rig">
          <primitive object={nodes.root} />

          <skinnedMesh
            castShadow
            receiveShadow
            name="Knight_ArmLeft"
            geometry={nodes.Knight_ArmLeft.geometry}
            material={material}
            skeleton={nodes.Knight_ArmLeft.skeleton}
          />
          <skinnedMesh
            castShadow
            receiveShadow
            name="Knight_ArmRight"
            geometry={nodes.Knight_ArmRight.geometry}
            material={material}
            skeleton={nodes.Knight_ArmRight.skeleton}
          />
          <skinnedMesh
            castShadow
            receiveShadow
            name="Knight_Body"
            geometry={nodes.Knight_Body.geometry}
            material={material}
            skeleton={nodes.Knight_Body.skeleton}
          />
          <skinnedMesh
            castShadow
            receiveShadow
            name="Knight_Head"
            geometry={nodes.Knight_Head.geometry}
            material={material}
            skeleton={nodes.Knight_Head.skeleton}
          />

          <skinnedMesh
            castShadow
            receiveShadow
            name="Knight_LegLeft"
            geometry={nodes.Knight_LegLeft.geometry}
            material={material}
            skeleton={nodes.Knight_LegLeft.skeleton}
          />
          <skinnedMesh
            castShadow
            receiveShadow
            name="Knight_LegRight"
            geometry={nodes.Knight_LegRight.geometry}
            material={material}
            skeleton={nodes.Knight_LegRight.skeleton}
          />

          <skinnedMesh
            renderOrder={50}
            castShadow
            receiveShadow
            name="Knight_ArmLeft"
            geometry={nodes.Knight_ArmLeft.geometry}
            material={materials.knight_texture}
            skeleton={nodes.Knight_ArmLeft.skeleton}
          />
          <skinnedMesh
            renderOrder={50}
            castShadow
            receiveShadow
            name="Knight_ArmRight"
            geometry={nodes.Knight_ArmRight.geometry}
            material={materials.knight_texture}
            skeleton={nodes.Knight_ArmRight.skeleton}
          />
          <skinnedMesh
            renderOrder={50}
            castShadow
            receiveShadow
            name="Knight_Body"
            geometry={nodes.Knight_Body.geometry}
            material={materials.knight_texture}
            skeleton={nodes.Knight_Body.skeleton}
          />
          <skinnedMesh
            renderOrder={50}
            castShadow
            receiveShadow
            name="Knight_Head"
            geometry={nodes.Knight_Head.geometry}
            material={materials.knight_texture}
            skeleton={nodes.Knight_Head.skeleton}
          />

          <skinnedMesh
            renderOrder={50}
            castShadow
            receiveShadow
            name="Knight_LegLeft"
            geometry={nodes.Knight_LegLeft.geometry}
            material={materials.knight_texture}
            skeleton={nodes.Knight_LegLeft.skeleton}
          />
          <skinnedMesh
            renderOrder={50}
            castShadow
            receiveShadow
            name="Knight_LegRight"
            geometry={nodes.Knight_LegRight.geometry}
            material={materials.knight_texture}
            skeleton={nodes.Knight_LegRight.skeleton}
          />
        </group>
      </group>
    </group>
  )
}
2

here is a video that might make it more clear

My guess is, I have to iterate the nodes and modify all relevant nodes, but in that case, I am a bit unsure on how to copy the material, and the apply it, without affecting the original. And also, it looks like there is only one material, So I am not sure why it’s not affected in the case of the non skinned meshes.

3

I solved this by manually setting either material or renderOrder for each mesh in a useEffect. My solution now, however, requires me to draw two instances of the model, one time normally, with a higher renderOrder, and a second time, as a shadow. I differentiate between the two modes with a prop asShadow (complete code below). Not sure how ineffective this is, but it works.

/*
Auto-generated by: https://github.com/pmndrs/gltfjsx
Command: npx gltfjsx@6.2.15 public/NPCs/knight.glb -t -s -r public/ -o ./src/components/characters/NPCs/Knight.tsx 
*/

import * as THREE from "three"
import React, { useEffect, useMemo, useRef } from "react"
import { useGLTF, useAnimations } from "@react-three/drei"
import { GLTF, SkeletonUtils } from "three-stdlib"
import { useGraph } from "@react-three/fiber"
import { useNpcActions } from "./useNpcActions"

type GLTFResult = GLTF & {
  nodes: {
    Badge_Shield: THREE.Mesh
    ["1H_Sword"]: THREE.Mesh
    Knight_Helmet: THREE.Mesh
    Knight_Cape: THREE.Mesh
    Knight_ArmLeft: THREE.SkinnedMesh
    Knight_ArmRight: THREE.SkinnedMesh
    Knight_Body: THREE.SkinnedMesh
    Knight_Head: THREE.SkinnedMesh
    Knight_LegLeft: THREE.SkinnedMesh
    Knight_LegRight: THREE.SkinnedMesh
    root: THREE.Bone
  }
  materials: {
    knight_texture: THREE.MeshStandardMaterial
  }
}

export type ActionName =
  | "1H_Melee_Attack_Chop"
  | "1H_Melee_Attack_Slice_Diagonal"
  | "1H_Melee_Attack_Slice_Horizontal"
  | "1H_Melee_Attack_Stab"
  | "1H_Ranged_Aiming"
  | "1H_Ranged_Reload"
  | "1H_Ranged_Shoot"
  | "1H_Ranged_Shooting"
  | "2H_Melee_Attack_Chop"
  | "2H_Melee_Attack_Slice"
  | "2H_Melee_Attack_Spin"
  | "2H_Melee_Attack_Spinning"
  | "2H_Melee_Attack_Stab"
  | "2H_Melee_Idle"
  | "2H_Ranged_Aiming"
  | "2H_Ranged_Reload"
  | "2H_Ranged_Shoot"
  | "2H_Ranged_Shooting"
  | "Block_Attack"
  | "Block_Hit"
  | "Block"
  | "Blocking"
  | "Cheer"
  | "Death_A_Pose"
  | "Death_A"
  | "Death_B_Pose"
  | "Death_B"
  | "Dodge_Backward"
  | "Dodge_Forward"
  | "Dodge_Left"
  | "Dodge_Right"
  | "Dualwield_Melee_Attack_Chop"
  | "Dualwield_Melee_Attack_Slice"
  | "Dualwield_Melee_Attack_Stab"
  | "Hit_A"
  | "Hit_B"
  | "Idle"
  | "Interact"
  | "Jump_Full_Long"
  | "Jump_Full_Short"
  | "Jump_Idle"
  | "Jump_Land"
  | "Jump_Start"
  | "Lie_Down"
  | "Lie_Idle"
  | "Lie_Pose"
  | "Lie_StandUp"
  | "PickUp"
  | "Running_A"
  | "Running_B"
  | "Running_Strafe_Left"
  | "Running_Strafe_Right"
  | "Sit_Chair_Down"
  | "Sit_Chair_Idle"
  | "Sit_Chair_Pose"
  | "Sit_Chair_StandUp"
  | "Sit_Floor_Down"
  | "Sit_Floor_Idle"
  | "Sit_Floor_Pose"
  | "Sit_Floor_StandUp"
  | "Spellcast_Long"
  | "Spellcast_Raise"
  | "Spellcast_Shoot"
  | "Spellcasting"
  | "T-Pose"
  | "Throw"
  | "Unarmed_Idle"
  | "Unarmed_Melee_Attack_Kick"
  | "Unarmed_Melee_Attack_Punch_A"
  | "Unarmed_Melee_Attack_Punch_B"
  | "Unarmed_Pose"
  | "Use_Item"
  | "Walking_A"
  | "Walking_B"
  | "Walking_Backwards"
  | "Walking_C"
type GLTFActions = Record<ActionName, THREE.AnimationAction>

type ContextType = Record<
  string,
  React.ForwardRefExoticComponent<
    JSX.IntrinsicElements["mesh"] | JSX.IntrinsicElements["skinnedMesh"] | JSX.IntrinsicElements["bone"]
  >
>

type Props = JSX.IntrinsicElements["group"] & {
  action?: ActionName
  asShadow?: boolean
}

export function Model({ action, asShadow, ...props }: Props) {
  const group = useRef<THREE.Group>(null!)

  const { scene, materials, animations } = useGLTF("/NPCs/knight.glb") as GLTFResult
  const clone = useMemo(() => SkeletonUtils.clone(scene), [scene])
  const graph = useGraph(clone)
  const nodes = graph.nodes as GLTFResult["nodes"]
  const { actions } = useAnimations(animations, group)

  useNpcActions(actions, action)

  const material = useMemo(() => {
    if (!asShadow) return materials.knight_texture

    const shadowMaterial = materials.knight_texture.clone()
    shadowMaterial.color = new THREE.Color(0, 0, 0)
    shadowMaterial.depthTest = false

    return shadowMaterial
  }, [asShadow, materials.knight_texture])

  useEffect(() => {
    if (asShadow) {
      nodes["1H_Sword"].material = material
      nodes.Badge_Shield.material = material
      nodes.Knight_Helmet.material = material
      nodes.Knight_Cape.material = material
    } else {
      nodes["1H_Sword"].renderOrder = 5
      nodes.Badge_Shield.renderOrder = 5
      nodes.Knight_Helmet.renderOrder = 5
      nodes.Knight_Cape.renderOrder = 5
    }
  }, [asShadow, material])

  return (
    <group ref={group} {...props} dispose={null}>
      <group name="Scene">
        <group name="Rig">
          <primitive object={nodes.root} />

          <skinnedMesh
            renderOrder={asShadow ? undefined : 5}
            castShadow
            receiveShadow
            name="Knight_ArmLeft"
            geometry={nodes.Knight_ArmLeft.geometry}
            skeleton={nodes.Knight_ArmLeft.skeleton}
            material={material}
          />
          <skinnedMesh
            renderOrder={asShadow ? undefined : 5}
            castShadow
            receiveShadow
            name="Knight_ArmRight"
            geometry={nodes.Knight_ArmRight.geometry}
            skeleton={nodes.Knight_ArmRight.skeleton}
            material={material}
          />
          <skinnedMesh
            renderOrder={asShadow ? undefined : 5}
            castShadow
            receiveShadow
            name="Knight_Body"
            geometry={nodes.Knight_Body.geometry}
            skeleton={nodes.Knight_Body.skeleton}
            material={material}
          />
          <skinnedMesh
            renderOrder={asShadow ? undefined : 5}
            castShadow
            receiveShadow
            name="Knight_Head"
            geometry={nodes.Knight_Head.geometry}
            skeleton={nodes.Knight_Head.skeleton}
            material={material}
          />

          <skinnedMesh
            renderOrder={asShadow ? undefined : 5}
            castShadow
            receiveShadow
            name="Knight_LegLeft"
            geometry={nodes.Knight_LegLeft.geometry}
            skeleton={nodes.Knight_LegLeft.skeleton}
            material={material}
          />
          <skinnedMesh
            renderOrder={asShadow ? undefined : 5}
            castShadow
            receiveShadow
            name="Knight_LegRight"
            geometry={nodes.Knight_LegRight.geometry}
            skeleton={nodes.Knight_LegRight.skeleton}
            material={material}
          />
        </group>
      </group>
    </group>
  )
}

useGLTF.preload("/NPCs/knight.glb")