Problems with Decal Quality, Color, and Text when Downloading GLB Files (GLTFExporter + gltf-transform/draco compression)

[Daniel211]

Hi everyone,

I’m building an editor that allows users to design their own sweaters.
This is done by providing blank models and allowing users to change the color, add text, upload prints, etc.

We also want to give users the option to download the modify 3D model (as GLB file).

The app is built with Next.js and mainly uses React Three Fiber.
I’m using a React ref to take the model from the scene and pass it to my download3DModel function.

import { GLTFExporter } from 'three/examples/jsm/exporters/GLTFExporter'
import { WebIO } from '@gltf-transform/core'
import { KHRONOS_EXTENSIONS } from '@gltf-transform/extensions'
import { draco } from '@gltf-transform/functions'


const download3DModel = async (object) => {
  const options = { binary: true }

  new GLTFExporter().parse(
    object,
    exporterCallback,
    errorCallback,
    options
  )

  async function exporterCallback(gltf) {
    const io = new WebIO()
      .registerExtensions(KHRONOS_EXTENSIONS)
      .registerDependencies({
        'draco3d.encoder': await new DracoEncoderModule(),
        'draco3d.decoder': await new DracoDecoderModule()
      })

    const doc = await io.readBinary(new Uint8Array(gltf))

    await doc.transform(
      draco() // Compress mesh geometry with Draco.
    )

    // Create compressed GLB, in an ArrayBuffer.
    const arrayBuffer = await io.writeBinary(doc)

    const optimisedBlob = new Blob([arrayBuffer])

    saveAs(optimisedBlob, 'sweater.glb')
  }
}

While this is broadly working, there are still several problems when it comes to the details.

I’ve been working with three.js for a few months now and would consider myself somewhere between beginner and intermediate by now, but the issues stemming from exporting are the hardest ones I’ve faced so far and in all three cases I’m not even sure how to even go about properly inspecting the errors.

Those problems are:

1. Decals Become Distorted

The design from the screenshot above turns into this when exported:

I have experimented with this extensively and noticed that some of the compression comes just with the exporting, but most of it comes from the compression.

Here is the code I’m using to render the decal:

<Decal
    position={[x, y, 0.1]}
    rotation={[Math.PI, 0, Math.PI]}
    scale={[scaleX, scaleY, scaleZ]}
    opacity={1}
    map={print}
    map-anisotropy={16}
/>

It is rendered onto the model’s body <mesh>.

I tried using a polygonOffset, but that didn’t work.

@vis_prime on Discord suggested running a loop and offsetting the decal’s vertices, but I’m still working on this.

I get the following console warnings when exporting:

THREE. GLTFExporter: Merged metalnessMap and roughnessMap textures. 
THREE.GLTFExporter: Creating normalized normal attribute from the non-normalized one.
[KHR_draco_mesh_compression] Skipping Draco compression on non-indexed primitive.

2. Color Changes

As you see from the above screenshots, the color is faded when compared to the one rendered on the canvas.

When using darker colors, the exported versions are much more glossy.

(note how the decal is also more faded)

I’m not entirely sure what to make of this and have few indications of what might be causing this.

3. Using Text Prevents Exports

When rendering text (as a decal) and trying to download the model, I get the following type error:
TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLCanvasElement or HTMLImageElement or HTMLVideoElement or ImageBitmap or OffscreenCanvas or SVGImageElement or VideoFrame)'. window.console.error @ next-dev.js:24.

Here is how I render text decals on the canvas:

<Decal
  position={[x, y, -0.5]}
  rotation={[Math.PI, 0, Math.PI]}
>
  <meshBasicMaterial
    transparent
    polygonOffset={true}
    polygonOffsetFactor={-10}
  >
    <RenderTexture attach='map' anisotropy={16}>
      <PerspectiveCamera
        makeDefault
        aspect={1 / 1}
        position={[0, 0, 1]}
      />
      <Text
        color={hex}
        fontSize={size}
        font={`/fonts/${font}.ttf`}
      >
        {content}
      </Text>
    </RenderTexture>
  </meshBasicMaterial>
</Decal>

I’m not really sure what to make of this as it almost seems like it’s an internal issue, but I feel like something might be wrong with the structure of my text decal.

---

I’d be thankful for any type of input as I’m trying to debug these issues.

Thank you everyone for your help!

• Daniel

[Daniel211]

@vis_prime on the Discord helped me figure out the problem of the decals looking fizzy.

The draco export probably causes an overlap in vertices and leads to the distortion.

The solution is to loop over the vertices and move them a little further away:

let offsetAmount = 0.1 // adjust this value (can be negative too) 
const positions = geometry.attributes.position
const normals = geometry.attributes.normal
const offsetPosition = new Vector3()
const offsetVector = new Vector3()

for (let i = 0; i < positions.count; i++) {
    offsetVector.fromBufferAttribute(normals, i)
    offsetVector.multiplyScalar(offsetAmount)
    offsetPosition.fromBufferAttribute(positions, i) 
    offsetPosition.add(offsetVector)
    positions.setXYZ(i, offsetPosition.x, offsetPosition.y, offsetPosition.z)
}

// tell three js position attribute has been updated 
positions.needsUpdate = true

I’m still trying to figure out how to fix the color and text issues, but those are less pressing issues.

[Fennec]

For the color issue, you may find some clues in this thread

Updates to Color Management in three.js r152

The upcoming release r152 will contain changes affecting color and lighting. These updates enable a “linear workflow” by default, for better image quality in both physically-based and non-physically-based scenes. By “linear workflow”, we mean that any sRGB input colors (as found in most textures, color pickers, CSS, and HTML) are converted from sRGB to a linear working color space for rendering, and the rendered image is converted back to sRGB for display. What does that mean for me? These ch…

[orion_prime]

yeah ,read the above post and experiment with texture’s encoding/colorSpace , general rule : diffuse/emission textures should be srgb rest all should be in linear encoding/colorspace

and for the text thing
stick to meshStandardMaterial for all the meshes which you will export as it’s the most widely supported.

if you are applying a renderTarget’s texture on a material it might not get exported directly (you might need to draw the rendertarget output into a html canvas and save a image from there.(better to start a new topic for this)

[donmccurdy]

It’d be helpful to see what the material is that you’re exporting – what type, what parameters, and how are any textures loaded.

[Daniel211]

Thank you for everyone’s input!

I took a step back and updated to r154 (was r145 before) and that mostly fixed the color issues.
The decals had to be refactored a bit and I changed the material, as suggested. It’s working quite well now

There might still be a hint too much metalness, but I’ll get back to that later.

I will probably make a separate thread for the text problem later. For now I’m just blocking the export function when the text feature is used.

Again, thank you so much for your help everyone! Really glad that it’s finally working properly.

[EatEddy]

Hey Daniel, I am so glad I found your question here because I have been working on exporting into a GLB/GLTF as well and the decal distortion/z-fighting has been doing my head in too!

Literally been pulling my hair out trying to fix the decal. Did you settle on a simpler solution or in the end was your fix to loop over the vertices? I am unfortunately a mega beginner and it doesn’t seem exporting is something people do often!

Thanks,
Edwin

[trusktr]

@Daniel211

I took a step back and updated to r154 (was r145 before) and that mostly fixed the color issues.
The decals had to be refactored a bit and I changed the material, as suggested. It’s working quite well now

Can you please share what refactors you made, and from what material+properties to what material+properties you migrated to? That would be very helpful for people reading this, to “pay it forward”.

[Daniel211]

Most of it was just updating to a newer version of three. If you read the changelogs between 145 and 154, there is one in there that specifically addresses this.

I can’t remember exactly how I refactored the decal at the time, but here is what it looks now:

<Decal
      position={[x, y, z]}
      scale={[scaleX, scaleY, scaleZ]}
      rotation={[Math.PI, yRotation, Math.PI]}
      map={print}
      debug={debug}
    />

I also learned that the rotation has a large effect on how the prints look. Often if the decals look bad, it is because of the rotation. Make sure to match that of the model it is rendered onto.

When it comes to the material properties I’m not entirely sure since I don’t have insights into any of the modelling process. I do remember that we set specularIntensity to 0, but I’m not sure if this was related to the problem at hand.

I hope this helps. Sorry I didn’t see it earlier.

And @EatEddy: sorry, I’m also just seeing your post right now. Hope you were able to fix it. The version update and the loop were the most relevant points in getting it to work.

[Danilo_B]

Hey @Daniel211, I am working on something similar and I am experiencing issues with rendering the .glb model with the decal applied to it. Could you maybe share the part of your code that handles exporting in more detail?

I have noticed you’ve used the useRef hook to get the modelRef for the export, however when I try to mimic what you’re doing I can’t get it to work.
I added this as my function for downloading via the GLTFExporter:

  const download3DModel = async () => {
    if (!modelRef.current) return;

    const options = { binary: true };

    new GLTFExporter().parse(
      modelRef.current,
      exporterCallback,
      errorCallback,
      options
    );

    async function errorCallback(error) {
      console.log(error);
    }

    async function exporterCallback() {
      const io = new WebIO().registerExtensions(KHRONOS_EXTENSIONS);

      const arrayBuffer = await io.readBuffer("/models/mod_cup.glb");

      const doc = io.createDocumentFromBuffer(arrayBuffer);

      await doc.transform(draco3d());

      const optimizedBuffer = await io.writeBinary(doc);

      const optimizedBlob = new Blob([optimizedBuffer], {
        type: "application/octet-stream",
      });

      saveAs(optimizedBlob, "custom_design.glb");
    }
  };

I have the ref attached to my

    <group {...props} dispose={null} ref={modelRef}>

And I have the decal applied and visually looking correct:

  <mesh geometry={nodes.Mesh_1_26.geometry}>
        <Decal
          position={[0, 0.05, 0]} 
          rotation={rotation}
          scale={scale}
          map={useTexture(image)}
        >
          {/* <meshStandardMaterial
						map={texture}
						toneMapped={false}
						transparent
						polygonOffset
						polygonOffsetFactor={-1}
					/> */}
        </Decal>
      </mesh>

I hope you’ll be able to help

Thanks,
Danilo