Normal map value reduce on dark texture

Hello everyone, can anyone tell me why, when I switch the texture from a lighter color to a darker color, the normal map starts looking much less realistic. Can anyone help me understand why?

  • Light collar shows woven texture. Texture is otherwise flat clipart and faux text gradient.
  • Light collar is more plausible on light background. Maybe harsh light without base shadow, not floating.
  • Black collar looks like space cutout with flying pumpkins. Psychologically the foreground recedes.
  • tweaking Material shininess factor may help find a reasonable balance. Some fibers are borderline plastic.
  • at work we have glow-in-the-bark pumpkin collars, which would be emissive.