Home Technical Talk

Trimsheet normals inverted when rotating UV islands

polycounter lvl 2
Offline / Send Message
TheSteamDyer polycounter lvl 2
I find myself stuck with a mistery that is making me loose hours of work without any progress whatsoever figuring it out.
I'm making a modular environment and I'm making some trimsheet textures. In this specific case let's take a door frame made of three pieces (two vertical, one horizontal on top. Everyting is cool and neat until I take a vertical face and rotate it in the UVs to match it to the horizontal trimsheet, and that's when the normal map appears inverted in my scene. The trims were baked with substance painter, the modelling and unwrapping is done in Blender, the final result needs to go to Unreal. Funny thing, if I flip the face normal it appears correct (but then I have a flipped face obviously).
I don't understand why this is happening nor how to fix it, I tried pretty much everything that came to my mind. If I rotate the UV island to be vertical the normals appear correct, but then it doesn't match the trimsheet anymore.
Here's a pic to describe the problem:

Here you can see the UVs of the frame on the left (they are overlapped to match) and the actual result on the right. The seam is in corner and it's evident that the normals don't match. This happens in Blender, but in Unreal I get the same result.
I also tried with a new geometry and this is the result:
if it's flat on the ground it looks correct...

but if I rotate it 90deg on the x axis so that it's standing like the door, inverted normals :D

Can we agree this doesn't make any sense? Or am I missing something?
To be clear: there are no flipped normals, I also recalculated them multiple times, no split normals or custom normals of any kind, the normal map was baked for DirectX but doing it for OpenGL didn't change anything, and as far as I know Painter bakes in Mikkt, in the material nothing is going on (I can confirm that because in Unreal I only plugged the normal map in the normal input at it gives the same result) and the color space is correct (no sRGB). The geometry is not the problem since as I showed I tried with a plane where I just insetted the only face it had and deleted the center. Maybe it's something with Blender? I couldn't try in other softwares like max or maya since I don't own a license.

If you read all of that thank you, finding someone with more experience that can help me figure out the problem is the only card I've left.

Replies

  • wirrexx
    Offline / Send Message
    wirrexx interpolator
    Maybe your uv is flipped? flip it and see if it works.

  • DavidCruz
    Offline / Send Message
    DavidCruz greentooth
    i'll try to add another possibility I've run into on any 3d models, reset X form (in max idk what it is in blender) and maybe double faces on that location ? I've had this its not fun.  Hope you figure it out, glad you did the flip normal cause it was my 1st thought, since it looks simple enough i'd just redo it, probably much faster than waiting.
  • TheSteamDyer
    Offline / Send Message
    TheSteamDyer polycounter lvl 2
    wirrexx said:
    Maybe your uv is flipped? flip it and see if it works.


    I checked, no flipped islands, but fllipping them corrects the normals. The problem is they don't align anymore if I do that
  • TheSteamDyer
    Offline / Send Message
    TheSteamDyer polycounter lvl 2
    DavidCruz said:
    i'll try to add another possibility I've run into on any 3d models, reset X form (in max idk what it is in blender) and maybe double faces on that location ? I've had this its not fun.  Hope you figure it out, glad you did the flip normal cause it was my 1st thought, since it looks simple enough i'd just redo it, probably much faster than waiting.

    I tried, resetting xform does nothing and no double faces either.
    As you saw I can't even redo it, because this problem appears even when I try with a simple plane, so it's a generic issue, not linked to this specific model
  • oblomov
    Offline / Send Message
    oblomov polycounter lvl 6

    You can't just rotate a normal map. The texel values need to change as well because they represent vectors and not colors.
    e.g. if you rotate a red texel (facing right) 90° it should become green (facing up or down, depending on whether this is a normal map following DirectX or OpenGL conventions). Blending and painting normal maps can also cause problems for similar reasons.
    You need tools specialized to work on normal maps to modify the normal map values (e;g in Substance Designer, Painter, NDO, UE4, Blender, ...) or rebake according to your object new orientation.
    (edit : in the OP case, you may be able to get it to work by rotating the tangent space for the left quad)

  • TheSteamDyer
    Offline / Send Message
    TheSteamDyer polycounter lvl 2
    oblomov said:

    You can't just rotate a normal map. The texel values need to change as well because they represent vectors and not colors.
    e.g. if you rotate a red texel (facing right) 90° it should become green (facing up or down, depending on whether this is a normal map following DirectX or OpenGL conventions). Blending and painting normal maps can also cause problems for similar reasons.
    You need tools specialized to work on normal maps to modify the normal map values (e;g in Substance Designer, Painter, NDO, UE4, Blender, ...) or rebake according to your object new orientation.
    (edit : in the OP case, you may be able to get it to work by rotating the tangent space for the left quad)

    I was told the same in another thread! I actually thought about that in the beginning, but I dismissed that theory thinking the conversion from tangent to world space wouldn't be affected by the rotation of the UVs. I did another test and turns out that's what it is. So now I'm wondering if there's any way I can get an horizontal trim on a vertical plane, and if not what the workflow is in these cases.

    Appreciate the meme btw :p

  • birb
    Offline / Send Message
    birb greentooth
    So now I'm wondering if there's any way I can get an horizontal trim on a vertical plane, and if not what the workflow is in these cases.
    There's a way: Object space normal

    All usual caveats apply, but it should work in this case since you're applying something baked on a flat plane onto other perfectly flat planes. 
  • Prime8
    Offline / Send Message
    Prime8 greentooth
    ...
    I also tried with a new geometry and this is the result:
    if it's flat on the ground it looks correct...

    but if I rotate it 90deg on the x axis so that it's standing like the door, inverted normals :D

    Can we agree this doesn't make any sense? Or am I missing something?
    ...
    I agree, it doesn't make sense. I cannot recreate this either, it's working as expected for me.
    Did you rotate in edit mode or object mode and applied transformations?
    Which version of Blender are you using? I guess there are no modifiers on the object?!

    Would be best if you could share an example file.
  • birb
    Offline / Send Message
    birb greentooth
    @Prime8 Did you rotate the object (outside edit mode) in your recreation? You should see it cropping up once you do. What steps exactly did you do?
  • Prime8
    Offline / Send Message
    Prime8 greentooth
    @birb I tried rotating in object and edit mode in every way I can think of, but it didn't invert any normals, as it should be.
  • poopipe
    Offline / Send Message
    poopipe polycount lvl 666
    This should be entirely straightforward as Prime8 says. 

    Tangent normal direction is relative to UV orientation, not geometry - as explained in detail by FrankPolygon above, and as experienced by the entire game art community for at least the last 15 years.


    My guess would be that some sort of wanky transform on the mesh is causing the issue (or something weird in whatever shader is applied) -  I know next to nothing about Blender so can't suggest any remedy 
  • WitchDev
    It's guaranteed just the light coming from a different direction relative to the surface after rotating. If you rotate the light as well as the geometry it'll look the same again. There are no normals inverting.
  • Kanni3d
    Online / Send Message
    Kanni3d interpolator
    Try giving it a preview in another realtime viewport like unreal or marmoset? Could just be a viewport issue in blender this entire time :tongue:
    .
  • MatheusDalla
    Blender view is fine, the thing is, the normals are relative to UV, so it's uv's are flipped


    Also, make sure the material is setup properly: (Non-Color)

  • Gannon
    Offline / Send Message
    Gannon greentooth
    Tried it out myself, I get the same results at @TheSteamDyer. Here's a maya scene for others to check it out. Keeping an eye on this to see what the quirk is.
    https://www.dropbox.com/s/nrphhrexfc63daw/wonkytrim.mlt?dl=0


  • TheSteamDyer
    Offline / Send Message
    TheSteamDyer polycounter lvl 2
    @FrankPolygon this is gold, thank you! I've thought about it more since the first two answers, and indeed I came at the conclusion that it didn't make sense. If normals couldn't be rotated then they would be world space normals, which are an entirely different thing, and it would mean that any object with a normal map would break upon rotation. So I looked at it more, and it turns out  that the shading is actually correct, but as you said the corner sharpness caused it to look unnatural and with a shading error. Here you can see a test I've made just now: on the left there's the flat geometry with the normal map, on the right actual geometry with no normal, and the light falls in the same way on both models

    @Prime8 Object mode and did not apply, Blender 2.83, no modifiers. Attached is the example file.
    To answer to the other posts: The geometry was a simple plane, inset to get the frame, deleted the central face, seams on the corners, unwrapped. No inverted UVs (I checked) and I already tried it in Unreal, where I get the same result. Ultimately I believe @WitchDev is right: it's just lighting making it look off. You can all take a look in the attached file.

  • srsface
    Offline / Send Message
    srsface polycounter lvl 2
    I've had similar f'd up mesh orientation dependent normals with some random assets that I've had to deal with inside UE4. Rotating them in the world normal debug view you could clearly see that the normals change based on the angle. Importing the mesh to Max and resetting explicit normals (and then re-doing correct smoothing groups) fixed it for me. Don't know how Blender handles that stuff.
  • Prime8
    Offline / Send Message
    Prime8 greentooth
    Now it makes sense, I recommend to develop a habit of looking at your stuff in different lighting conditions frequently.

    On a side note, you can use a RGB curve to change the normal swizzle.

Sign In or Register to comment.