[3DSMAX] Baking Object Space NM with 3DSMAX and converting to Tangent with XNormal.

OccultMonk

Converting an OSNM from 3DSMAX to Tangent in XNormal gives strange results. I tried inverting Green Channel and even Red Channel. I Swapped Green for Blue, Red for Blue, Green for Red, but nothing works. 

Can anyone bake an OSNM in 3dsmax and convert it successfully to Tangent space in XNormal? I really want to use 3dsmax since I have a custom baking script for adding shells and setting everything up. 

Handplane worked well, but does not support UE4 4.7+

Obscura

Well. Now there is a support for MikkTspace in Unreal, so you can make perfect normal maps in Xnormal or Substance. 3dsmax bakes works also, but you need the smoothing splits, or some bevels/supportloops.

As far as I know Handplane doesn't support UE4 at all... Also I don't see the point of bothering with Handplane becuase you have like 6 other options for getting good results.

OccultMonk

I really want to convert an Object Space Normal Map from 3DSMAX in XNormal to Tangent MikkTSpace. This should give good results (as good as baking in XNormal). Only I can't get it to work. I use a custom script, which helps me with baking.  

TorFrick uses the same approach for baking in MODO: Generate an Object Space NM in MODO and use XNormal to convert the OSNM to a Tangent normal map. 

OccultMonk

Are there any other programs like XNormal and Handplane that can convert an OSNM to Tangent? MikkTspace for UE4

OccultMonk

Can someone please give a few tips why this isn't working? Maybe try it out? It seems like a good workflow. But I have already spend almost a day testing.

Obscura

If you have xnormal open already, you could have just bake there instead of trying to get this working. It could have take like 2 minutes to set up everything, instead of a day...

OccultMonk

I am baking a lot of separate objects, and have a script that automatically adds the projection cages, and also links the high and lowpoly objects among other things. I also want to be able to use the rounded edge feature in 3dsmax. I only need the OSNM to Tangent working. There is no reason why it should not work. 

musashidan

OccultMonk said:

Are there any other programs like XNormal and Handplane that can convert an OSNM to Tangent? MikkTspace for UE4

Cryengine has a standalone application called Nomaps that does this. It might be time to follow @Obscura 's advice and get with the mikkt crowd   I know it's painful giving up an existing workflow that you're comfortable with but using mikkt does away with a lot of these old hacks that used to be necessary. 

OccultMonk

I still think/hope it's something minor that can be solved.

OccultMonk

Cryengine has a standalone application called Nomaps that does this. 

Thanks, Does this standalone app also work for UE4? Is it truly MikkTSpce? I can't seem to find the installer for it. Must the full Cryengine be installed first? Can't I download it separately?

I am still really hoping for someone to know the answer on my original question. What is going wrong. Should I switch a R/G/B channel for an other R/G/B channel. Should I invert certain channels? Or should I do something different while baking. Some tips would be great. :-)

musashidan

I think it comes with the Cryengine install unfortunately. It just opens independently by the looks of it.

As for your original issue, I ran the same test as you last night and got the same results. No matter what I tried the conversion didn't work as expected. Maybe you should contact the dev? That tool might be bugged in 3.19

AlecMoody

This isn;t a channel flipping issue- Your shading is rotated. Its probably a y/z up difference. You should be able to fix this by rotating your low 90 degrees, freezing transforms, and then sending that out for tangent space conversion.

OccultMonk

musashidan said:

I think it comes with the Cryengine install unfortunately. It just opens independently by the looks of it.

As for your original issue, I ran the same test as you last night and got the same results. No matter what I tried the conversion didn't work as expected. Maybe you should contact the dev? That tool might be bugged in 3.19

Thanks for testing it out! Maybe it's a bug in XNormal, but it seems to work for OSNM bakes from Modo (Videos TorFrick).

AlecMoody said:

This isn;t a channel flipping issue- Your shading is rotated. Its probably a y/z up difference. You should be able to fix this by rotating your low 90 degrees, freezing transforms, and then sending that out for tangent space conversion.

I have thought about that too. That's why I copy and pasted the channels (not flipped them). Blue in Green, Red in Green. Nothing worked. Freezing transforms before baking and export or only before export? I have already exported it with FBX (y-up and z-up) But that did not help.

huffer

I succesfully converted object space to tangent space with 3.18, and if I remember correctly it was a both a Y/Z orientation and a channel flip, but I can't remember unfortunately, maybe ask on the polycount slack channel. Try baking object space from 3dsmax with Y flipped and exporting mesh with Y up, then Z up, and combinations of these.

OccultMonk

Thanks, I got it working when I rotate the object Y/Z and flipped the Y channel. Freezing transforms was not needed. unfortunately it does not give a result that is the same/as good as MikkTSpace bake directly. It is better than the NM bake from 3dsmax, but still has shading errors. Is this a limitation of 3dsmax OSNM bakes or does this also occur when converting OSNM form modo in XNormal?

AlecMoody

AlecMoody said:

This isn;t a channel flipping issue- Your shading is rotated. Its probably a y/z up difference. You should be able to fix this by rotating your low 90 degrees, freezing transforms, and then sending that out for tangent space conversion.

I have thought about that too. That's why I copy and pasted the channels (not flipped them). Blue in Green, Red in Green. Nothing worked. Freezing transforms before baking and export or only before export? I have already exported it with FBX (y-up and z-up) But that did not help.

You can't copy and paste channels to rotate shading.

OccultMonk

I got it working!!! If you don't export FBX with tangents and binormals. You rotate the mesh Y/Z and invert the Y. The bake convert very good!