ZBrush Vs. Xnormal Bake Aberration Query. Thank You.

Pwwka

Thank You.

After four full working days of hand-wringing and research I've finally caved: I cannot solve this problem on my own.
Apologies if this has been covered, but I really cannot find a solution anywhere.

I cannot figure out what's causing the artifacts in the 1st normal map above.

Viewed in DirectX/Stingray 3DSMax viewport 

PIPELINE: ZBrush>GoZ>3DSMax(ForUVs)>GoZ>ZBrush(ForBake)

I flipped the Y / flipped the UV on ZB export.

I'm since aware that there could be more padding in the UV islands.

At no point were there smoothing groups applied to base mesh until applying the Normal Map in 3DSMax.

ISSUES:

a) Wireframe clearly visible on top mesh (this problem's been solved by employing xNormal for the bake, but I still would like to know what was going on - the ZBrush Map exporter would be a great asset if it worked for me).
I assume it has something to do with smoothing groups?

b) Some areas of the mesh seem to be flipped, as though there were flipped normals. There are no flipped normals.

Polycount, any help at all would be greatly appreciated: I'm at a loss.

cryrid

Issue A is a classic issue between 3DSMax and Zbrush. It has a tendency to split every single face on the UVs, and then flip all of those individually. So while it looks like the same UV set to human eyes, it's all disconnected and so you get visible 'wireframes' on bakes. 

I wouldn't worry about it because I wouldn't recommend baking inside of Zbrush to begin with. Zbrush doesn't use vertex normals for its bakes, you get no control over which tangent basis to use, and you are limited to using subdivision levels on the same mesh rather than different models for the high and low poly. 

You'll probably want to apply smoothing groups before baking a normal map. If you change them around after the map is baked, then you're changing the vertex normal information that the normal map was created with/for. 

Pwwka

This is exactly the explanation I was looking for. I wasn't getting any hits for "ZBrush/Tangent Basis", and this is exactly the issue. I can't thank you enough, Cryrid. I suppose it was too convenient to be able to just hit "export all" and go make coffee.

Now if anyone can solve "Issue B" I'll be sucking diesel.

Scruples

Issue B, your green channel is -Y and being interpreted as +Y, solutions include Baking as +Y in Xnormal, inverting your green channel in Gimp/Photoshop or telling the program to interpret as -Y (that one isn't possible in max).



Shortlist of programs and their (default) normal map swizzle
3dsMax +Y
Blender -Y
Maya -Y
Substance +Y
Unity -Y
Unreal 4 +Y
Zbrush -Y

There is also the possibility of it being a gamma issue where the normal map gamma is being processed as if it was a diffuse texture when loaded. I don't have max installed right now but it should show the gamma when you are loading the texture.

Pwwka

Thanks. 

I have both baked and Photoshop "hand-swizzled" the normal maps accordingly, and I will continue to look in that direction. I had gamma/LUT correction switched off (Max2016), but will play with the settings a little more. It could be that my islands are just laid out badly with respect to U/V axis(!)? 
I'll be sure to post as soon as  I solve it.

I'm beginning to understand why the industry runs on The Pipeline.

JedTheKrampus

You can still bake normal maps in Zbrush if you bake them in world space and convert them to object space in an external tool like Xnormal or Handplane. This is useful if you used HD Geometry and don't want to have to jump through hoops to export it, for instance.

Pwwka

Nice one. I hadn't thought of that workaround.

Pwwka

I've pursued the Object-Space NM conversion method using both XNormal and Handplane. After >100 tests I cannot get either to produce a result. I used both a Lopoly Obj from Max and Zbrush. The Object-Space Nmap input was produced in ZBrush. 

The workflow of exploding models just isn't practical for complex models in one UV space - a simple conversion method seems to be what's called for.

If anyone out there has any insight into the correct settings or approach to convert a Zbrush-produced Object-Space Nmap into a Tangent-Space normal map I would be a very happy man.

Scruples

I still think it's a viewport/rendering issue rather than incorrect normal maps, I'm going to send you a pm with my email and you can send me the lowres+normal map if you want me to check them out for issues.

JedTheKrampus

What if you use the object-space normal map from Zbrush in 3ds Max? IDK how to set up such a map because I don't use Max for rendering much, but it could be worth a try to see if it's a baking problem or a viewport problem.

Pwwka

SOLVED initial problem (I think)

In ZBrush, I completely overlooked "Adaptive Scan Mode" and "Smooth Lores Normals" in Tool>Normal Map. I'm not 100% sure (pending more research) but I think these options might be intended for base meshes originating in zbrush which may not have smoothing groups previously assigned.

Helpful Link (which I found only after I knew what to look for...)

So

An uncomplicated and embarrassingly easy fix which I overlooked. This sent me chasing solutions in smoothing groups/Renderers/Game engines/viewport display issues/UV issues/Channels... blech.

Conclusion

xNormal/Handplane may well be superior/more accurate Normal map bakers, but the workload involved for complex organic/combo meshes seems negligibly massive, no?

I'm ultimately going to go for a "convert object-space Nmap using xNormal/Handplane" workflow, as it seems most adaptable. I'm having problems with it, but I think that's for another thread.

Thank you for your help, Polycount (and if you're a random googler, good luck resolving your problem).