Normal Map problem
polycounter lvl 14
I'm having a big problem with my normal maps when I render them in XNormal, but they look fine when I render them in Max.
In Max:
Now when I do the render in Max, everything looks fine. No seams, everything is nice and smooth and clear.
In XNormal:
But when I render the maps in XNormal, I suddenly get all these weird artifact things. Not only that, but there are gigantic seams where the model seems to cave in on itself (you can see it on the leftmost rounded edge). And just ignore the double ring on the bottom, that's because I exported it unexploded for this particular bake just to see something, but I get the same result minus that ring in an exploded bake.
Also, for some reason XNormal seems to ignore any cage I have on the model. No matter what I do with the cage, I get the same bakes with "use cage" checked or unchecked.
Thanks for the help! (And sorry for not putting this in the XNormal thread, I just thought it would get answered faster as it's own thread)
In Max:
Now when I do the render in Max, everything looks fine. No seams, everything is nice and smooth and clear.
In XNormal:
But when I render the maps in XNormal, I suddenly get all these weird artifact things. Not only that, but there are gigantic seams where the model seems to cave in on itself (you can see it on the leftmost rounded edge). And just ignore the double ring on the bottom, that's because I exported it unexploded for this particular bake just to see something, but I get the same result minus that ring in an exploded bake.
Also, for some reason XNormal seems to ignore any cage I have on the model. No matter what I do with the cage, I get the same bakes with "use cage" checked or unchecked.
Thanks for the help! (And sorry for not putting this in the XNormal thread, I just thought it would get answered faster as it's own thread)
Replies
If that is what's happening, then xNormal is baking from a messed up highpoly and you get those weird errors.
I suggest re-exporting from Max to xNormal but uncheck all optimize options, then try baking it again.
Though all of that is just a guess
You're also baking with straight rays and not a cage (I believe), since the lower most ledge on the chest has accidently caught the normals from the lid of the chest, and the area behind the ring has baked the ring aswell.
But, the highpoly is at fault here.
Eld, in the first post I mentioned that the ring was just me quickly exporting with an unexploded object, so the ring had nothing to do with the errors, it was just me being lazy. But I see what you mean with the bottom ledge now... But even when I set up my projection cage in Max and check it in xNormal, it doesn't seem to be recognized and makes no difference. Unless I have the idea wrong about cages, and the cage needed in xNormal is different from the projection modifier in Max?
EDIT: Okay, I tried turning off optimization options and I get identical results. But now I understand what it is doing. If I collapse my high poly (Turbosmooth and Editable Poly), I get An Editable Mesh that has no differences in it. But if I then convert this to Editable Poly, my high poly suddenly has all those errors that get baked (And they are smoothing errors I think; from a certain angle they dissapear). I've not had this problem before, does anyone know how to correct it?
Have you tried using mesh-smooth instead of turbosmooth?, I'm not too sureit will make a difference, but there might be some ways that they work different in terms of poly and mesh.
Once you get to the step where you have a fully collapse detailed smoothed model, then there shouldn't be any more issues.
As with the cage, someone with more knowledge in xnormal have to tell you how cages are transfered to xnormal, but they should be editable in xnormal if I remember right, and once you bake with the cage in xnormal you will get rid of the edge-artifacts.
I then tried the SBM exporter which also worked a bit, but had its own host of problems. It allowed me the use of my cage, but for some reason it ignored my exploded mesh on frame one and used the unexploded mesh on frame zero (Which is easy to work around but just annoying). It also gets rid of the nasty edges, but on every UV seam it makes a very clear seam in the normals... I think I'm just gonna give up on this one. I must have messed something up on it somewhere.
Thanks for all the help though guys
=======================================================
This is my method with some instruction from Santiago Orgaz;
1. Detach all the high and matching low poly elements, group them together and move them far away from each other, then ungroup the pairs.
2a. Add an STL check modifier on all meshes in edit poly mode, do all the tests individually and make sure there are no errors.
*Do some renders if you are unsure and or turn off smoothing modifiers to go in manually fixing errors like double vertices, edges, polies.
2b. Check all your uv's, make sure they are as contiguous as possible with no stretching and all vertices have been welded that require it.
2c. Set all the low poly smoothing groups and make sure they render correctly or appear error free in the viewport when rotated at various angles.
*You would be best to have all the lp elements on a single group where possible.
3. Reset xform on all the objects, centre the pivots and deselect all subobjects.
4. Collapse or convert all the meshes to editable mesh including any modifiers, finally make sure all stacks are empty.
5. Change renderer to Mental Ray and select your 1st low poly object.
6. Goto render to texture, enable projection modifier and add high poly target, reset the cage in the rollout and put in a push value of around .5 to 1.0 and press enter.
In rtt add a largish normals map 1024/2048 and do a test render, iron out any problems by tweaking the cage or altering the low poly.
*Make sure the low poly is always slightly encompassing the high poly and all should be okay.
7. Repeat this process for all the object pairs.
8. Again make sure all objects are editable mesh with empty stacks but leave the low poly projection modifiers on.
Select them all individually and goto file, export selected, SBM format.
On the low poly with the cage tick ALL the boxes except for smooth normals. On the high poly I just tick export normals but that's probably just habit.
9. Once inside xnormal load up all the high and low objects;
On the high meshes uncheck ignore per vertex ao for a more precise result, it takes longer but is worth the wait! Make sure use exported normals is selected and visible is checked.
On the low meshes make sure all the boxes are unchecked except for visible.
10. Baking options - some of this may be different if you use 32 bit but for 64 bit full quality I use this setup;
Main: Closest hit checked, edge padding 4, bucket size 64, antialiasing 4x
Maps: Normal map - tangent space, leave other options as default.
Ambient occlusion - rays 2048, distribution uniform, bias 0.001000, spread angle 179.5, attenuation 1.00000|0.00000|0.00000
*Providing your 3ds max tests work okay you should be confident enough to set full quality baking in xnormal so sit back and let the magic happen:thumbup:
BUT! I'm pretty sure I figured out my problem, and this is probably very important to any others with this problem (The problem of the artifacts appearing once collapsing to an editable mesh with turbosmooth on). It seems that, with Isoline display on in turbosmooth, you will get bad artifacts when you collapse the entire modifier stack back into an editable poly. Now, if you turn off isoline display before collapsing, this problem dissapears!
I'm sorry if everyone already knew this and I'm just being a terrible noob, but I'm quite excited with my discovery. Might save me some headaches in the future