A solution to bad baking in 3ds max; xnormal?

stickadtroja

so i have for a while been really bothered by seams at the uv bordes in the normal map. i have used 3ds max for the baking, and it concerns orgnanic stuff like creatures.
so ive checked online for solutions but havent found any. after a while i thought, "you always get seams to some degree" and accepted it, trying to hide them in smart places.
but now, i got a couple nasty ones, and decided to try xnormals instead. and yes, almost all of the seams disappeared! so im happy!
but, should i let it rest now, accept that 3ds max doesnt work properly for me, and use xnormal for the rest of my days? does anybody else have the same experience?
im kind of afraid that im taking the "easy" way out, and that it should be possible to do in 3ds max, and that i miss out on learning if i dont solve it there...

Joao Sapiro

without pics is hard to help you out, there are lots of variables on how a bake is turning bad. take a look at this thread and read every reply :

http://www.polycount.com/forum/showthread.php?t=81154

Quack!

Xnormal and 3dsmax can bake almost identical normal maps. The issue here is probably a lack of understanding what is causing the error.

Like Joao said pictures will help.

Here are the images we could use to help you best

2D Normal Map from max
3D Model with no texture, just smoothing groups.
3D Model with ONLY the Normal Map applied with a spec value of 70 or higher and a gloss value of 40 or lower.
Your UV layout.
Are you using a cage or ray distance/projection?

stickadtroja

thanks guys for the suggested help, but i cant really show any pics, since its on a unreleased project. going to check with my boss though, it could really help me out.

my question was more like, is this common, to get way better results from xnormal, or am i only bad at using 3ds max? i guess quack said it, it should be possible in both, im just suprised that i had so much trouble with max and then it works almost perfectly in xnormal instantly.

stickadtroja

aight, heres some pics;






grateful for any help.

and joao; ive read a lot of stuff in that thread, have tried some, but havent solved it yet.

Edit: im using a cage in 3ds max, and ray distance in Xnormal

Joao Sapiro

where are you previewing the model ? in 3ds max ? also be sure to read the part about bitangents , max tangents in older versions arent syncronized to their own baker , so that means that when you preview them on the program it will give you some weird shading errors sometimes , also try to separate the uvw islands as separate smoothing groups in the model, see if it helps !

stickadtroja

yes, i use xoliul shader 2.
the thing about bitangents, does it happen in max 2012?

and yeah, im going to try it with smoothing groups, but i thought that was only the other way around? that you have to split uvs when having different smoothing groups?

stickadtroja

so i still have some trouble with uv seams on normal maps for organic things. i get seams on the splitted uv borders in some places;



i checked all kinds of stuff, like normals, smoothing groups and if there is unwelded verts. but nothing.
is it possible to avoid seams completely? or do they always occur to some degree in split uv borders?
is the solution to keep the whole mesh as seamless as possible, stiching together the legs with the torse and try to fit it in the uv square as good as i can?

pior

In an ideal world, the one and only way to avoid such seams is to have a viewing environment and a baking environment that are completely synchronized (and there is much more to that than just the normalmaps "looking good").

Unfortunately, almost none of the display environments we use (CG software, games engines) are synchronized with themselves or each other, with very rare exceptions. As a matter of fact the only one that I know of is IDtech, which has its own internal baking tool. Besides that Maya has been generally pretty good over the years when it comes to displaying its own bakes, but the new normals averaging options introduced in 2013 kindof threw a wrench in that ; and bakes made in Max where originally perfectly synchronized with the scanline renderer, but not the viewport. I am not sure if that is still the case as I have not used any of the newer Max releases.

Everything else is just a guess, with Handplane being a fantastic tool to actually make things work. Even if it is not directly related to your problem, this video might clarify a few things. It deals with Source Engine and warped surfaces, but the same idea applies to the Max viewport showing hard edges at seams : if the baking tool is not synchronized with the display environment, you will get artifacts of one kind or another

[ame="http://www.youtube.com/watch?v=QeP-tOKryxU"]Source engine comparison with handplane - YouTube[/ame]

You have to be aware of all that in order to react accordingly, which sometimes means using the necessary measures that you mentioned : hiding the seams as much possible to minimize the problem, and so on. It is indeed a shame that an environment like Max is not capable of properly displaying its own maps. And of course things get even worse when exporting the asset to a specific engine ...

What is the ultimate destination of your asset ?

stickadtroja

hey thanks for the answer.
this confuses me a bit though. when you say that the baking and display software has to be synkronised, it makes it sound like there is no objective "right" normal map. i thought that, if i bake it correctly it doesnt matter where its displayed, it would still look right.
is this totally wrong? is normalmaps always dependent on the display software?

this is intended for a game, and the result i get from xoluil shader 2 in 3ds max looks the same as the ingame engine.

also, i somewhat solved it by having the uvs of the legs splitted from the body further up, so the uv seam is moved away from the folds of the leg. there are still visibly seams, but its not as apparent. its not the best though, since it will make later painting seams more visible.

ZacD

You can model, uv, and use smoothing groups in certain ways to avoid normal map shading errors, but if you have a synced workflow, you do not have to do that.

pior

it makes it sound like there is no objective "right" normal map.

It's a bit more complicated than that, as it involves how specific environments handle various tangent basis. This is precisely what the Handplane video is demonstrating

i thought that, if i bake it correctly it doesn't matter where its displayed, it would still look right.

Unfortunately you assumed wrong This is why there are tons of normalmap baking issues and artifacts in many games - even in AAA titles with huge budgets. The problem occurs when the Tech Art teams settles for "hmm, that seems to look good enough" instead of precisely establishing art pipelines based on accurate specifications. And on top of that, in-engine optimizations can cause issues too. If I remember correctly this was one of the reason why some UT3 assets had some obvious seams showing.