1st Time Noob Normal/AO bake in max==Epic Fail for half a day of work

spectre1130

I don't even know where to begin. I am finally trying to do a HP/LP game model with map baking. I watched the tut on CG tuts+ for the knife in-game model, and have followed it to the tee. No matter what I do, my maps will not bake right. I have reset my x-forms a million times, re-unwrapped objects, etc. etc.! Ao is basically non-existent and my normal map just won't work. I have reset the cage 30 times and reworked it-as of this render I'll show next, the cage is so far out it's stupid. And the uvw's showuing in red? When I select those objects' cage and use the push spinner, it moves inwards instead of out. If I move the spinner the other way, they just don't show period.:poly127:

This is the reason I've never even wanted to attempt this type of modeling. I knew I would suck at it. It's just an easy Medieval style bench-figured I'd start even smaller than my last half attempt at the HP/LP modeling.

:whyme:

without further ado:




And my UVW unwrap with overlap turned on to show what I mirrored.




Any comments/help are godsent right now...thanks,
Damian

trebor777

you need to shift your overlapped parts to +1 U or V (makes no difference).
Keep the cage close to the highPoly.

also can we have a screenshot of your hp and lp ?

spectre1130

I'll get some screenshots up shortly. Gonna merge the 2 exploded meshes into a new scene and rebuild the models. Something I probably should have done in the first place. I saw a guy on another tut have a seperate scene for each model, then merged both into a new scene to do the explode/bakeout. Live and learn I guess. I don't know what you mean about moving the parts +1 though.

Obscura

First, you don't really need to explode your mesh, because you can keep it together during the bake, if you use material ID's to render the different elements(check "hit only matching material ID's" in the render to texture options). If you trying to bake the ao with exploded mesh, you will never get proper ao map, because the pieces are far away from each other.Second, You don't need to use the "symmetried" pieces during the bake.You can detach them, and when you finished the baking, you can attach them back. Third, to me, it looks like you are trying to "symmetry" the top-bottom side of your desk. Maybe I see it wrong, but if not, this is also bad.
Fourth, it looks like your mesh is using hard edges, careless for the uv's splits. You need to split the smoothing along the uv seams.

ps - when you are baking ao for something what will be "on the floor", then its useful to place a plane under the mesh (under, but leave a little empty space between your mesh and the plane) during the ao baking.

spectre1130

Here is my reworked uvw render along with some screens of the LP, HP, and them sitting on top of each other for comparison. I have them also in another scene now with the new uv's as well; but in an exploded form. I had to rework that LP deco piece on the middle brace to fit the HP version better, then had to re-unwrap. Also of note, I did make sure and give all my LP pieces proper smoothing groups.

Mark Dygert

There isn't much that will be captured on the AO and normal passes. You have a lot of long flat surfaces so not much AO or normal detail to capture. AO is calculated when two pieces of geometry have a difference in angles, such as indented corners (like the inside of a box), indents and extrudes.

Aside from the edge padding being a disaster and the background color being set incorrectly on the normal map (thanks to mental ray) it looks more or less correct, IF you are offsetting the mirrored pieces, one tile to the right, left up or down.

If all of the UV shells stacked inside of the rendering area they will render on top of each other, this is bad... You want only one instance to render and the other shells to reference that one. You do that by offsetting all but one shell.


The instanced shells will be outside of the rendering area so they won't be captured, but they land exactly one tile to the right.


A quick way to offset UV's in 3dsmax is to, click #1, and in #2 type 1.0, hit enter and toggle off #1. This will move whatever you have selected over one unit/tile to the right.


Also the background color for the normal map is wrong which is linked to the edge padding being a disaster... With some older versions of max if you are rendering with mental ray and have more than one output map set in the Render To Texture window it will screw up the padding. If you do them one at a time they pad correctly.

Mental ray gets the background color of normal maps wrong because it pulls the background color from the environment background instead of from the background color in the RTT window (which is set correctly by default). I think they fixed that in 2013 and if not hopefully in 2014, it's such an easy fix and such an annoying bug.

Good luck, hope that helps. Everyone runs into snags like this, don't feel bad. Once they punch you in the face, the next time they throw a punch you duck and come up swinging.

spectre1130

Wow thanks for all the info guys. I'll give the moving a try. So I don't really need an explode for this either??? In know that there isn't that much to capture on the normal, but I just wanted to get the more smooth corners onto the LP from the high. I'll post some updates a little later. I was up till 6 am working on this and just got up.

spectre1130

More Noob nothingness going on here. IDK.

trebor777

it's a lot better, but it seems you still have weird stuff going on.
maybe you're not ready for Max Baking.
Something simpler might help. watch that tutorial for xnormal
http://eat3d.com/free/xnormal_overview

this should help. at least it can bake stuff without a cage

Mark Dygert

Can you post the 3dsmax file somewhere like dropbox and link to it?
It seems like some of your stacked pieces are stacked/offset over the wrong shells.

trebor777 wrote:

this should help. at least it can bake stuff without a cage.

Max has the option to turn off the cage and cast rays strictly along the face normals of the low poly, just like xnormal. The advantage of using a cage in max is that you can edit it and tweak it without having to jump in and out of several apps.

I do think it's beneficial to learn xnormal because it has some strengths and you need to know when to leverage it to take advantage of those strengths. But I think its more important to learn proper baking practices in max, it saves a bunch of time and is easier to work all in one program for quite a few things.

trebor777

thanks for that!
I shall hold my tongue forever now
(awesome gif btw)

spectre1130

Okay. I'll post the dropbox link. Thanks guys for helping me with this. I do have xnormal, but don't want to move into that until I'm comfy doing this stuff in Max. I know its good to learn other platforms, but what if I am looking for work down the road and the companies only bake in Max? Anyhow, the file I shared is the newest scene with the HP/LP models reassembled with current unwraps. I have been trying the normal mapping/ao without the explode as mentioned earlier by Obscura. If this is bad I can always start a new scene and explode them real quick. Hopefully if you take a look at it Mark, you'll catch my noob follies.

Thanks!!!!

http://https://www.dropbox.com/s/r78nrblwy6stq2u/bench_hp_lp01.max?m

EarthQuake

Mark Dygert wrote: »

The advantage of using a cage in max is that you can edit it and tweak it without having to jump in and out of several apps.

Also, using a proper cage means no seams where you have hard edges. If you turn off the cage in max, or use the default ray distance setting in XN you'll get visible seams in your normals, ao, etc around any hard edges/smoothing group splits.

Mark Dygert

1) It created a shell and looks like it was baking previous versions to the final bake. In the material editor use the eyedropper to sample the surface of your low poly object and see the shell material.

Before you bake again, apply a new default material and in the RTT window under Baked Material Settings check on Render to Files Only. That way it won't try and create a new material and automatically assign it to your mesh each time you bake.

2) You don't need any kind of light setup to render AO, it is strictly going off of the angle of the geometry. The only time you need lights for AO is if you're going to do a diffuse render with lights and shadows, which technically isn't AO but it can be effective at getting different effects.

3) there are still two pieces that are overlapping, in the editor window go Select . Overlapping faces and it will highlight them.

4) The detail sitting on the crossbar has unique UV's it could be mirrored and offset.

Unfortunately I have the full version and not the student version so saving a file would be useless to you but this is what you're after right?

Mark Dygert

I'll make use of this double post!

Also, using a proper cage means no seams where you have hard edges. If you turn off the cage in max, or use the default ray distance setting in XN you'll get visible seams in your normals, ao, etc around any hard edges/smoothing group splits.

Ooohh good point.

spectre1130

Yes it is. Thank you so much for this help! I will try this when I gety home from dinner. Also, from what you saw in the scene, do I need to do an explode for the normals to bake properly or not???