Sand crap over my 3dsmax AO bake

kanga

max 2010
win 7 rc

Hi
First AO bakes with mentalray. I am very impressed with the quality. Only I have grain stripes and splotches over the bake. I thought this was the element settings so I put them up to 100 samples and increased MR's sampling to 4/16. Took a while but you can see the result below with the local dirty bits. There are loads of settings but maybe someone has had this also and have a tip.

Thanks in advance.

pior

It looks very directional tho, you might have to change your settings, as it will bite you in the back pretty quick!!

kanga

Thanks pior
Yeah, you right about that. Would be nice to get this right.
Forgot to mention that this is a bake from the high poly to the lo cage. I did a quicky bake from the normal map to the cage and that was as clean as a whistle,... buuuut much lower quality. I couldn't find others with this problem, that means I am doing something wrong and it's going to be hard to find the setting, or thingy that's faulty.

Cheers

conte

try this(not mine)

kanga

Hey Conte!
Woo hoo, There are a couple of really key settings in there that I am missing with my bake. Thank you for the great step by step. Got one in the oven now and as soon as its finished Going to pump in this stuff. Wonderful

Cheers.

kanga

Used the settings above and got an even more beautiful render, mouth watering in fact The problem remains though, the sand is still there. It would take a lot of work to touch it out.

I even isolated the torso and baked it separately as was suggested on 'my mentalray' forums. Still no joy. The rays are at max and it took a loooooooong time to bake, still sand.

Very frustrating but thanks very much for the outstanding help.

Cheers

conte

try to off antialiasing filters and render without it- double size image.
after just resize it and get smooth result.
i know i know, it will take forever to bake ao in such dimension, but worth to try)
good luck, kanga!

-Askhat.

[HP]

People still use Max for bakes?

Mark Dygert

^ All the time. What else would you use? Xnormal? Doesn't Xnormal give great results but take a long time and have a crazy interface, not to mention its another app you have to export to and from? I thought it was mostly for Maya guys that found baking with transfer maps a chore? I hear they've upgraded the Maya transfer maps system since the last time I used it so maybe thats not the case?

Anywho whatca using?

@ kanga: Like Pior pointed out, make sure you're opening the file it saved and not saving what it displays in the render preview window.

The render preview window is a full render of the object including directional lighting and artifacts. Even in a scene without lights it still captures the viewport lighting.

[HP]

Well yeah, xNormal is a good alternative, for like 80% of the stuff I do. (Env Art)
I only use Max baker when the object I'm baking is a bit complex and requires a lot of fiddling around with the cage, test renders, etc. I only tried to import a custom cage into xnormal once, and It's a nightmare.

With that being said, xNormal normal maps always look better to me. I do agree that it takes a few more seconds compared to Max's baker for normal maps, but it's nothing massive.
Although AO render may take several minuts, especially with AA.

kanga

conte wrote: »

try to off antialiasing filters and render without it- double size image.
after just resize it and get smooth result.
i know i know, it will take forever to bake ao in such dimension, but worth to try)
good luck, kanga!

-Askhat.

Awwwwwww, so logical is why I never thought of this!
Gonna give this a go. I will just fire up my xbox in the meantime ha ha! Lately been killing off lots of Italians in GTA, which is counter productive because there can never be too much spaghetti in the world.

[HP]
These are my first bakes, but I did download the latest rc version of xnormal. It is really quick but even after measuring the ray distances and playing with other settings the result is very washed out and nowhere near the depth and range I am getting in mentalray. On my 05 bake test. Still messing with it.

Thanks very much for the guidance
Cheers

conte

here is good xnormal ao bake tutorial, btw

kanga

conte wrote: »

here is good xnormal ao bake tutorial, btw

That looks great conte.
Going to use this right after dinner

Thankyou.

Zack Fowler

I've seen this issue before on my own stuff. Trying to remember what I did to fix it.

mental ray seems to have problems when your low poly and high poly are perfectly coplanar and overlapping, regardless of cage, so maybe try dropping something like a Push modifier, slide it underneath your Projection modifier, and set it to .02 or .05 and see if that changes things.

Shogun3d

I noticed RenderHJS's UV Toolbox utilizes Mental Ray rendering methods, is there any particular reason this is preferred over Lighttracer or Skylight bake method? I think xNormals uses a lightracer algorithm but i'm not too sure.

Any info on this is appreciated.

kanga

ZackF wrote: »

mental ray seems to have problems when your low poly and high poly are perfectly coplanar and overlapping,

I think this is it. Tried the fix and haven't got it working yet but I think you hit it.

Thanks

Mark Dygert

You could try selecting your low poly, going down to element mode and hide all, then try rendering?

artstream

I've seen this issue before. You can do a few things: turn off the filtering (something someone I think already said) or reset the XForm on both objects. This link on the wiki explains why those artifacts happen. http://wiki.polycount.net/Normal_Map

kanga

Vig, thanks will try that.

kaoticvisions, this should tell me what to avoid

Found a mess of t-verts so redoing the uvs and will reset x like you say kaotic. will try the fixes once the uvs are tied up.


Thanks for the loads of solutions and support! This is my first game character so I am afraid I wont be much help to other members yet,.. working on it.

Cheers

cman2k

resetting all your xforms and setting it up from scratch is probrably the best option for highest quality results.

Another viable hack solution: put a 0% opacity material on your high-poly. This will resolve the issue with coplaner faces. Stupid, but i've gotten it to work when I was stuck on what else to do.

kanga

eek cman! didnt see you there will try this one now Thanks

conte wrote: »

try to off antialiasing filters and render without it-
-Askhat.

The big map worked with the push modifier trick but not a second map. Going to do this one now but cant find filter maps to turn them off in render settings/ renderer. Even enabling GI doesnt show the function anywhere. No luck with the manual.

Where do I turn off anti aliasing in max 2010? It's probably staring me in the face as usual

Thanks in advance Askhat
Cheers

Mark Dygert

I think he might be referring to Scanline rendering? There was an issue like this when rendering AO with the skylight method. Turning off Global Super Sampling fixed it for some people. I'm not sure which AA settings inside MR could cause something like this =/

kanga

Vig,.... ah so, ok

Just a couple of screen shots,... the main occlusion turned out smooth as a baby's butt. The model has the occlusion in the diff slot of a standard max mat and normal map applied. The model weighs in at a fat 15496 tris, but has a full set of teeth. Seams are due to the push, not collapsed yet.


Thanks alot for the help. Now to paint!!!
Cheers

[HP]

badass!

It's like the feminine version of NanoSuit2.0

kanga

Thanks [HP]!
Yeah I am a total fan of thew nanosuit, so I guess this is fan art Also the new design of the suit rox.

Cheers

Junkie_XL

Vig wrote: »

I think he might be referring to Scanline rendering? There was an issue like this when rendering AO with the skylight method. Turning off Global Super Sampling fixed it for some people.

Boy am I regretting trying this at 4096...looks nice but SLLOOWWWW...

MR is way faster but I too get the missing pixels here and there.

kanga what did you end up using to make it work for you? You kept saying "worked", "didn't work", so many times I lost track in this thread...

kanga

Junkie_XL wrote: »

Boy am I regretting trying this at 4096...looks nice but SLLOOWWWW...

MR is way faster but I too get the missing pixels here and there.

kanga what did you end up using to make it work for you? You kept saying "worked", "didn't work", so many times I lost track in this thread...

No you arnt regretting it! You are going to halve your sheet later probably and end up with uber textures

Well the fix worked half the time. As you can see in the screenshots my model is complex. I had to split the cage into 2 parts. The mask overlaps the face and when it is animated I dont want it to pull the eyepieces with it. Plus the shoulder pads would give the same problem with the with the upper arms. So I have a single sheet but split into the main body, arm and leg and helmet, and then the face, sword and shoulder pads which I will connect up at the end of baking.

My mesh is sitting ON my high poly, not inter penetrating like conte's tutorial. Also I redid my uv's about 6 timeswhile I was altering the low poly so the uv channel is probably fubar. Putting a push modifier in the stack under the projection modifier and sucking the main cage onto the high poly so it penetrated worked. It did not work for the second half of the sheet. I received about 10 fixes for the problem in this thread, but none of those fixes worked for the second half of the uvs, just the first.

xnormal gave me a clean AO bake right off, but because there are bits overlapping (like those wings on the back by the shoulder blades and the floating tubes on the sword) there was a lot of shadowing. Xnormal doesn't use a projection cage. MR also had over shaddowing, but less.

Cheers

Target_Renegade

Vig wrote: »

You could try selecting your low poly, going down to element mode and hide all, then try rendering?

I get great results all the time with this. Its not obvious the first time you start baking AO maps in max, once you find this golden nugget all is good.

kanga

Thanks target!
Tried that one but it didn't work on the second set. I think that with the next character I am going to have to make my mesh penetrate the high poly better.

Cheers