Home Technical Talk

Maya AO maps

1
Daz
Daz
polycounter lvl 18
Offline / Send Message
Daz polycounter lvl 18
OK, what gives?
I get this beautiful clean result out of XSI on the left, but no matter what I tweak in transfer maps, this nasty blown out pos on the right outta maya.

We currently only have one XSI license and it aint on my machine so its a pain. So, until we get more, does anyone have any tips on how to get a better result out of Maya? I mean it's essentially the same renderer in theory, right?

Any help appreciated, maybe I'm missing something obvious here.

Replies

  • SouL
    Options
    Offline / Send Message
    SouL polycounter lvl 18
    Hand paint those AO maps!
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    you might have already tried,but maybe editing the levels ? sometimes normals get very bright, wich is easily fixed in level, but damn i wish i knew maya so i could help frown.gif
  • warby
    Options
    Offline / Send Message
    warby polycounter lvl 18
    how exactly do you generate them (in maya)? its though to tell what you do wrong if without knowing what it is you do exactly
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    I'm not sure that it's really a question of doing anything 'wrong' per se, since the maps above are using default settings from both packages and I'm talking about the huge quality difference.

    In Maya transfer maps you pretty much only have these options to fuck with:

    File format
    Bits per channel
    Occlusion Rays
    Occlusion Max Distance

    mental Ray Common Output Settings:

    Normal Direction
    Map size
    Samples
    Alpha mode
    FG quality
    FG reflect
    UV range
    Fill texture Seams

    My point is, that no amount of me fucking with these settings produces a map anywhere remotely approaching the quality of XSI using the same models.

    So in a nutshell, if anyone has played extensively with these settings, and gotten a magic formula that produces a result anywhere near as clean and tasty as XSI's, I'd gladly attempt to reproduce with them.
  • Tulkamir
    Options
    Offline / Send Message
    Tulkamir polycounter lvl 18
    Doubt it'd make a difference, but you could try using a texture bake set instead of transfering maps. I typically get good results with it at 3 samples, and 256 rays. I don't think they actually do anything different, but ya never know (Well, maybe you do, but I don't). :P

    Other than that, you could try using the dirt map plugin for maya. We've found that it can give better results than regular ol' maya occlussion passes some times. http://animus.brinkster.net/index.html
  • diZzyWalnut
    Options
    Offline / Send Message
    diZzyWalnut polycounter lvl 18
    just try CINEMA 4D 10 , it's very easy to use -
    You never know , maybe You will find Your happiness wink.gif-


    http://www.maxon.net/pages/products/products_e.html

    Up tools bar -
    go to Render Settings/Ambient Occ./Apply to Scene
    and then You can change some value if You need
    (for ex. dirt map -128 default setting, You can go down to make more dirt Simples)etc. etc.In a materials You can add Environment about 20% to get more bright (white) rendering

    It depends for what You're looking for.

    weapon9.jpg

    It's rendering with SKY.
    Up tools bar -
    Objects/Scene/Sky
    (not Floor)
  • diZzyWalnut
  • Jay Evans
    Options
    Offline / Send Message
    Jay Evans polycounter lvl 18
    Not that you want another software package suggestion..but Xnormal makes some pretty nice ones, and its a whole lot shorter than that maya tutorial:P
  • Whargoul
    Options
    Offline / Send Message
    Whargoul polycounter lvl 18
    Bake first, transfer as colour map later. Do two bakes: one with a floor and one without and blend 50/50. Make the spread as wide as possible, and play with the falloff (not max dist) to get a more XSI like one.
  • Jeremy Lindstrom
    Options
    Offline / Send Message
    Jeremy Lindstrom polycounter lvl 18
    Here's another tutorial for maya 8 and higher, i usually up the samples to 256 like he does and haven't seen any issues.


    http://www.game-artist.net/forums/spotli...chemist101.html
  • fogmann
    Options
    Offline / Send Message
    fogmann polycounter lvl 17
    I feel your pain, Daz. I had the exact same problem each time - the AO maps done with mental ray in Maya end up looking horribly blown out and they take forever to render frown.gif I'm surprised to see XSI handles them so well, also using mental ray - really weird. I was messing with settings before, but I don't remember ever being satisfied with the bakes. So I just gave it up and use xNormal instead. The latest versions will let you render out AO only, and bakes look much better.

    Meh, in that game-artisan tutorial they also get the blown out AO map frown.gif I guess baking the AO map onto high poly first and then transferring this map to the low would have better results, but that's way too much trouble for something that is supposed to work right out of the box - not to mention that it could be impossible to UV high poly in some cases.
  • East
    Options
    Offline / Send Message
    East polycounter lvl 14
    You could give Illuminate Labs' Turtle a try. It's probably a good investment if you do most of your baking in Maya. Always given me good results the few times I've tried it.

    But yeah, I normally have problems with blown out AO results no matter what I use.
  • Maddness
    Options
    Offline / Send Message
    Maddness polycounter lvl 11
    is that a side boob?
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    Thanks for the help all. I'll look at the alternative suggestions, and I could go back and spend more time with xnormal, but I'd much happier If I could do this from within maya.
    Whargoul, you hit the nail on the head thanks man. Using batch bake mental ray from within Maya, falloff is the key parameter here. Upping the value to 50 or so gives me beautiful AO maps, with a quality equal to the XSI ones.
    However, as fogmann touched on, that's a little bit of a longwinded solution, and it requires UV's on the high poly, and it also requires a much bigger map than you need for the first bake, to allow for degradation from transferring the AO map into a different UV layout when you use transfer maps. So not ideal. So, in a nutshell, can anyone explain why I can't access the falloff parameter from within transfer maps? That one simple thing would make my life very easy. I'll probably contact Autodesk about it, and I'll speak with our mel guru TD tomorrow. But it seems really bizarre to me. The one parameter you need to get nice quality AO maps using transfer maps isn't available from within the transfer maps menu, therefore making the creation of AO maps from within transfer maps pretty much completely redundant. How very odd, sigh.
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    [ QUOTE ]
    Here's another tutorial for maya 8 and higher, i usually up the samples to 256 like he does and haven't seen any issues.


    http://www.game-artist.net/forums/spotli...chemist101.html

    [/ QUOTE ]

    Hah that's pretty interesting. fogmann noticed what the author of the tutorial apparently didn't. The first half of his tutorial on setting up AO shows nice results, the latter half of his tutorial which explains making AO maps *from within transfer maps* produces useless blown out crap, doh.
  • Ruz
    Options
    Online / Send Message
    Ruz polycount lvl 666
    strange though, because 3dsmax produces blown out shite also.
    I have never got results like in that first xsi bake you posted Daz
  • Illusions
    Options
    Offline / Send Message
    Illusions polycounter lvl 18
    [ QUOTE ]
    So, in a nutshell, can anyone explain why I can't access the falloff parameter from within transfer maps? That one simple thing would make my life very easy.

    [/ QUOTE ]

    I'm pretty sure its in there. I was looking through transfer maps yesterday to see if I could provide any help, but I do remember seeing falloff there. It is most likely under one of the closed toggle menus, and I think it might even be under the toggle menu with Final Gather in it. I don't have access to Maya at the moment to give you an exact name.
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    Trust me, I've looked, looked again, and again, and falloff is nowhere to be found from within the transfer maps menu, either hidden or showing. I'm on 8.0 though. If it's available as a parameter in 8.5 I'm all ears....

    tm.jpg

    I was thinking there might be a way to do this properly using the 'custom' shader option, so I'll tinker around with that.
  • Illusions
    Options
    Offline / Send Message
    Illusions polycounter lvl 18
    Damn, sorry Daz, what I was seeing it in was in fact Batch Bake options...d'oh.
  • rooster
    Options
    Offline / Send Message
    rooster mod
    sorry for dragging up this old thread, but I've been messing with this stuff lately and this thread came up in my searching. I think I've got the solution so I figured I'd update for anyone else who comes across it.

    Dunno if theyve made the transfer maps options better in maya 2008, but this works for 8.5 and probably some other versions:

    - like Daz suggested, you need to create a custom shader. make the shader a mib_illum_lambert

    - make sure the ambience value is white, now create a mib_amb_occlusion node, and plug it into the previous material's ambient value

    -now you should have all the ambient occlusion controls you're used to available in transfer maps

    (i don't think this is neccecarry, but I remember reading somewhere to check 'maya derivitives' for rendering transfer maps with mental ray. its buried in the performance dropdown of the translation dropdown in the mental ray render tab)
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    Hey, almost missed this, thanks a bunch Rooster.

    Following your steps however, i actually can't get this to work. It *might* be 'cos Im on 8.0, but I'm clinging on to the hope that it isn't. After fumbling around with some settings, I get *something*, but it's nothing resembling AO. Strangely it looks more like a one light directional light source.

    Here's what I have, plugged into transfer maps as a custom shader:

    AO_shader.jpg

    Does that look right?

    One thing I noticed is the output mode. Know anything about that? Also, i assume that in the transfer maps options, you need to perhaps untick 'use mental ray common settings' so that it uses the settings from this shader instead?

    Any help appreciated.
  • Whargoul
    Options
    Offline / Send Message
    Whargoul polycounter lvl 18
    Try adding an ambient light with all the values turned to zero - sometimes MR adds the default light in and blasts your AO with "light" on top. Lame.
  • Emil Mujanovic
    Options
    Offline / Send Message
    Emil Mujanovic polycounter lvl 18
    This thread has been really handy, as I've been put in charge to do some lighting R&D for our current project.
    Most of the AO bakes I've done in Maya are generally really grainy and over bright. Thanks for all the extra tutes/scripts/etc, guys and girl.

    -caseyjones
  • rooster
    Options
    Offline / Send Message
    rooster mod
    GODDAMN EXPIRED FORM THINGY. pressing back does not always work..

    from the top..

    I've made a jpeg of what I think are the relevant settings, I had a bit of bother getting it working too so I probably missed something in my initial post. One thing I did notice was that if you leave it on 'all other meshes' for the source, it includes the low poly which makes a mess. I also changed the diffuse to black

    the colours are weird so that if it comes out black or white, I know theres a problem, but if it's yellow or pink then it's probably my AO settings

    (sorry for the layout)
    http://redprodukt.com/wp-content/uploads/2007/12/maya_ao.jpg

    I also uploaded the ma, I belive theres a trick to get it working in older versions that involves editing the ma and changing the version number in the file
    http://redprodukt.com/post/maya_ao.ma
  • Dr. Rudi
    Options
    Offline / Send Message
    Dr. Rudi polycounter lvl 17
    Yeah Daz, been having the same problem. I've been using ultimapper like you.... works great.... BUT WHY NOT MAYA?! burns my britches. I tried xnormal when I temporarily lost the xsi license but couldn't get decent results. If you find a solution for maya let me know... I think I've tried everything.
  • bugo
    Options
    Offline / Send Message
    bugo polycounter lvl 17
    since i got turtle trial i would buy it, because its amazing, for AO, skylight, whatever it gives me all in one
  • rooster
    Options
    Offline / Send Message
    rooster mod
    dr. rudi, did you try those settings I posted? I have the full set of regular ao options working with transfer maps
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    Hey rooster, I meant to say thanks for the taking the time to figure this out and write all this up. Unfortunately however, after significant time invested in messing with it I actually cant get it to work properly in 8.0 at all. It seems strange that it wouldn't but perhaps some relevant code changed between versions I really don't know.
    If you're curious, no matter what I tweak, I get this: http://www.daz-art.com/ao_prob.jpg
    So on the left you can see the hi geo in silver, and the lo in red. Note in the AO map result at right you can see where the two meshes intersect, and that its only rendered the hi poly stuff that goes out beyond the low poly, not the stuff within. It seems like some kind of enveloping or distance issue, but umpteen gazillion combinations of tweaking that stuff provide nothing new. Bummer. Of course we have the whole 'cant upgrade cos were mid project' usual annoyance.
  • rebb
    Options
    Offline / Send Message
    rebb polycounter lvl 17
    Daz : I guess the "umpteen gazillion combinations of tweaking that stuff" include having set the "Search Envelope %" to something like 0 or 1 and the "Search Method" in the "Advanced Options" Tab to "Closest to Envelope" ?
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    thanks Rebb but Im afraid they did yeah. It does seem to be related to enveloping and I tried umpteen different enveloping settings. Its really odd. Basically it will render nothing of the hi poly that sits inside the the low. Its almost like the low is 'in the way' of the ray casting when it really shouldn't be considered a rendering object in the AO process. I guess I could tweak the low so that literally none of it pokes out further than the hi poly, but that would be a royal pain.
  • rooster
    Options
    Offline / Send Message
    rooster mod
    hi man, that pic looks something like what I got when I left the settings on 'use all other meshes' in the source meshes. You probably set up the source objects though so we're looking at a different problem :/

    what about, using a custom attribute to discount the low poly mesh from the ao calculation?

    I don't have maya on me at the moment and I don't remember the particulars, but there is a setting in the ao node that means you can say 'dont use objects with this tag' and if you then add a custom attibute to the object and give it the same value, the ao will ignore it. This seems like your best hope since the ao isn't using the transfer maps habit of ignoring the low poly.

    All I can do at the moment unfortunately is mention that you can do that; without my test scene I can't give you the details, curse my leaky memory smile.gif. It is mentioned in the maya help about the ao node, but it took a bit of googling to work. (maya's help is a pile of sh-)

    I'll get those details when I'm back home, unless you've already given this a go

    one thing I do remember- in the ao node you put a negative value as this variable, to say that you want to discount an object (for example -1) and a positive to say you want nothing but this object (1). maya's help tells you to put the same value into the custom attribute you create for the object, but what it really means is that in both cases, you put 1.
  • rooster
    Options
    Offline / Send Message
    rooster mod
    gah im getting that problem now! working on it...
  • rooster
    Options
    Offline / Send Message
    rooster mod
    i think I cracked it!

    turn off all the options in the following tabs:

    the render stats of the low poly, the mental ray tab (both in the shape node), and also, the mental ray 'flags' tab in the other node tab (expand mental ray, then expand Flags) once all that is off (realistically its probably only a few of those that make the difference), no more intersecting!

    no intersecting low poly! woot
  • dur23
    Options
    Offline / Send Message
    dur23 polycounter lvl 19
    In all seriousness maya makes me want to stab myself in the god damned face.
  • TWilson
    Options
    Offline / Send Message
    TWilson polycounter lvl 18
    I just created a thread semi-related to this. I dont think I made it even as far as Daz however. I run out of memory at like 21mbs. So I'm trying to figure out how to raise my Maya memory limit.

    So far I've tried transfer maps and xnormal. Both give me blown out grainy maps. I was hoping TransferMaps would work after I raised my memory so I could play more. Apparently I was wrong?

    I hope someone finds a solution!

    I've been using the green channel from my normal map but that gives seams sometimes.
  • Daz
    Options
    Offline / Send Message
    Daz polycounter lvl 18
    Thanks rooster, I'll try it.

    Yeah that's a definite limitation TWilson. I can get get some pretty insanely dense geo into maya, but of course, start trying to actually manipulate it or god forbid select all edges and soften them and a quit to desktop usually ensues.

    Actually though, I know this is a maya thread but I've been getting great results from persevering with xNormal of late and been using that. I was put off it initially because my default result was always super blown out, but then I discovered that a combination of the spread angle cranked up as high as possible and tweaking mesh scale gives me really nice results.
  • rooster
    Options
    Offline / Send Message
    rooster mod
    its ridiculous that it should be such a head bashing exercise to bake a nice ao map. why the hell didnt they just include all the relevant options in the transfer maps ao?
  • MoP
    Options
    Offline / Send Message
    MoP polycounter lvl 18
    Heh this thread came up when I did a Google search for a Maya 2008 transfer maps issue I've been having - not AO, but just normals, and I think it might be the same issue that Daz is having.

    Basically from what Daz has posted, it looks to me like the baked texture is taking information from the target meshes (the lowpolys) as well as the highpoly source - this is why it's only looking "right" on the sections where the highpoly pokes through the lowpoly.

    Now I seem to be having the same problem, but with normals - I have my scene set up, 2 lowpoly target meshes inheriting normals from about 10-20 highpoly meshes (it's a mechanical object), yet whenever I use Transfer Maps to try and grab normals, it appears to grab the lowpoly normals too! Not only this, but it grabs them as though all the edges are hard (they aren't).

    I can tell it's reading the source meshes properly, because the normals are coming out fine whenever the highpoly mesh pokes through the lowpoly, but beyond that I don't know what's up. I rendered out normals for a simpler object yesterday with exactly the same settings and it came out fine (with intersecting meshes and everything), now today it just won't seem to ignore the lowpoly when baking normals frown.gif

    It's like it's baking the lowpoly to itself, but with no soft edges... looks like crap and is very annoying. I asked our resident Maya guy about it and he couldn't figure it out either. All my settings appear correct.

    I'm gonna try merging everything into a new scene, importing as OBJ etc. and see what fixes it. But it seems that it's a Transfer Maps issue, not just an AO issue.
  • Illusions
    Options
    Offline / Send Message
    Illusions polycounter lvl 18
    Have you tried deleting history? I had an issue earlier today where the transfer maps wouldn't work right with the diffuse transfer setting because the target mesh still had some history on it. Other than that, I wouldn't know. I've never run into an issue like that (unless you're using all objects in your scene as a source mesh, including the low poly).
  • rooster
    Options
    Offline / Send Message
    rooster mod
    does it actually let you select the same mesh as target and source? I don't think you can unless you duplicate it

    Mopster dunno if I can help but I'll take a look at the scene if you like?
  • katzeimsack
    Options
    Offline / Send Message
    katzeimsack polycounter lvl 17
    hmmm, tried some renderflags? Normally you shouldn't have to, but removing the "Primary visible" flag of the lowpoly geometry could help.
  • MoP
    Options
    Offline / Send Message
    MoP polycounter lvl 18
    Ohh... I bet I know what it is. I've been using the "High quality display" to preview the smoothing in the viewport, I just remembered that it doesn't do that in the software renderer, so it's probably just baking out the cages (which look pretty much like the lowpoly mesh, hence the confusion).

    Bit silly that it doesn't do that, now I'll have to add a proper Smoothing thing to all the objects... why can't it just assume that I want to render them smooth if I'm viewing them smooth?
    As with many things in Maya, I'm finding it's several steps to the same end result as one single step would get you in Max.

    Also I find it incredibly annoying that Maya doesn't store Transfer Maps settings per scene/object, you have to save/load them every time you want to bake a different scene or set of objects, which seems like a horrible workflow, unless there's something I've missed?
  • Mongrelman
    Options
    Offline / Send Message
    Mongrelman polycounter lvl 18
    Maya also likes to unmerge uvs when you import a model (incase you didn't already know). So I find standard practice is import model>merge uvs>select all edges>soften edge.

    Yay for productivity.
  • rooster
    Options
    Offline / Send Message
    rooster mod
    is that true lupus? dang need to keep an eye out for that one
  • Mongrelman
    Options
    Offline / Send Message
    Mongrelman polycounter lvl 18
    Aye, I've had a dig through import options to see if I could get it to stop doing that, but had no luck (if anyone else knows how please share!).

    You can smooth the edges, but if you forget to merge the uvs you can still get artifacts when baking maps. If you're bakign AO and doing so by baking it into a texture for the high poly and then transfering that across as diffuse, remember to merge the uvs of the high poly too.

    Doesn't take long, just select the object and in polygons menu set, go to edit uvs>merge uvs (don't to go into uv editor and select all uvs then file>merge, that takes longer and can cause maya to crash with a high poly mesh).
  • sprunghunt
    Options
    Offline / Send Message
    sprunghunt polycounter
    [ QUOTE ]
    strange though, because 3dsmax produces blown out shite also.
    I have never got results like in that first xsi bake you posted Daz

    [/ QUOTE ]

    You can adjust the settings in max to reduce the blown-out look. If you increase the 'spread' that works.
    Or you can not use mentalray and use the light tracer instead. That also fixes the blown-out look.

    It sounds like the method being used in maya just doesn't provide any options to fix it.
  • rooster
    Options
    Offline / Send Message
    rooster mod
    lupus: does it happen to you constantly or just sometimes? also with all formats? I'm not getting split uvs from importing a .mb
  • arshlevon
    Options
    Offline / Send Message
    arshlevon polycounter lvl 18
    the split uvs is a zbrush specific problem, if you import an obj from zbrush in max the uvs are also split. i personally do not get split uvs from importing an obj that was exported from maya back into maya.
  • Mongrelman
    Options
    Offline / Send Message
    Mongrelman polycounter lvl 18
    Not just from zbrush unfortunately, I also get split UVs when importing from Silo and Blender frown.gif

    And happens all the time for me Rooster. I have the viewport info thing up all the time (displaying polys, uvs etc.) and watch to see the uv number shrink when I do the merge. Always goes down (quite a lot) when I import a model and merge.

    On a slightly different note, one of the guys at work had one of the strangest problems I've ever seen. He had a head UV mapped, and it was looking relaly weird with the texture, even though the texture was fine and the uv map was fine. Turned out maya had taken each UV face in the map and rotated each one individually by 90 degrees, keeping the UV map looking exactly the same.


    I didn't even know that could be done.
  • dur23
    Options
    Offline / Send Message
    dur23 polycounter lvl 19
    Lupus: theres a button that does that. Worst button ever.
1
Sign In or Register to comment.