Home Technical Talk

Maya Transfer Map diffculties (or, do I really need to break this mesh up?)

Hello everyone. I've run into some technique issues that I need some help resolving and appreciate any advice.

Basically, it comes down to grabbing the Normals and AO from my high poly mesh and giving them to my low poly mesh (typically Maya Transfer Maps has done this for me). However, I'm having difficulty with this current mesh due to its complexity and how close pieces are to each other (likely throwing off the Transfer Map envelope).

Models (High<->Low): dlKnX.jpg
Resulted Normals: bb9A7.jpg
Resulted AO: M8Z62.jpg

As you can tell from the maps, the degradation is pretty bad (you can tell without needing to see it rendered on the geo). My goal with the AO was to get all the nice detail from the high geometry to utilize in my Diffuse map, though perhaps given the complexity of the piece that may not be possible. I suppose I could forget about that and just get whatever AO from within Unreal/Marmoset via the second UV set.

Starting to ramble. I suppose I just want to know if the best way to get those maps is to just Separate these meshes into smaller pieces and bake them out individually or if there's something else I'm overlooking. And if there's a way to get all the high detail AO, or if I have to forgo some of it.

Thanks as always.

Replies

  • throttlekitty
    You'll need to explode your high and low poly models before baking. We JUST had a thread on this, but i can't find it right now. Basically set a keyframe at the start/end so you have your original positions, and at the model bits spaced out. This way you won't have the overlapping areas for your normal map.
  • fatihG_
    Offline / Send Message
    fatihG_ polycounter lvl 14
    or if you dont want to break your model.. you could try using blendshapes?
  • Soul_of_Solace
    Thanks for the suggestions. Yeah, I'm just going to break it apart, a bit of a hassle trying to keep the parts organized, but that seems to be the best way to prevent transfer map degradation, which is an even bigger hassle to try and fix. Was just making sure there was no "quick fix" out there that I wasn't aware of.

    Obviously the thing to take away from this is to better organize the meshes when building them so that they are easily exploded for this scenario.
  • throttlekitty
    Was just making sure there was no "quick fix" out there that I wasn't aware of.
    ==
    Obviously the thing to take away from this is to better organize the meshes when building them so that they are easily exploded for this scenario.

    Very much this. In this case, you have many smaller modular pieces that are easily grouped.
  • passerby
    Offline / Send Message
    passerby polycounter lvl 12
    keep it all as separate objects, and keyframe everything in the correct locations, than explode it all out and set a new keyframe.

    that way you can go between normal, and exploded, but just toggeling between frames 1 and 2.

    i generally, have the maya outliner opened all the time, and try to make it look pretty in there too, so renaming things to logical names (not pcube21) and grouping things as i got, really helps out later.
  • Soul_of_Solace
    The Normal map vastly improved, but the AO map is still kind of spotty (random black spaces on flat cylindrical areas even with no intersecting geometry present) and it takes forever to setup and bake out. I'm wondering if I should give Xnormal a try since a lot of people seem to like it.

    EDIT: Here's a collage of info about the current setup (areas of random darkness/shade on those flat cylindrical areas, and my transfer map settings for AO/MR):
    w8xfw.jpg
  • throttlekitty
    Could be smoothing/normal errors, or unapplied transforms. (voting on the latter)
    I would suggest using xNormal for AO, results are typically better and done in half the time.

    One trick I use is to do two AO maps: one high>low in exploded form, and one low>low in assembled form. I typically do the low>low at a lower quality for speed, and selectively mix that into the final map. This gives you all the detailing from the high poly, while letting you choose which proximity-based shading to use with less hassle.
  • Soul_of_Solace
    Yeah, I was planning to do a low->low AO map utilizing Batch Bake which usually works pretty well. It's the high->low map that has always been a hassle, but I'd like to have it for the high poly mesh's tiny details. I definitely think I'll try XNormal.

    But I'm curious what you are meaning by "unapplied transforms" and what would I look for to indicate that such a thing is occurring.

    Thanks again!
  • throttlekitty
    But I'm curious what you are meaning by "unapplied transforms" and what would I look for to indicate that such a thing is occurring.

    It was from a thread a few months back, i've only seen the issue once. In this case, the model was moved/rotated/scaled arbitrarily in the scene and gave strange artifacts in the ao. Doing Freeze Transforms on it fixed the issue.
    What to look for? In the attribute editor or channel box, the numbers you see in trans/rot/scale represent how the shape has been transformed from it's original position.
  • Soul_of_Solace
    I gotcha, so it was an issue of the transformation having not been frozen at the time of transference. I'll double check my individual pieces to see if any of them got tinkered with during all my tweaks and combinations/separations to get the asset to explode properly (only the move transforms should be active in the proper keyed positions). Thanks!

    Still going to look at XNormal anyway :)
  • Soul_of_Solace
    Ok, so I've been testing out Xnormal using this mesh, and of course, trying to use it as is will not get good results given its complexity (similar to the above maya transfer map results). And unfortunately the mesh is too complex for a simple fix by tweaking the Max Front and Rear ray distances.

    I was wondering then if it is possible to just export and use my "exploded" versions from Maya in XNormal? Or is this a case where I'll need to create a custom cage? If so, are there any good posts/tutorials handy discussing the creation/usage of custom cages in Xnormal? (I wish there was a way to search within big threads like the main Xnormal thread)

    Thanks again!
  • m4dcow
    Offline / Send Message
    m4dcow interpolator
    You wouldn't really want to explode your objects for an AO map because AO is generated on proximity.


    Obviously you can use the exploded versions for the mesh in xNormal for the normal map.


    To make a cage in maya I suggest:
    1. Duplicate your low poly mesh, and then hide your original copy.
    2. In vertex select mode, and using the translate tool, change the translate tool settings move axis to: Normal.
    3. Then select your different bits of geometry and move them along normal so it completely encompasses your high poly.
    4. The reason I prefer to do this instead of envelopes in transfer maps is that you can move the verts out different amounts, instead of one number.
    5. You might also need to bake a low poly AO and mix it in with the high poly bake, so the intersections match the low poly instead of just the high.
  • throttlekitty
    The cage that Transfer Maps can have its vertices tweaked, history and transforms cleared, exported to xNormal, and used as a cage just as in Maya.

    Same deal with the keyframe-explode method, just export everything all blowed up and go!

    Have you solved the wonkyness in the AO yet? I'm curious about this.
  • Soul_of_Solace
    m4dcow wrote: »
    To make a cage in maya I suggest:
    1. Duplicate your low poly mesh, and then hide your original copy.
    2. In vertex select mode, and using the translate tool, change the translate tool settings move axis to: Normal.
    3. Then select your different bits of geometry and move them along normal so it completely encompasses your high poly.
    4. The reason I prefer to do this instead of envelopes in transfer maps is that you can move the verts out different amounts, instead of one number.
    5. You might also need to bake a low poly AO and mix it in with the high poly bake, so the intersections match the low poly instead of just the high.
    Thanks a ton! I'll definitely give this a shot for making the cage. :thumbup: (and thanks for the tip of doing a low-poly AO bake too. I had planned on doing so since I usually have far better luck getting AO out of Maya Batch Bake than transfer maps)
    Have you solved the wonkyness in the AO yet? I'm curious about this.
    Here is the last AO map I managed to get out of Maya transfer maps. I made sure to ensure there were no unapplied/unfrozen transformations beforehand. It looks better than the original attempts, but still has issues (maybe I didn't make my envelope far out enough on some parts).
    K6T6a.jpg
  • Soul_of_Solace
    Oh wow. Using the cage technique described above, I managed to get this AO map out of Xnormal. Which, taking into account all the intersecting geometry and such, is pretty damn accurate considering (though I'm still thinking about getting an AO map out of the exploded geometry and blending these in Photoshop). AND (most importantly), it only took about 15 seconds to generate the 1024 map (as opposed to say, 10+ minutes in Maya transfer maps)
    2zR96.jpg
    So yeah, I'm so sold on Xnormal! :)
  • haiddasalami
    Offline / Send Message
    haiddasalami polycounter lvl 14
    Maya's transfer maps is single threaded so Xnormal will beat it always :P
  • bugo
    Offline / Send Message
    bugo polycounter lvl 17
    solution? if you can? use TURTLE! I said that here manytimes, but the problem is that Autodesk still doesn't have on the regular Maya, which sucks. But Turtle IS the best baking plugin/app ever. Fastest and all packed with game dev features.
  • haiddasalami
    Offline / Send Message
    haiddasalami polycounter lvl 14
    bugo wrote: »
    solution? if you can? use TURTLE! I said that here manytimes, but the problem is that Autodesk still doesn't have on the regular Maya, which sucks. But Turtle IS the best baking plugin/app ever. Fastest and all packed with game dev features.

    correct me if Im wrong but isnt Turtle included in Maya 2013?
Sign In or Register to comment.