Home Technical Talk

Why you should NOT trust 3ds Max's viewport normal-map display!

15681011

Replies

  • Ben Apuna
    Any updates on the UE3/UDK output support?

    Are object space normal maps all made equal? In other words, will we be able to use the NSpace converter on xNormal generated object space maps and convert them to UE3/UDK tangent space maps?
  • EarthQuake
    Eric:

    http://forums.cgsociety.org/showpost.php?p=6400502&postcount=246
    There are still issues to avoid when using Maya's RTT tool (for example). It's not perfect, though it does generally create better results than Max's RTT.

    These are issues with every baking tool, not specific to maya.

    [edit] Not entirely true i guess. These issues exist in max, but you can fix them by manually adjusting the cage. The "cage/envelope" in maya only sets the ray distance, not direction. In general terms, this is a side effect of doing any sort of baking with an entirely averaged projection mesh.
  • kenpimentel
    It's too bad we're only now discovering this thread. We actually have a mechanism for reporting problems:
    http://usa.autodesk.com/adsk/servlet/item?siteID=123112&id=5600504&linkID=9241177

    You'll find we're more responsive when this is used. We are studying the issue and think we see the problem. No news as to when we'll address it, but feel free to keep bringing it up until we do.

    Autodesk 3ds Max team
  • Eric Chadwick
    Thanks man. I'm starting to learn Maya, will be playing around with this some. The whole normal mapping process seems way messier than it should be.

    Another example. And I posted a new thread on CGTalk for Ken Pimental, Normal map discrepancies in Max.
    max2010_normalmap_compare.png
  • fozi
    Ben, yes. The only difference is axis transformation between apps/engines. But as long as the WS normal map matches the mesh (reset xform/freeze/wtv), it should be fine. You will be able to output a WS map from any app, including xnormal.

    I might have some news this weekend.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Hey Eric, super-cool that you are getting this to Ken. I sincerely hope they will invest some more development in realtime relevant features for Max, since a few are quite broken.
    Your example is pretty cool and makes perfect sense, i just don't understand why the RTTnormalmap.fx has trouble with the Nspace converted one ? Doesn't that mean that shader has a batantly incorrect normalmap caluclation ?

    EDIT: Ken, I submitted a few of my issues already. Fingers crossed that you guys will look into them.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    kenpimentel: Thanks for posting. I didn't actually report this officially since it's not really "broken", in fact it works perfectly as long as your shader/game engine calculates tangent basis in the same way as Max's scanline renderer.

    I guess the lack of a "standard" when it comes to tangent space makes it hard to get normal-maps looking correct in all cases.

    EricChadwick: Nice example!
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    the viewport being not synched should be considered a bug so. As well as the tangent space breaking when clipped in scanline view
  • fozi
    The fact that Nspace has trouble with RTTnormalmap.fx is because the tangent basis used in DirectX Shader Materials does not match Max's standard material using "Normal Bump". Plus, I wasn't able to figure those out 100%, as they differ (albeit slightly) from anything in the docs or samples.

    By looking at the sample code, one can verify that Metal Bump 9 also seems to use its own tangent generation code.

    So i'd say there's at least 3 (like someone mentioned earlier), possibly 4 (Normal Bump, DX Mat, Metal Bump and Scanline), either slightly or completely different basis in Max. Excluding, of course, the TB included in exported meshes.
  • Ben Apuna
    fozi:

    Thanks for clearing that up for me and continuing to work on NSpace :) I bet I'm not the only one looking forward to getting some perfect tangent space maps into UDK.

    kenpimentel:

    Thank you for looking into this issue.

    cw & EricChadwick:

    Looks like you guys did a great job getting some attention onto this issue, hopefully Max can get fixed up.
  • cw
    Offline / Send Message
    cw polycounter lvl 17
    the viewport being not synched should be considered a bug so. As well as the tangent space breaking when clipped in scanline view

    That's a really good point and should probably go as a seperate bug - I don;t have the details but if whomever discovered the bug would be willing to report it through the bug reporting link it would be a good thing to do! :D
  • Mark Dygert
    cw wrote: »
    That's a really good point and should probably go as a seperate bug - I don;t have the details but if whomever discovered the bug would be willing to report it through the bug reporting link it would be a good thing to do! :D
    If Ken says their on it then I don't think you need to worry about filling out a bug report. He was saying it would have been nice to get that officially submitted months ago.
  • Eric Chadwick
    Testing the workarounds for tangent-space maps...
    max2010_normalmap_workarounds.png

    I wonder if I've set up the GenericIBL shader incorrectly? Tried flipping the red channel too, but that was much worse.

    The best results seem to come from creating hard edges (split vertex normals) that match the UV seams. As others have stated before.

    Also, as noted elsewhere, adding a raw Edit Normals modifier seems to alter the vertex normals slightly, even if no edits are made. This improves the viewport shaders' results a little bit, it's not a drastic change but it's generally for the better.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    EricChadwick: Even on the one with the green tick, you can see a slight normal issue on the side :)
  • Eric Chadwick
    Heheh. You're too picky.

    Still playing around, testing in Maya, Unreal.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    That RTT shader must do some changes to the normals. Messy to find out though: it's not written very clearly and there's tons of techniques. Which one is it set to use in that green marked cube, Eric? Also it doesn't seem to be 100% perfect: you can see a slight hint of that diagonal streak still present (edit: MoP beat me).
  • Eric Chadwick
    The default technique... TangentSpace_2_0.

    For some reason the bevelled versions have seams, though Chai seems to have avoided them. Gonna ask him.
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    I am not sure what you are trying to do Eric, looks like a lot of effort in simply proving that different tangent space definitions and different shaders (whether they renormalize per pixel or not) exist.

    and by this thread autodesk was surely made aware of their inconsistencies. it's much easier for them to hop at the critical sections in code and make sure stuff is consistent, then you going thru all kinds of combos ... I hope I don't sound like an ass now, but I think you are wasting your time. Once AD has "fixed" their inconsistencies, it's simple for shader authors to adjust to the official way, and the crap will be gone.

    we know that the scanline method works, and I can assure you that same quality can be achieved in viewport (as was shown by my own results in this thread).
  • Eric Chadwick
    No that's cool CB. I'm learning though, that's why I'm doing this. AD might fix it, they might not, track record is a bit spotty there. Good to know what works, now.
  • cw
    Offline / Send Message
    cw polycounter lvl 17
    I'm so excited! And I just can't hide it!

    I know

    I know

    I know

    I know

    I wanna bake maps,

    clean maps. :)

    (to the tune of that song - you know the one?)
  • Chai
    Offline / Send Message
    Chai polycounter lvl 18
    Hey nice test EricChadwick, should definitely be on the polycount wiki or such ;)

    I don't see any problem with your bevelled example, however the normalmap looks wrong, it should be a gradient around the edges - make sure you set them to one smoothgroup.
  • loganfredericksen
    the horrible triangulation is because your smoothing groups are messed up. make sure before you render to texture, you select your low poly and reset the smoothing groups and the click auto assign smoothing groups. fixed it for me :)
  • EarthQuake
    the horrible triangulation is because your smoothing groups are messed up. make sure before you render to texture, you select your low poly and reset the smoothing groups and the click auto assign smoothing groups. fixed it for me :)

    You've sort of missed the whole point of the thread here, i suggest you give it a read.
  • Vrav
    Offline / Send Message
    Vrav polycounter lvl 11
    Tangent space is really damaging to my self-esteem.

    No matter what combination of edge breaking, unwrapping and highpoly fitting I try, I cannot, for the life of me, get a normalmapped 12-tri cube to render rounded edges. There is ALWAYS some sort of seam on the corners or the edges, even in using xNormal to bake; it makes me want to cry, when I see you guys doing it... or making equally boxy fingers appear cylindrical. Just seems like something I am entirely incapable of doing—whine. Like trying to draw but finding out your pencil is a twig.

    Or trying to be competent and finding out you're not, and may never be.

    silo2010030918594398.jpg

    I can't even get a beveled cube to look perfectly rounded.

    [/emo vent]
  • PredatorGSR
    Offline / Send Message
    PredatorGSR polycounter lvl 14
    I was never able to get a good bake reliably with xNormal, although other people seem to do fine with it. Using Mayas RTT has resulted in really good bakes if everything was set up correctly. It takes a lot of frustrating trial and error, and the occasional "aha" moment when you figure out how to get the best results.
  • Eric Chadwick
    Vrav, how are you displaying your model? (what engine, which method?) In Max I seemed to get the best results using hard edges in the same places as my UV seams. Bevels seem like not as good of an approach, since they tend to make long thin tris, and are generally a pain with more complex models than a rounded box.

    Konrad, is your flakk cannon using mostly Photoshop-generated normal maps with a bit of 3D-baked detail? I don't see many gradients in the flat bits.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    I think that's because his cannon has so many tri's the normalmaps don't have to do a lot of those smooth gradients. Most people don't go that high in tri's since it's so much work to unwrap.
  • Eric Chadwick
    I don't think so. You only get completely-flat blue if you bake with all hard normals (unique smoothing groups for each flat area).
  • EarthQuake
    No, looking at his texture he simply is using lots of hard edges. Pretty much anything with a major angle change. If he had lots of bevels etc on the edges you would still see more gradiations. However, there is still a lot of detail there which you can everything was baked from a high, the nice soft edges on all the shapes, some gradients, etc.

    The likely cause for why he couldn't get good results in xnormal was all of the hard edges, XN has a pretty backwards workflow to get an averaged cage(1 click or default settings in max/maya) so that explains why he would have trouble with it.
  • Vrav
    Offline / Send Message
    Vrav polycounter lvl 11
    Vrav, how are you displaying your model? (what engine, which method?)

    Here are four (unwrap type vs averaged or hard seams) xNormal baked variants displayed in the xN 3D viewer, with the promising ones opened up in Marmoset.

    tangent-cubes.jpg

    However, I think it is something of a moot point, because when applying the normal map as a texture to the cube, it should appear solid - but there remain obvious seams in the lighting information wherever there are seams in the UVs.

    I imagine this can be circumvented in practical use, but is still interesting. So, I should bake tangent space in something other than xNormal? Or perhaps direct Santiago towards this thread...
  • EarthQuake
    You can fix the hard edge split problem in xnormal, but it involves an overly complex workflow.

    you need to:
    1. load the 3d viewer
    2. turn on the cage editing tools
    3. select all of the verts
    4. weld all verts
    5. expand the cage as you normally would
    6. save out your mesh as .sbm(or whichever xnormal format it is, i dont rememeber)
    7. make sure XN is loading the correct mesh
    .....
    8. repeat process any time you change your mesh =)

    All this to get the same functionality as what is essentaily a 1-click option in maya/max.
  • Vrav
    Offline / Send Message
    Vrav polycounter lvl 11
    One thing related to Max that has been on my mind is the creation of smoothing groups. With a model that has smoothing groups defined in Max, is anything special exported in the obj? I ask because in using Silo, the only way to create "smoothing groups" for me is to simply break the edges, duplicating verts. I thought that's all normal smoothing was, but if there is something invisible being saved, then I am out of luck.

    I'm also confused as to what a cage is when you talk about it. I was under the impression that it's just a duplicate of the lowpoly adjusted to better receive rays from the highpoly. Sort of the 3ds/maya automated mumbo jumbo for that, anyway. But I didn't think the cube needed a cage, and could just be baked directly; as such, I have no idea what the cage in xN is. I just create "cages" in Silo and export them as an alternative obj for baking...
  • EarthQuake
    Yes, in max and maya, when you create smoothing groups, or hard edges, the verts are not saved as split, just the mesh normals are split. So when you forcibly split your edges with that workflow in modo, or silo, or any similar app, you're preventing the baker from ever giving you a seamless result.

    To the game engine, it is essentially the same thing, however.
  • rasmus
    EQ: You're getting so good at hammering out the laws of normalmapping on every corner of this forum, pretty soon I think you guys should consider just compiling ten commandments and making them the front page (booming recorded voice optional)
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    There was a sticky thread on all things normal mapping, it got moved into the archives
    http://boards.polycount.net/showthread.php?t=48305

    now someone could make the work to compress it, but then again educating oneself is part of the job, and reading through threads like this is actually less work than googling the web ;)
  • Eric Chadwick
    Working on it, though it's definitely a side job, would appreciate some help. CB you added some on the Tech-Artists.org wiki awhile back, right?

    Nothing like writing it down to reinforce learning...
  • [Deleted User]
    Offline / Send Message
    [Deleted User] polycounter lvl 18
    When I export a box with the sides on different smoothing groups with the xNormal SBM exporter from Max, I get the same result whether the cage is exported from Max without changing it or I go in and weld the cage verts in xNormal. It looks like this:
    nufiv.jpg
    (with the map applied, with "show normals", and the cage)
    Is there anything else that has to be done to make that work?
  • EarthQuake
    fly_soup wrote: »
    When I export a box with the sides on different smoothing groups with the xNormal SBM exporter from Max, I get the same result whether the cage is exported from Max without changing it or I go in and weld the cage verts in xNormal. It looks like this:
    nufiv.jpg
    (with the map applied, with "show normals", and the cage)
    Is there anything else that has to be done to make that work?

    This is *most likely* because you are using hard edges on everything, but your unwrap isnt split at those edges(ALL EDGES, on something like a cube).
  • Vrav
    Offline / Send Message
    Vrav polycounter lvl 11
    EQ (or someone), would you please show me the contents of an obj exported with proper hard edges defined? Are there more vn entries than vertices? Or does it use s 1 [faces] s 2 etc in declaring the geometry?

    It seems like most importers ignore vertex normal information when importing obj, but if xN does it differently maybe there's a way to parse something with broken edges (all I can create without using Max) into something welded but with hardened normals.
  • [Deleted User]
    Offline / Send Message
    [Deleted User] polycounter lvl 18
    Ahh, yes, that worked. I had it unwrapped the other way. Thanks, EQ!
  • MRico
    Offline / Send Message
    MRico polycounter lvl 10
    Hey fozi, any updates? I'd love to see a version working for unreal.
  • Eric Chadwick
    What's the current best method for baking UDK normalmaps?
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    This is a little OT but i figured it was the best place to post.

    Whilst trying to wrap my head around this thread (awesome btw), i thought i would do some tests in Blender 2.49b
    Could i get a general opinion on this bake?
    Is it safe to say that Blender is borked as well or am i just seeing triangulation errors?

    It the model that EQ posted a few pages back with no hard edges and both models set to smooth before the bake.

    th_Normal_249b.jpg th_Normal_249b_n.jpg

    Thanks for looking :)
  • Racer445
    Offline / Send Message
    Racer445 polycounter lvl 12
    Oh also I noticed that xNormal has a plugins page with selectable tangent basis calculators. The only ones I can find are the default one and the nvidia meshmender plugin. I'm surprised nobody has done anything with this yet.
  • Eric Chadwick
    8Monkey Labs has, in fact I think EarthQuake was the driver for this feature.
    http://www.8monkeylabs.com/technology
  • Harry
    Offline / Send Message
    Harry polycounter lvl 13
    EarthQuake wrote: »
    You can fix the hard edge split problem in xnormal, but it involves an overly complex workflow.

    you need to:
    1. load the 3d viewer
    2. turn on the cage editing tools
    3. select all of the verts
    4. weld all verts
    5. expand the cage as you normally would
    6. save out your mesh as .sbm(or whichever xnormal format it is, i dont rememeber)
    7. make sure XN is loading the correct mesh
    .....
    8. repeat process any time you change your mesh =)

    All this to get the same functionality as what is essentaily a 1-click option in maya/max.

    i absolutely hope the xnormal guy is reading this and making it a 1-click option in xnormal. that would really unbreak my heart
  • cw
    Offline / Send Message
    cw polycounter lvl 17
    This is a little OT but i figured it was the best place to post.

    Whilst trying to wrap my head around this thread (awesome btw), i thought i would do some tests in Blender 2.49b
    Could i get a general opinion on this bake?
    Is it safe to say that Blender is borked as well or am i just seeing triangulation errors?

    It the model that EQ posted a few pages back with no hard edges and both models set to smooth before the bake.

    th_Normal_249b.jpg th_Normal_249b_n.jpg

    Thanks for looking :)

    is the viewport grab orthogonal not perspective? 'm thinking there is usually a problem in 'user' view in max with viewport shaders so maybe that is a factor?

    Having said that those smoothng errors are reminiscent of the others I have seem from 3dsmax using this test geometry.

    good luck! :D
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    @cw,
    Thanks for the reply!
    The screenie is in Ortho but the errors also appear in perspective view . :S

    If i can confirm this is an error on Blenders behalf, they should fix it pretty quickly i would think.

    If anyone wants to take a closer look,you can grab the .blend HERE
  • yiannisk
    Offline / Send Message
    yiannisk polycounter lvl 14
    Ok. I know i will sound like an obnoxious smartass again to many. But the result after 17 pages was the obvious "you have to generate the normal maps in a tool\way your engine supports"?

    Eric we do not use any "special" workflow or tool. Create the model in 3dsmax or maya, bring it in zbrush, sculpt it, bring it in max (retopologize when needed) generate the normals, touch up if needed in photoshop, sometimes overlay some specific pattern or other detail, then use maps in the engine. We might occasionally additionally boost some color channel (blue) in the engine to enhance the effect and augment the final result by the use of a generic detail map especially for environment materials.

    Settings vary depending on the model, the amount\type of detail, and ofc the engine. There is no one fits all for settings.

    UDK documentation offers some nice tips on the creation and treatment of normal maps from max and a little troubleshooting regarding various mystical accidents that might happen in 3dsmax.
    http://udn.epicgames.com/Three/CreatingNormalMaps.html

    Something important is to assign your normal map the appropriate compression setting!!! some people disregard how important this is, and the right streaming channel.
    http://udn.epicgames.com/Three/NormalMapFormats.html

    The uncompressed format often gives crispier results despite the fact that its resolution goes down to half. Try it.

    I hope this was helpful.
  • Eric Chadwick
    Answering my own question, Epic seems to use mostly Max's RTT.
    http://wiki.polycount.net/Normal_Map#T

    Edit... ah, thanks yiannisk
15681011
Sign In or Register to comment.