Home Technical Talk

Why you should NOT trust 3ds Max's viewport normal-map display!

15791011

Replies

  • CJE
    Offline / Send Message
    CJE polycounter lvl 11
    Where are you supposed to enter the commandline info for the converter to work? I tried nspace.exe but whenever I try somethign the window just disappears.
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 16
    well actually the normal maps, max tangent + nspace tangent, should match or?
    Which they don't do now.

    I thought the purpose of the tool was being able to translate into another tangentspace of the app you don't use/have. And to say benefit from speed-ups in xnormal or whatnot. So you can take any baking tool you want, and generate tangent-space maps for any other tool.

    the max viewport != scanline issue is something else
  • EarthQuake
    What i'm saying is that, if he's converting to the max "realtime" result, but displaying it in the scanline renderer, its going to look wrong, and the default max TS bake is going to look right, which is what his image seems to represent. Which makes sense, as the tool isnt here to improve the quality in the offline renderer, but in realtime.

    [edit] Yeah it says is his image he is using scanline, which explains exactly why it isnt working. =)
  • Eric Chadwick
    fozi wrote: »
    Keep in mind though, the shader fix only works if you use normal maps converted by NSpace, which is prepared to output "Normal Bump" compatible tangent maps.

    I assume this means they should render correctly in scanline using the Normal Bump map. When I get a fixed shader, I'll try it in the viewport. I tried doing the fix myself, though it's a bit beyond me at this point.
  • rollin
    Offline / Send Message
    rollin polycounter
    CJE: read what I had written above

    you run this tool per command prompt like

    program.exe -inputvar1 value1 -inputvar2 value2

    so you need to start the command prompt or crate a bat file in which nothing else is but your input command and the way it should look can be read from the readme



    --

    yes .. we should call the tool "normalmap format converter" the other thing is the "3ds max viewport scanline oddness" ;)


    btw eric: as far as I understand you don't need a "fixed" shader bc the converter does already transfer into the max viewport normal format ..
    so normal-bump is already enough
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 16
    would be good if a second thread was split from this one? with the title rollin suggested
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    Eric: I just assigned the normals to a regular material in 3ds max (like you would for anything) and then turned on "Show hardware map in viewport".
    One thing I didn't mention though, before I exported from 3ds max I had to triangulate the mesh.
    Now when I applied the normals to a non-triangulated mesh (the original mesh) it showed up a little off at first. I quickly found that if I turned 1 edge which didn't match the original triangulated mesh, it worked perfect. In-short, triangulate your mesh, but when you apply your normals to a non-triangulated mesh, you have to make sure all your edges match up, or else there will be problems.

    Now I'm sure there is still limitations to this, but that's what I need to find out next. How far can I push this? I mean, you can't just go ahead and assign a gun for example 1 smoothing group and use this method and get a perfect normal map, can you?

    CJE:
    So your baking the normals using world space instead of Tangent space and then converting it back to tangent space after?
    Yes! :)

    Yeah, the same thing happens for me with the command line. I commentted about this a page or two back, and posted the solution. :P
    Anyways, you need to copy the code to note pad. there are examples in the file as "usage.txt" and you can have a look at "from_max.bat" by opening it with notepad.
    Basically you take the code, example:
    nspace -m input/BulletTest02_Advice.obj -n input/BulletWSN_Advice2.tga -on output/BulletTSN8_Advice2.tga -op 8

    Make sure the names match the files you need:
    object= BulletTest02_Advice.obj
    world space normal map = BulletWSN_Advice2.tga
    converted tangent space normal map = BulletTSN8_Advice2.tga

    Then you simple save the notepad you just opened and change the ".txt" extension to ".bat"
    this simply makes the program (I think its and executable), so when you click on it, it will run nspace.exe
    and convert the files you filled in.
    So just make sure you create that .bat file and have the correct names in there and your set. Piece of cake. If you don't understand this, hang tight, because I think fozi is going to make an easy to use program with an interface for us.

    [edit] Oh yeah, and if you see the program run real quick after this, then shut down. It crashed. The reason being will be because you didn't export your .obj correctly. Make sure you follow the example for export settings.
  • Eric Chadwick
    Ahh, "Show Hardware Map in Viewport" does work. Interesting. But it doesn't work with the 3ds Max tangent map, and it really doesn't work with the 3ds Max world-space map. Even if I add an Edit Normal modifier.

    Nspace_ShowHardwareMap.jpg
  • EarthQuake
    Right, which is correct no?

    Max tangent = broken by default, the reason for this thread
    Max World = Attempting to load a world space normal as tangent
    Nspace = "Fixed" tangent space that syncs to the correct realtime preview
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    So do you guys know of a specific situation where this won't work?
    I'm trying to understand the limitations of this workflow.
  • Eric Chadwick
    EarthQuake wrote: »
    Right, which is correct no?

    Max tangent = broken by default, the reason for this thread
    Max World = Attempting to load a world space normal as tangent
    Nspace = "Fixed" tangent space that syncs to the correct realtime preview

    Yes, if "Show Hardware map in Viewport" is the "correct" way. Is it? I would think FX shaders would instead be the gold standard. Although there is the whole mess of Max FX syntax vs. the FX syntax everyone else uses...

    I think it's still in flux Brad.

    Anyhow I think the whole point is to get normal maps out of Max and looking correct in your game engine of choice. Need to test these in Marmoset, UE3, CryEngine, etc.
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    I have my results for Marmoset:
    Not good I'm afraid. :(

    I guess there needs to be a converter for each specific engine that the normals are going, since max handles the normals differently than other engines.

    This is a big let down, because honestly I don't have any need for viewing normals in max like this, since ultimately it always ends up going into Marmoset or Unreal.

    Anyways, here are the results:
    MarmosetTest.jpg
  • fozi
    Hey guys, thanks for the feedback. Very appreciated.
    This is a big let down, because honestly I don't have any need for viewing normals in max like this, since ultimately it always ends up going into Marmoset or Unreal.
    Brad, don't bother testing on those yet. I released this version of the tool as a proof of concept.

    This tool doesn't solve Max's mismatch. For that you'll have to wait for CB's tool.
    It also doesn't work with Max's scanline renderer because the only target supported is Max viewport, and as we all know, they don't match.

    What it does is provide you a solid common ground, WS-normalmap-like quality using TS, that will hopefully match all targets (engines) supported by NSpace. However, you will end up with 1 tangent map for each target.

    I haven't had time to add any extra targets, but that IS the plan. I have a big task ahead of me. Not only do engines compute different basis, they may also split verts in different ways.

    Next features in order of priority:
    - Precision improvements for 24bit normal output. (wip)
    - A nice and simple visual user interface.
    - Popular targets: Unreal, CryE, Marmoset, Unity?
    - Smoothing groups / obj normals. Currently forcing only 1 sm group.
    - Multiple target output.
    - Per-pixel tangent basis, as a possible standard target.
    - Correctly combine several normal maps from different sources.
    - High resolution normal maps.
    - Ideas?

    I'll open a new thread once I release the next version.
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    Fozi: this is awesome! Your the man!
  • osman
    Offline / Send Message
    osman polycounter lvl 15
    Hey Fozi, I've had very little time to try it out, but I really like it so far, thanks again:).

    As for the GUI, I've made my own very quick GUI to make it easier for me to test this thing out, so if someone wants it until Fozi wraps it in a nice looking and easy to use GUI, here it is:
    http://gameartist.nl/wp-content/uploads/2010/02/ozNspaceGUI.rar

    Simply drop it in the same folder as nspace, and run it :).

    nspaceGuiTEMP.png
  • pior
    Offline / Send Message
    pior godlike master sticky
    Speaking of obj - is it a solid enough format for such conversions? I know it's fairly good at transferring raw basemeshes geo between apps, and even smoothing groups/vertex normals can be supported in some precise cases, but isnt it so badly implemented in most exporters/importers that it makes it a pain to work with?

    Especially when it comes to tangent basis.

    What would be a better format for that? .X, .SBM ? Maybe FBX ?

    For organics its all good but as soon as we start using the UV split/hard edge technique to make for stronger maps, isnt it going to be a nightmare?

    Anyways, the tool seems to be shaping up awesome! Congrats and thanks!
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    osman: Nice work with the GUI! However, I tried running it and I get a super fast message that says "input mesh file type not supported". I can only see the message for approx. .1 seconds then it crashes and closes.
    However, I am able to create a .bat file and the program works perfect. Windows 7 64 bit o.s. here. Any ideas?
  • ivars
    Offline / Send Message
    ivars polycounter lvl 13
    This is awesome fozi! What would be even more awesome, is if you could somehow input your own tangent-calculations.
  • fozi
    osman, excellent.

    pior, indeed. I've got an additional FBX loader ready to go.

    ivars, that feature was planned but it somehow slipped my mind.
  • Eric Chadwick
    osman, nice one!

    fozi I wonder if you could also add tiling support? My last example has a seam in the upper left corner, the edge lies on U0/U1. The Nspace output has different normals on either side, but the input has tiling normals.

    Would 16bpc/64bpp be high enough precision? I don't see a way to output 24bpc from Max. Unless you just mean you want to use a higher internal precision.
  • fozi
    Eric, I think I might be missing something here, can you provide a test model?

    16bpc should be more than enough. I already have support for it, both input and output. Ideally, the input would be a 16bpc WS map and the output a 8bpc TS map. Unless you want to do 16bpc to 8bpc conversion in photoshop for better results.
  • Eric Chadwick
    I think I simply misread your comment about wanting better 24bit support.

    I guess 8bpc would be the default, but it would also be helpful to have an option to output 16bpc TS, for massaging/editing in Photoshop.

    Test model here, max file is 2010.
  • osman
    Offline / Send Message
    osman polycounter lvl 15
    nSpaceGui.png
    Got a new version, waiting for Fozi to test it, and then I'll upload it or let him do it:).
    This one captures the console ouput, so it's easier to see the errors. They wont dissapear in 0.1 seconds hehe.
  • osman
    Offline / Send Message
    osman polycounter lvl 15
    Here it is, a slightly newer version of Fozi's tool. He said it was ok to post it up while he's doing more awesome updates to it:
    http://gameartist.nl/wp-content/uploads/2010/02/NSpacePack_02a_GUI.zip
    As you can see in the screenshot, you can now select what output format you want;
    rgb24 and rgb48
    (rgb48 only to .png atm)
  • FAT_CAP
    Offline / Send Message
    FAT_CAP polycounter lvl 15
    Just tried this and it is working great with the Normal Bump prievew inside MAX :)

    The ability to input the tangents from other shaders/ programs would be the icing on the cake though as I think most people will be using a dedicated .FX shader or some third part app to preview/ showcase their work.
  • fozi
    Eric, I realized that I can't add support for tiling. I'm doing a 1-to-1 pixel conversion. You'll have to rely on padding or texture wrapping in the shader.

    osman, nice work!

    FAT_CAP, I tried to match the tangent basis passed into the shaders by Max but was able to match only two of the three vectors needed. At the moment, there's no way I can do it without requiring a slight modification to the shader. I'll get back to you if I find a better solution.
  • Eric Chadwick
    OK, can work around the tiling issue pretty easily. Would love FX shader support in Max, if possible.
  • EarthQuake
    eric: you may be able to get around the tiling issue by using a simplified mesh on the tile section, IE: just a plane, or by offsetting uvs
  • Eric Chadwick
    Yeah, makes sense.
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    I see this thread got buried pretty quickly.

    Fozi: Any updates on the converter program?
  • bugo
    Offline / Send Message
    bugo polycounter lvl 12
    heh, we are going to the 15th page into one simple subject, thats why I love polycount. Yes, I would like to see more updates into this tool
  • fozi
    I'm working on output support for UE3/UDK. Possibly this weekend.
  • BradMyers82
  • FAT_CAP
    Offline / Send Message
    FAT_CAP polycounter lvl 15
    Nice to hear about the UE3 update Fozi :)

    Does this mean that you are closer to being able to let the user input their own tangents? I ask as this would be a great addition to a project which is mid way through, so there is no chance of changing their shaders to match the tangent space of which the normal maps are currently output at. If you can output the maps to "UE3 format" does this mean that being able to enter our own tangents and output the maps in the right "tangent format" is close?
  • fozi
    FAT_CAP, indeed.

    I'm currently working on a more robust, professional grade, cheap commercial app. I'll include all the features we discussed here, and then some. I'll give you guys a heads up once we enter the beta stage.
  • Eric Chadwick
    Looking forward to it!
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    Sorry to bump the thread perhaps un-necessarily, but my mind cannot resolve this nonsense -
    I just baked some normals in maya for the first time and it was a piece of cake and displayed just perfect (even displayed in max viewport with a dx shader just fine after flipping green channel of the tga!)

    please tell me if I am crazy...but this is how it seems to be-

    Max bakes normals which are only suitable for its scanline renderer. If you want to use max alone to create game art you will be forfeiting accuracy/correctness/quality because workarounds and wrangling will be needed to get stuff to look nice. (not saying it isn not possible, clearly people use max a lot and get by)

    It just seems really naughty of autodesk to have this feature working in such a shonky way! If there was a toggle for 'game/render' to toggle basis calculator I wouldn't feel so annoyed.

    I guess it's not such a big deal, we should all just buy all the applications so we can do things properly, or get (more) used to bodging things!

    grrr! sorry for the rant but having just used the MAYA equivalent of the RTT dialog I want to cry a little. The grass is always greener, but sometimes it seems there is something to it!

    someone sensible, please tell me I am not crazy..
  • Eric Chadwick
    You are not crazy. Maya bakes tangent space maps better.
  • undoz
    What's neat is that using a tool like fozi's you can do one bake of your model and then convert the it to whatever engine spec you like.
    I don't think that maya's normals are 100% compatible to every engine out there, so there is still an issue.
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    tis true but mirroring and other tricks are not object space friendly :(

    Ideally 3dsmax would work properly, but then again, that would give me nothing to whine about. :)
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    I have been baking tangent space maps for several years now, but I haven't really baked object / world space maps much until I started getting interested in fozi's converter.

    But anyways, if mirroring uvs is an issue with object space baking. Couldn't you simply move the uvs you want mirrored off the uv space when you bake. (offset 1 unit).

    Then create a tangent space map from the converter, and move your uvs back into place?

    I haven't attempted this, but I am assuming it would work, and you could mirror to your hearts content. :)
  • undoz
    Yes, offsetting should work. You don't even have to move them back.
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    cheers folks will give it a try. :)
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    well something strange happens when I do the usual tangent space routine of offsetting the UVs. I will have to dig in and triple check I didn't do something dumb along the way.

    The converted tangent space map is way too colourful compared to the maya baked tangent space map, suggesting something odd has occurred!
  • Eric Chadwick
    Then it's probably an object-space map instead. Can we see?
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    I can't post that example its under NDA :)

    The texture has changed from the OS map generated by max, but doesn't match the tangent map from maya, it's still very object-y looking.

    I'm not going to plough any more time into fiddling with this at the moment, I want to press on with the production work instead, confident that I can at least generate decent tangent maps using something which isn't max. :) Especially as it's an organic mesh and I was only having a try with fozi's magic for shits and giggles.

    I'll setup a NDA safe example and post examples in a mo so you can see i'm not simply crazy or a n00b. I've been doing this a while and it's mysteriously only now that the inaccuracy of the max RTT is massively bugging me!

    ok, no need for examples, I just backtracked and re-read some older posts (thanks BradMyers82!) and had a few things wonky in my .obj export.

    Now the converted texture looks much more like a TS map. Thank the lord. :)




    It's all working great now! Some tiny seams in the local->tangent workflow, the maya bake was perfect off the bat, so still winning. :) Either way I am happy enough for now - thanks for all the assistance chaps!
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    I might switch to maya, there's a chance I can convince work to get a seat or two in for just this sort of problem! I'll give xnormal a go as well, that might be a cheaper option.

    Thanks for your sharp eyes guys, it helped me get to the bottom of that issue and now i'm much the wiser. :)
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    I dont know what good it will do but I've been on a mission sending bug reports and allsorts to any email address I can find.

    Hopefully someone who cares will see it and that might look into this. I have all my appendages crossed! NOw i will stop thinking about this it is doing my head in. :)
  • Eric Chadwick
    Robin Ball pointed me to this thread on CGTalk. Good news, looks like Autodesk is now aware of the issue.
  • cw
    Offline / Send Message
    cw polycounter lvl 15
    yeah I was posting there and spamming off a few emails and bug reports over the weekend - looks like we might get something fixed! Fingers crossed! :)
15791011
Sign In or Register to comment.