Home Technical Talk

handplane - new tangent space tool in public beta

124

Replies

  • Farfarer
    Object space normal maps are absolute - they give the explicit normal that the high poly surface is facing in.* The low poly model doesn't have to do anything other than record that in it's UVs.

    Tangent space normal maps are relative - they alter the normals of the low poly model to point up/down left/right of the current surface normal of the face they're applied to (technically it's more complex than I've described - tangent space is a matrix calculated per-vertex or per-pixel, rather than per-mesh like object space is, and vectors are transformed from object to tangent space there - but it'll work for this example). This is recorded by creating a new "space" called tangent space, which is relative to the normal of the mesh and the direction of it's UV coordinates. Which is why it's so tightly linked to the UVs and normals of the mesh - changing any of those after baking, distorts the tangent space and so the normal map isn't quite right.

    Object space isn't useful for tiling textures because it's an absolute rather than relative value. i.e. if you applied an object space tiling texture to a cube, the normals would all still point in exactly the direction the normal map dictates. Whereas a tangent space one the normals point relative to the surface it's applied to.


    *Ok, that's not entirely true, object space is still relative, but it's relative to the transform of the model and doesn't depend on the geometry contained within the model. World space is truly absolute. If your low poly mesh is at rotation 0,0,0 then object space and world space are identical.
  • S_ource
    Offline / Send Message
    S_ource polycounter lvl 9
    And also i think that local space normalmap do not work good with mirroring.
  • fearian
    Offline / Send Message
    fearian greentooth
    Ah, that's sorted it out in my head! I getcha!

    would there be a possibility of a game engine converting object space normal maps to tangent space normal maps in editor when the game is 'compiled', similar to say, baking lighting or updating bsp geometry.

    I mean, for udk or unity or whatever to be updated with an equivalent of hand-plane inside of it tailored to the engine. I guess a pipeline like that would carry its own problems, but it would alleviate pressure from studios to develop a synchronised pipeline in house.
  • EarthQuake
    S_ource wrote: »
    And also i think that local space normalmap do not work good with mirroring.

    You mean object space, local space is actually another word for tangent space(see: idtech).

    But you're right, object space normals can't be mirrored unless you have a special shader set up to do it correctly.

    Rotating your mesh(in your 3d app), or creating rotated instances doesn't work even if you have a shader that works with mirroring.

    Animation does work, despite common understanding, your shader just needs to keep track of the vertex transformation.

    You can also rotate models with object space maps, again contrary to common understanding. However, you need to rotate the object in a level editor that keeps track of transformation. So you can't do it in Max or something like that.

    That should sum up the usual object space misconceptions.
  • S_ource
    Offline / Send Message
    S_ource polycounter lvl 9
    Okey, i thought it was the same thing local and object since you check local when you bake object space in 3ds max. :)

    And also a alternative to convert a tangent normalmap from one engine to another would be great.
  • Ace-Angel
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    Hey guys, I was wondering if I could drop a message with some files a friend of mine who is trying to mod a game is having issues.

    He forwarded me the files, and I tried getting HP to spit out a tangent map from Object an map, but it doesn't seem to be working, no matter what combination I try. I also tried several versions of Max each with different FBX options, but no dice.

    Cheers.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Yeah, forward it over and I will take a look soon. alecmoody@gmail.com

    Also, what errors is he having? 99% of the time it is either a y/z flip issue (map comes out weird colors).
  • Ace-Angel
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    Cheers Alec!

    From what I gathered, he's trying to make some custom body parts in Skyrim on some user-based mods, which were baked in XN since that's what most peeps use.

    I tried flipping around the certain channels, but they don't seem to be changing anything? I'm just getting hue shifting (EI: Green hues become Red, etc).

    I tried everything by the way, but being the pillock that I am, I wouldn't be surprised if I forgot one combination or my FBX export was wrong :P
  • Skiffy
    Offline / Send Message
    Skiffy polycounter lvl 15
    I love this tool. Its made my life so much easier. And oddly enough the unreal based normals render better with realtime shaders inside 3dsmax than the default 3dsmax versions.... madness say.
  • hamzaaa
    Offline / Send Message
    hamzaaa polycounter lvl 11
    Hey guys,

    I just want to wrap things up in my head and recite them here just to be safe I get it correctly.

    So all these techniques going on with UV splitting, Smoothing Group splitting etc. were all done because of the tangent bases not being in sync from baker and engine. Now with handplane being able to convert an object space normal to a tangent space normal map this process becomes obsolete. This is because object space normal maps ignore the surface normals of the low poly object (I would be happy with a deeper explanation.. ^^).
    This makes it possible to basically leave the lowpoly object on one smoothing group, even where UV splits and >=90 degree angles are (is this correct? Are there exceptions? Examples?). There is still the need for supporting geometry to get a correct projection going but this supporting geometry can be removed after baking as long as the UVs stay the same. Object space normal maps themselves can't be mirrored, but with handplane converted tangent space normal maps it indeed is possible.
    Did I forget something or got something wrong? Please raise your voice! :)

    Another question: Will there be a CryEngine3 support or how is handplane working for that engine?
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    hamzaaa wrote: »
    Hey guys,

    I just want to wrap things up in my head and recite them here just to be safe I get it correctly.

    So all these techniques going on with UV splitting, Smoothing Group splitting etc. were all done because of the tangent bases not being in sync from baker and engine. Now with handplane being able to convert an object space normal to a tangent space normal map this process becomes obsolete. This is because object space normal maps ignore the surface normals of the low poly object (I would be happy with a deeper explanation.. ^^).
    This makes it possible to basically leave the lowpoly object on one smoothing group, even where UV splits and >=90 degree angles are (is this correct? Are there exceptions? Examples?). There is still the need for supporting geometry to get a correct projection going but this supporting geometry can be removed after baking as long as the UVs stay the same. Object space normal maps themselves can't be mirrored, but with handplane converted tangent space normal maps it indeed is possible.
    Did I forget something or got something wrong? Please raise your voice! :)

    Another question: Will there be a CryEngine3 support or how is handplane working for that engine?

    I think you have the right idea. However, with a few of the outputs there are still minor shading errors and you will still need to use some amount of supporting geometry or shading splits in order to get a perfect result. It isn't 100% clear in some of the cases why it is happening, we can test the tangents at each vertex and see that they are correct, and we have the interpolation working correctly. Most likely in each case it is something going on in the shader code that we can't compensate for. James O'hare pointed out that the unity errors are from the shader not normalizing per pixel (per vertex).
    Also, I would not suggest anyone bake all their assets on 1 smoothing group. That is generally a bad practice since you are forcing your normal map to work really hard (check out all the gradients) and you will have more issues with mip maps and creating lod stages. I would at least split smoothing along all of your UV border edges (except edges in which two shells make up a smoothing contiguous surface).

    One really cool workflow advantage with handplane is that you can bake in xnormal without custom cages, just tell xnormal to smooth your lowpoly model so it will project correctly.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Just saw this:
    Another question: Will there be a CryEngine3 support or how is handplane working for that engine?

    Cryengine should be correctly synced to 3dsmax tangent basis so handplane's 3dsmax output should work correctly with it. We haven't done any testing yet and would appreciate having someone who works with that engine chime in.
  • hamzaaa
    Offline / Send Message
    hamzaaa polycounter lvl 11
    Yeah I will definitely try that out with CryEngine 3. I will posts my result here as soon as I am ready. Thanks for your answers!
  • Marcus_Aseth
    Offline / Send Message
    Marcus_Aseth polycounter lvl 9
    One really cool workflow advantage with handplane is that you can bake in xnormal without custom cages, just tell xnormal to smooth your lowpoly model so it will project correctly.
    Baking without have to set up a cage sounds awesome,can I have more information about how to do that pls :)
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    sure-
    Xnormal's min/max ray distance method for creating projections will split the projection along any unsmoothed edges (which creates double edge issues). In the past the only way to deal with this was to either not use any smoothing splits on your models (not a good option and super limiting), or bake with a custom cage (either made in or imported into xnormal). handplane uses object space bakes and object space bakes aren't affected by your lowpoly normals. This means that in xnormal you set the lowpoly model options to force smooth your mesh and bake correctly with the min/max ray distance settings.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Any chance we'll get more options for different app choices for baked in? I'd like to be able to use softimage and not have to export to xnormal, I didn't realize there was variation in how these are generated as well.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    "Baked in" is just a combination of channel inversion and y/z flips so any object space normal map can be made to work. We can add a softimage option in the future if you can tell us the +/- orientation of each axis. The latest build has max, maya, and xnormal as options, one of these will most likely work.
    That said, we have done zero testing with xsi fbx files. I would guess that they would work since autodesk owns all three applications, just make sure your fbx plugins are up to date. Also, please post your results here.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Yeah none of the application presents generated a normal looking tangent space normal map.

    eyE9s.png

    R: X+
    G: Y+
    B: Z+
    (I'd assume white = +)

    Maybe there should be a custom option that lets you match the rgb +/-?
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    ZacD wrote: »
    Yeah none of the application presents generated a normal looking tangent space normal map.

    eyE9s.png

    R: X+
    G: Y+
    B: Z+
    (I'd assume white = +)

    Maybe there should be a custom option that lets you match the rgb +/-?

    Hey zacd,
    if you hit "more options" there are coordinate swizzles on next to the object space import. Don't flip around the second set on the tangent space output. Also, the flip y/z button is a 90 degree rotation of the shading, and not equivalent to inverting a color channel. You can also just send me your fbx and object space map and I will figure it out. alecmoody@gmail.com
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    these settings work for the softimage normal map and the provided fbx file:

    The whole flip y/z thing should be much clearer in the future as we have a reliable system to auto detect the problem and warn the user. Part of what makes this issue more complicated is that we can't guarantee the y/z orientation of the fbx file. Even if we know what the object space map was baked in, we don't necessarily know what software the fbx was exported from. We will add a softimage drop down to the list of options to make things clearer. In reality, xnormal, maya, and softimage all bake object space maps the same way so switching between those options won't actually be doing anything.
  • mospheric
    Offline / Send Message
    mospheric polycounter lvl 11
    Ran some tests and got good results without using a cage. This tool is going to speed up workflow so much :D Thanks Alec!
  • Farfarer
    Been playing with this recently - it's very handy. Nice to be able to rely on the solidness of object space maps.

    My only issue is the wording of the interface.

    It works but doesn't often leave me feeling confident that what I've picked is right and what the options I'm fiddling with are really doing. The presets obviously have default values that work - but I'll be damned if I know what they are because all of the options remain at "Default". Is it worth removing the "Default" option and showing the actual settings?

    Perhaps rewording "Identity" and "Inverted" with +X/-X for red, +Y/-Y for green, which I think are terms more people are familiar with? "Flip Z/Y Up" perhaps given two explicit options; Z Up and Y Up (removes the "flip" word which feels like a gamble).

    They're suggestions - most folk will likely just prod stuff until something works for them then hopefully save it as a preset - but I'd like to know what's being switched or inverted.

    More presets for major 3D packages would be nice, but allow people to create their own "Baked In" presets, too.

    Oh, and this is a small personal preference, not sure how others feel about it, but an option to replace the suffix, rather than append it. I've been saving object space normals as model_normal_obj and I end up with model_normal_obj_source rather than model_normal_source or model_normal which I'd prefer.
  • Ged
    Offline / Send Message
    Ged interpolator
    good suggestions.

    I would like to try this at work but Im using maya on a mac, will this be released for mac?
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    I'm having trouble with displaying normal map in Ce3. I modelled in Modo. Baked in xNormal. Converted normal map in Handplane to 3ds max (since from what I know it's synced with ce3... ?). I used converter fbx => cfg for ce3 to import mesh. Used Photoshops crytiff plugin to export normal map into tiff. And that's what I get

    Ce3 result :

    KvdWMV.png

    Here is normal map:

    jpm6Pe.png

    And here are meshes. I used 1 smoothing group since I want to check this synced workflow.

    5Je50C.png

    Any idea why it looks so bad in Ce3 ? I tried it also in 3ds max. With quality modifier from 3point studio it looks perfect. Also in xNormal viewer all vertex normals are ok. When I don't apply normal map in Ce3 it also looks like it's 1 smoothing group (I don't know if you can view vertex normals in ce3... ?). I don't think fbx => cfg exporter destroys normals. Since normal map works in 3ds max I don't think I can blame normal map... Any idea how to make it work ?
  • rolfarth
    Looks really sweet. Will try this workflow out! Will post some images later on with the results.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Mronetwo,
    Thanks for posting crytek tests, I would really like to get this working. I haven't ever worked with cryengine but I know that in the past it wasn't correctly synced with max. My understanding is that at some point recently that has changed. I would make sure you are on an up to date build. Also, the other big thing that could be happening here is triangulation not getting passed correctly through the fbx->cfg conversion. Try triangulating the mesh, exporting an fbx, generating your tangent space map, and then bringing that mesh into ce3. There are a bunch of other things that could be happening in the 3dsmax to cryengine pipeline but first lets eliminate those two possibilities.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Racer445 did some testing for me today on max to ce3 baking and it is definitely not synced correctly. Also, it is now my understanding that ce3 splits smoothing along any UV border edge, this means you will need to do the same prior to baking (or using handplane) to get a reasonable result.

    This was the result of racer's testing baking out of max. Keep in mind there there are a lot of smoothing splits being used here and the result still isn't great.
    ce3_bake.jpg

    Assuming the tangents are correct at each vertex, the likely causes for these problems are ce3 not using the correct interpolation method for the tangents across a triangle or some funny business in the shader code messing up the result. If it is just interpolation, we can fix that with handplane. Otherwise, it is a needle in a haystack problem.
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    Thanks for you help Alec.

    Here are my results with triangulation.

    OsR7b5.png

    Looks better but it behaves like it would in 1smoothing group and not synced workflow. Also it is one smoothing group but like you said it breaks smoothing wherever the seams in uvs are.

    y9MMUU.png

    You can see that there is break in smoothing where uvs aren't continous (left side). It is smooth where top face is connected with side face (right side of the cube). Here are uvs to show the only edge where smoothness continous from side to top.

    e9X2u5.png

    So is there any way to use synced workflow in Ce3 ? I think I will jump to UDK since right now it's really long process to import asset into engine and still I can't benefit from you great tool <.<
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    I think I'm just terrible with Handplane... :P I tried with UDK and I can't make it work... my normal map after Handplane looks like that

    8SDeqS.png

    No gradients just flat normals... I used Modos fbx. Baked in xNormal and then coverted into tangent Ue3. I got normal map which looks like object space but with tangent space color palette ;p.

    Here is object space:

    yylST4.png
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Your lowpoly model's smoothing information isn't being read. Modo's fbx files are a third party implementation and it looks like they don't store smoothing information the same way as the autodesk products (max, maya, softimage). Do you have access to one of the autodesk tools? It could also just be the export settings.
    I would like to get modo support working at some point but it is lower on the priority list. I did a quick look on google and didn't find any documentation for how their fbx format works so getting modo working will likely require custom i/o plugins. We also don't have a modo license to test with.
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    I actually checked exported meshes. When I imported Modos fbx into 3ds max 2009 it maintained smoothing information but it lost uvs. 3ds max 2013 read both uvs and smoothing. I also tried it in Marmoset and checked in xNormals viewer. The normals looked like they should. When I imported mesh into 3ds max 2013, checked smoothing exported again and used Handplane the normal map looked more like it should. I will make some additional test to check if 3ds max fbx works with the same geometry from modo (which doesn't work).

    In Modo you can't really change much in export settings. I can check and un check smoothing information (and it's checked of course). Thanks for your help. I will soon post results with the same geo but 3ds max workflow.
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    Ok I tried max workflow. It works ok. Here is the result:

    dAzctO.png

    I noticed minor shading problems on side faces. Can I avoid it or is it maximum I can get from this workflow ?

    Interesting thing is that I first exported not triangulated mesh. Results form Handplane looked like that

    TvvGAx.png
    Same normal map I used in previous screenshot from UDK.

    Then I remembered I should triangulate the mesh and thats what I got:

    DHfHW6.png
    I checked it few times and it is the same mesh but triangulated. Don't know what to think about that one ;p

    Sorry for the alpha in those screens...
  • almighty_gir
    Offline / Send Message
    almighty_gir ngon master
    been getting some really good results from xNormal > unity with handplane. will be trying out some stuff in UDK soon i hope!

    awesome tool guys!
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    MrOneTwo:
    Try collapsing your modifier stack and converting to edit poly after triangulating. I have noticed that max fbx files are sometimes picky about sending smoothing groups (at least moving from max to maya). As far as the overall quality goes, It should be comparable to the xnormal/imported tangents workflow that was added last summer:
    unreal_comp.jpg

    Your image looks a little worse but it could just be the lighting/materials. Make sure you aren't importing tangents, using explicit normals, and that remove degenerates isn't turned on (also get the triangulation working).
    From what I have been told, Unreal is throwing away precision in the render code in a way that makes it impossible to compensate for. That is based off of talking with developers who have full source access - I don't know the specifics. The big advantage of the handplane workflow for unreal is that you can use the improved shading with skeletal meshes.
  • Farfarer
    For what it's worth, I've had no issues with modo exported meshes and handplane (it's what I use to do all this stuff with). Normals export fine with FBX.

    Modo 601 SP3, by the way.
  • mospheric
    Offline / Send Message
    mospheric polycounter lvl 11
    Hmm I'm having issues. I can't get handplane to generate a decent tangent map using an object space map.

    I start with this:
    7wJbI.jpg

    And I get this:
    5fDaT.jpg

    Using Max 2013 with the newest XNormal/FBX exporter.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    that is a y/z flip issue. This problem will be automatically addressed in the release build but for now:
    More options button -> Flip Y/Z

    This cannot be accomplished by inverting channels
  • mospheric
    Offline / Send Message
    mospheric polycounter lvl 11
    Thanks for the quick reply Alec! I tried using Flip Y/Z only to get worse results. :/ I've tried various FBX exports, OBJ exports, Xnormal bake settings, etc.

    aeGq3.jpg
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Email me the files and I will take a look
    alecmoody@gmail.com

    I need the fbx and object space bake, the scene file would be helpful too.
  • roosterMAP
    Offline / Send Message
    roosterMAP polycounter lvl 14
    cant do 8k? :(

    is there a way to force it to?
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    are you getting this error?
    "error at 8k D3dXCreateTextureGutterHelper Failed!"

    That's a good bug and will be fixed for the final build. We are planning the first week of January for release.
  • mospheric
    Offline / Send Message
    mospheric polycounter lvl 11
    Finally got a good bake and conversion! The problem may have been creating the .obj and .fbx at different stages of my model creation. This resulted in the blue channels being inverted (thanks Alec!). I reexported both and baked an OS map in Xnormal. Officially part of my workflow now!

    wIXNs.jpg

    Still a few errors in the bake, but that's my error now :P
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    Still have problems with Handplane. I noticed that everytime I export fbx form Modo or 3ds max 2009 triangulated it gives me results like in post #183. If I import the mesh into max 2013 and export into fbx (triangulated) it works fine. It reads smoothing information fine with 2013 export but has problems with Modos or 3ds max 2009 triangulated meshes.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    are your max 2009 fbx plugins up to date?
  • MrOneTwo
    Offline / Send Message
    MrOneTwo polycounter lvl 12
    Ohhh I didn't know there is a plugin for max 2009... Thanks man! : ) I updated fbx plugin to 2009.3. Now everything works fine except I checked Import tangents on, Remove degenerates on and Explicit normals off. From what I understood those are settings opossite to what you wrote in post #185. I hope I'm not creating problems which are not there because I don't understand some simple thing... (I think I understand it though...).

    ndrT6H.png

    Does this also mean that Modo 601 sp3 uses fbx2009.00 ? Farfarer said he didn't have any problems... Am I that special ? ;p

    Edit: I found out that since 401 Modo uses fbx higher than 2009.3... so I still don't know what I'm doing wrong with Modo.
  • duke
    Offline / Send Message
    duke polycounter lvl 13
    Could I suggest you provide a reference model (a teapot) that we can take into the host app, bake, and feed the result back to handplane so it can work out the swizzle wizzles?
  • duke
    Offline / Send Message
    duke polycounter lvl 13
    Allllso:

    1) Toggle below destination file field: "Same as Input Map" - to use the same output directory as the input.

    2) Load/Save buttons - same as xnormal so I can save settings to bake a particular file and simply reload that file if I need to rebake, rather than relocating files.

    3) Obj model format support - unless this would be inappropriate because it doesnt contain enough data.
  • Farfarer
    One other request - is it possible to have it store the settings inside of a file that's kept with the exe, rather than in Windows registry?

    I keep little tools like this on my USB stick (along with modo) so I can have it with me and keep all the settings consistent across workstations.
  • Computron
    Offline / Send Message
    Computron polycounter lvl 7
    Modo is portable? Does it require any special permissions or anything along those lines?
  • Farfarer
    Takes a little fiddling, it's harder now that you can't give a relative path to the license file since 501, but yeah. Harder on Mac, though.

    http://forums.luxology.com/topic.aspx?f=37&t=68584
124
This discussion has been closed.