Home Technical Talk

Maya normal map bad shading, smoothing errors

I baked my normal map in maya and the results bad shading, smoothing errors.

i just read somewhere that i can fix this with "Raytrace renderer"

how can i do that in maya?

here's my Normal map

http://i48.tinypic.com/2a9wqys.png


Thanks.

Replies

  • EarthQuake
    Options
    Offline / Send Message
    Displaying in what? Images of the actual model will help.
  • haiddasalami
    Options
    Offline / Send Message
    haiddasalami polycounter lvl 14
    Yeah need to see the model :)
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 18
    Seems like quite flat areas have gradient issues. This is because the associated vertex normals differ from each other (are pointing in different directions).

    These gradient issues can cause artifacts, banding, larger file sizes, harder to overlay other details on, etc.
  • EarthQuake
    Options
    Offline / Send Message
    kodde wrote: »
    larger file sizes, harder to overlay other details on, etc.

    I'm curious as to why you mention these, as IMO they aren't actually problems. Unless you mean you have to use a larger texture to get the required precision when dealing with more gradated maps(which can be true). Otherwise this isn't really correct, as uncompressed, or compressed, the filesize is going to remain constant to the resolution, irrelevant of the image content.

    As far as harder to overlay details on, i've never really know this to be the case, can you expand on that?
  • sasuki
    Options
    Offline / Send Message
    Guys there is an option in MAX when you bake a normal map "RayTrace"

    My question dose Maya Have a Raytrace option for normal map baking.
  • EarthQuake
    Options
    Offline / Send Message
    All normal map bakes are "raytraced", i dont think you understand the question you are asking. Please post some images of the actual model with the normal map applied.

    EarthQuake wrote: »
    Displaying in what?

    This also still needs to be answered.
  • sasuki
    Options
    Offline / Send Message
    EarthQuake wrote: »
    All normal map bakes are "raytraced", i dont think you understand the question you are asking. Please post some images of the actual model with the normal map applied.

    Ok here is the model with the normal map applied

    98g0tl.png

    9s6jki.png

    24o91j7.png

    10wnw9t.png

    and here is the whole normal map

    rro76h.png
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 18
    EarthQuake wrote: »
    I'm curious as to why you mention these, as IMO they aren't actually problems. Unless you mean you have to use a larger texture to get the required precision when dealing with more gradated maps(which can be true). Otherwise this isn't really correct, as uncompressed, or compressed, the filesize is going to remain constant to the resolution, irrelevant of the image content.

    As far as harder to overlay details on, i've never really know this to be the case, can you expand on that?

    I was under the impression that you had a variable filesize based on the contents of lets say DDS. In that case as you said, you'd get less artifacts the more flat areas you have. If it's constant file size wont differ yeah.

    "Harder to overlay details on" was a clumsy way to put it. For one thing you don't have to bother with any form of mixing/overlays if your just adding details on flat surfaces as this, say an indented detail on a flat metal plate. Also you might have trouble with copy/paste or tile a certain region if it has a skewed base like these gradients.

    My personal preference is to not bother with these issues when working with organic models. But with hard surface if I can afford to have bevels I usually go with the superspecular manner and let the bevels take the smoothing and flat surfaces have a flat normal map color.
  • EarthQuake
    Options
    Offline / Send Message
    Kodde:

    DXT compression is a fixed size, a 24bit compressed image is always going to be 1/4th the size of an uncompressed TGA.

    I see what you're saying about overlaying etc, however i dont know that i would often do it the way you mention, as even if my lowpoly normals match the high very well, i'm likely to have some areas that are a little curved or something.... To me having a good way to overlay is simply a must, and i wouldn't want to rely on a more complex and also more limited workflow.

    Sasuki:
    The reason you're getting smoothing errors is because the app you are Baking in(maya) and the the app you are viewing in do not have the tangent bias synched up. This means that each app reads the normal map data a little differently, so you'll get inconsistencies in normal direction which result in smoothing errors.

    To fix this there are a couple things you can do, first you can use a program/shader that syncs up properly to your bakes, in your case a maya viewport shader would be a good call. There is also 3ps shader with quality mode for max: http://www.polycount.com/forum/showthread.php?t=72861

    There are a couple things you can do with your geometry as well, the first and probably easiest thing is to simply use some hard edges to cut down on the extreme angles of your mesh normals, and of course you must remember that anywhere you use hard edges, you also need to split your uvs. Look at this thread for more on the UV issue: http://www.polycount.com/forum/showthread.php?t=73593

    You can also bevel edges on extreme angles, however if your baker and viewer aren't synched up, this will only "help" the problem but never "fix" it entirely, you'll still get slight errors or even quite obvious errors in some cases, you should experiment with this, and you'll likely want to do a little bit of both methods.

    There is another method that Kodde mentions, which you can find here: http://www.polycount.com/forum/showthread.php?t=66139 Personally i wouldn't bother with this, as to me its sort of a weird workflow and iwould prefer to deal with these issues using standard modeling tools and not have to bother with clunky workflows to manually edit vertex normals, especially on a complex mesh.

    This is a thread for max, but explains a lot of the basic principals:
    http://www.polycount.com/forum/showthread.php?t=68173
  • sasuki
    Options
    Offline / Send Message
    EarthQuake:

    Sorry for this dumb question, I bake my normal maps with 3Point Shader in max?


    or 3Point Shader is only for model presentation?



    here's how it looks with 3Point Shader


    2it0mx3.jpg
  • 3DRyan
    Options
    Offline / Send Message
    3DRyan polycounter lvl 8
    I had the same problem with my bakes when I was first learning normal mapping and such. I was using the low poly version of the model to bake onto, which was bad. Each edge in a normal bake starts a gradient, and ends when it meets the next edge. To solve this, duplicate the low poly and sub divide it. Use the sub divided model to bake onto. If you took the model into zbrush or mudbox previously, you should have such a subdivided model to begin with. As long as you keep your uv borders intact, the normal map will work when applied to the low poly no problem.

    This is how it works for me. My models go from maya to xnormal, so I'm not sure if this applies to other programs.

    Hope this helps.
  • EarthQuake
    Options
    Offline / Send Message
    Sasuki: If you plan on viewing your model with 3ps Shader, you will need to rebake in max. Loading your normal map from maya will give you the same problems as baking in maya and viewing in marmoset(again different tangent biases being used in each app will cause issues). If you rebake in max and still have problems with 3ps Shader, read over the website/polycount thread and make sure you've got everything set up correctly.

    If you want to stick with baking in maya, your best bet is to view in maya, even as simple as throwing a blinn material on your mesh and turning on "high quality" in the viewport should render your mesh correctly, though there are more complex shaders you can download.

    3dRyan: That is basically something you should never do. Baking on a mesh with completely different topology and mesh normals is simply a terrible idea, and is not advice i would recommend *anyone* follow. I suggest you read my earlier response to Sasuki and the links provided so you can understand the issue a bit better.
  • 3DRyan
    Options
    Offline / Send Message
    3DRyan polycounter lvl 8
    Really? The only thing that's different about the two meshes is that they're subdivided. The UV layout borders remain the same. That's how I've always done it, and it's never given me any problems, at least in the UDK.

    Sasuki, I guess it just boils down to preference. Do whatever works for you. It'll take awhile to find a good workflow, but when you do, it's fairly smooth sailing from there. Just research the crap out of normal mapping on the web, and you'll eventually find what you need.
  • EarthQuake
    Options
    Offline / Send Message
    Sub-dividing the mesh changes the mesh normals, and will introduce new/different smoothing errors yes. If you manage to change the mesh normals of only the projection mesh, but not the lowpoly mesh itself, this would work. However i do not know of any way that can be done or?

    Even if you could only change the normals of the cage, this would only help with "skewed" details, and do nothing for smoothing errors.

    Here is an image to go with the point, you see the math simply does not work this way.

    edgeremovenormaltest.jpg

    The normal map is baked to the exact normals/tangents of the lowpoly mesh, once you start editing those normals(removing geometry, changing smoothing groups, etc) after the bake, you're introducing even worse errors.
Sign In or Register to comment.