Home Technical Talk

A short explanation about custom vertex normals (tutorial)

124

Replies

  • JamesFSky
    @mannyn
    Please, take my first born son as compensation for this gift. Tried, loved, saving your script on my Maya shelf fo evah
  • mannyn
    Offline / Send Message
    mannyn null
    how does it work on a curve?

    That depends on how your curve is structured. You'd want the inner facets of your curve to be smoothed (averaged, not soft!), on the sides that are facing laterally away from the curve (where your bevels are) those you'd want fixed with our script.
    JamesFSky said:
    @mannyn
    Please, take my first born son as compensation for this gift. Tried, loved, saving your script on my Maya shelf fo evah
    Haha! Thanks, but that's not necessary, but if you do work for a studio I'd love for you to check out the startup we've just launched: instaLOD.io , it does AAA-grade mesh optimization and saves you gazillion hours of work :smile:

    We've developed a bunch of other productivity and workflow scripts that we're using for Horror City, maybe I'll make a dedicated thread for them. There are some gems to be found!
  • uk_resistant
    Offline / Send Message
    uk_resistant polycounter lvl 17
    There used to be a script for Maya that did this but as far as I know, it isnt compatible with more recent editions. Is there a quick way to do this in Maya, without it being toooooo fiddly?
  • mannyn
    Offline / Send Message
    mannyn null
    There used to be a script for Maya that did this but as far as I know, it isnt compatible with more recent editions. Is there a quick way to do this in Maya, without it being toooooo fiddly?
    Check out the script I've posted yesterday. It's on page 5 at the end. :wink:
  • Fansub
    Offline / Send Message
    Fansub sublime tool
    Nice code MANNYN ! I did one for my personal projects a few months ago too :smile:
    Just bind it to a hotkey/shelfMM button or whatever.Select the faces you want to apply the effect on and run the script.

    1. if ( size(`filterExpand -sm 34`) < 1) {
    2.           error "This operation only works when a face component is selected ! \n";
    3. }
    4. string $Faces[] = `filterExpand -sm 34`;
    5. select -cl;
    6. for ($Face in $Faces) {
    7.           select -r $Face;
    8.           string $faceNormal[] = `polyInfo -fn`;
    9.           $array = stringToStringArray($faceNormal[0], " ");
    10.           float $X = $array[2];
    11.           float $Y = $array[3];
    12.           float $Z = $array[4];
    13.           ConvertSelectionToVertices;
    14.           polyNormalPerVertex -xyz $X $Y $Z ;
    15.           ConvertSelectionToFaces;
    16. }
    17. select $Faces;
    18. print ("All vertex normals oriented to their face normal(s) ! \n");

  • HatPoweredSpyCrab
    Offline / Send Message
    HatPoweredSpyCrab polycounter lvl 10
    @mannyn thx for the example. I didn't realize there was that little but important detail about the normals. Thx for this gift, it's a game changer. I am going to add your script to my toolset :smile:

  • mannyn
    Offline / Send Message
    mannyn null
    @mannyn thx for the example. I didn't realize there was that little but important detail about the normals. Thx for this gift, it's a game changer. I am going to add your script to my toolset :smile:

    Thanks. Glad I could help :smiley: 
  • Mr_Flamey
    A while ago I made a script to allow quick editing of custom normals in Blender so that I could use it on the game I was working on that used cell style outlines heavily so we couldn't use split normals. It basically works by using the standard normal calculation but removes any influence from elements that are not selected. This means that you can just select areas of your mesh that you want to have all the influence on the shading (i.e. not the chamfers or support loops) and run the script to get much better smoothing. Because this is a somewhat manual method, you can get very nice normals with it. I recently added face weighted normals calculation support too, since it's a fairly simple algorithm to implement, so you can run FWN on the whole object to create the base smoothing, then go in and clean up anything that doesn't look right with the selection mask based smoothing. FWN also works on local face, edge and vertex selections too, so you needn't apply it to everything.

    More information here: http://blenderartists.org/forum/showthread.php?383532
    (yes, I should probably change the example image to use a less crappy model...)

    Although I haven't really dug too deep yet, I think there are probably a lot of improvements that can be made to these algorithms to get better results more automatically, and I'd also like to try doing a mix of custom normals and split normals (my script doesn't deal with edges marked as sharp yet). Blender could also use a real editor for hand orienting the normals, as my script and others support this to some extent, but I haven't seen anything that's particularly easy to use yet :(
  • Justo
    Offline / Send Message
    Justo polycounter
    I think I'm understanding it, but I find the result will simply not look as good and smooth as a lp with TSNM. Am I missing something? Tried this on a car piece; areas marked with red arrows show horrible smoothing. Is the only solution for this to throw more geo at it?

    Of course, I'm not implying this should always look better than TSNMs, this is just another tool yada yada..


     
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    I had a similar case with a car part. Not sure if adding more geo help by itself. But in such cases, you still have the option to rotate  those normals around manually, until you get the result that you want. Thats what I did, and it worked.
  • 0xffff
    Offline / Send Message
    0xffff polycounter lvl 3
    Justo said:
    I think I'm understanding it, but I find the result will simply not look as good and smooth as a lp with TSNM.
     
    No, but in terms of performance it is literally free. You can't really say that about using a high-res normalmap texture...
  • pasha_sevez
    Offline / Send Message
    pasha_sevez polycounter lvl 13
    0xffff said:
    Justo said:
    I think I'm understanding it, but I find the result will simply not look as good and smooth as a lp with TSNM.
     
    No, but in terms of performance it is literally free. You can't really say that about using a high-res normalmap texture...
    Agree. Bevels baked in normal map look nice for sure until you have to scale your working 2k or 4k down to ingame 256, 512 or 1k. Here the difference gets more obvious. Custom normals ALWAYS retain clean shading (if applicable - i.e. for tech staff), while normal maps loose the crispness and get blurry. The hard edges beneath become more visible as normal map blurs.
  • Justo
    Offline / Send Message
    Justo polycounter
    Nice to know that it's better performance and visual-wise with lower budgets, no wonder this is popular in the mobile scene.

     Obscura said:
    Not sure if adding more geo help by itself. But in such cases, you still have the option to rotate  those normals around manually, until you get the result that you want. Thats what I did, and it worked.
    I can see this method getting very confusing when editing dense meshes manually. That's kinda insane! If you rotate around the normal of one vertex, then the other two or three vertices next to it start looking worse.

    Hopefully as this method grows more popular, people start posting some cheat sheets showcasing cool tricks, much like the wiki has a section of tips & tricks pics in the subdivision modeling page. Stuff like where triangles can solve some issues, where not to put triangles or it'll fuck with how the normals are being averaged, etc...
  • Mant1k0re
    Offline / Send Message
    Mant1k0re polycounter lvl 8
    There's a wiki page on vertex normal already, I think.
  • Justo
    Offline / Send Message
    Justo polycounter
    Indeed: http://wiki.polycount.com/wiki/Vertex_norma The closest it gets to show some practical stuff is with foliage and links to scripts for different apps. Hopefully it'll grow more as this method is more frequently used. 
  • throttlekitty
    Offline / Send Message
    throttlekitty ngon master
    I outlined the flow of edge rings, I think if you extend the horizontal loop into that long vertical edge and adjust it's shape, you'll get much better shading. So basically, moar geo in this case. Hope this helps:


  • Goobatastic
    Offline / Send Message
    Goobatastic polycounter lvl 8
    Deleted post. :) 
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    I'm looking for a max script guru, who could tell me, if its possible to make some of the other features working correctly when using face weighted normals. Like some kind of a fix for getting flipped normals and stuff like that on using mirror or symmetry, or the resetting normals on detach. It really gives us a huge headache at the moment. Custom symmetry and attach-detach ? :D
  • ken0y
    Offline / Send Message
    ken0y polycounter lvl 7
    Hey, thanks for putting together a great post.

    Coming from the traditional high to low bake workflow background, I have a couple of questions.

    Firstly, with FWN you do get the high poly look at the edges of the model but without a normal map you are missing out on a lot of the procedural texturing techniques (quixel, substance) that rely on the normal, curvature and AO maps, edge wear being the first example that comes to mind. Is this just something that you would sacrifice for a more generic tiling material?

    Secondly, how do you avoid seams when using tiling materials on these large scale FWN meshes, would there be some kind of tri-planar projection on the shader in engine?

    Regards,

    K
  • musashidan
    Offline / Send Message
    musashidan high dynamic range
    @ken0y you can still bake a curvature map in SP using the same mesh as the target and using per-vertex instead of per-pixel.
  • Justo
    Offline / Send Message
    Justo polycounter
    @ken0y you can still bake a curvature map in SP using the same mesh as the target and using per-vertex instead of per-pixel.
    Golden words were spoken musashi, thank you. 
  • ken0y
    Offline / Send Message
    ken0y polycounter lvl 7
    Cool, thanks for the info.

    K
  • demigodssw
    Offline / Send Message
    demigodssw greentooth
    Thanks for the script mannyn
  • demigodssw
    Offline / Send Message
    demigodssw greentooth
    i found another one which might help
    (http://www.froyok.fr/blog/2012-12-frotools-3-2-new-enhanced-ui-and-tools#comments)
    it has an option to set the normals of all the vertex u have selected towards u, its for maya

  • r3aktor
    Offline / Send Message
    r3aktor null
    Here is a little script for doing this in Blender:
    https://blenderartists.org/forum/showthread.php?372785-Addon-Weighted-Normals-Calculator

    I noticed that for dynamic lights this approach is superb, but for baked lighting, not so much.
    For example, I have issues with UE4 Lightmass, exactly on those verts/edges where normals are modified.
    No matter how good the lightmap UV is, there are black lines on those edges, and increasing the lightmap resolution, its even worse.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    @r3akrot - you should post a picture from that error, so maybe we could figure out where is the issue coming from. I never experienced something like this.
  • r3aktor
    Offline / Send Message
    r3aktor null
    Sure, here they are:

    The white cube and beam have M_Base_wall material and are exported with custom normals,
    the green beam on the right is smooth mesh with normal map baked from hires mesh.

    This is with dynamic light, and it looks ok:

    and after baking the light:

    detailed lighting:


    If you like to look at, here are the box and the beam as FBX

  • Justo
    Offline / Send Message
    Justo polycounter
    Maybe I'm missing something silly, but when I try using the Edit Normals modifier in Max, I can't seem to  ctrl+click multiple faces to make a selection; dragging the mouse is my only choice (which selects unwanted faces). I go to Select By>Face mode...nothing. Oddly enough, sometimes ctrl+click suddenly works, some other times it does not. Anyone know about this? 
  • r3aktor
    Offline / Send Message
    r3aktor null
    Obscura said:
    @r3akrot - you should post a picture from that error, so maybe we could figure out where is the issue coming from. I never experienced something like this.
    Nomatter what I try, I cant get rid of the lightmap artifacts.
    I noticed that other people have the same problem.
    @Obscura ,
    Can you send sample cube as fbx, and what import settings are using for UE4?
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Sure... I just tried it out for you, and I don't get that error. Maybe its something to do with your lightmap UVs?

    FBX here:
    https://www.dropbox.com/s/nbwoddavbrr0n3k/beveledcube_test.FBX?dl=0



  • jRocket
    Offline / Send Message
    jRocket polycounter lvl 18
    I have been using FWN and making bevels for all of my edges, but sometimes I mess up or want to go back and change something. Making modifications after beveling is a pain. Is there script that can unbevel and edge in Maya? I have seen Seneca's unbevel script for Modo, but I couldn't make much sense of it or how I would translate Modo perl code over to python in Maya.
  • r3aktor
    Offline / Send Message
    r3aktor null
    Obscura said:
    Sure... I just tried it out for you, and I don't get that error. Maybe its something to do with your lightmap UVs?

    FBX here:
    https://www.dropbox.com/s/nbwoddavbrr0n3k/beveledcube_test.FBX?dl=0
    Many thanks!
    I think i found the reason.

    This is your box - OK

    and this was mine

  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    I don't think thats the reason ;)
    Can I see your lightmap uvs?
  • Fuboski
    Offline / Send Message
    Fuboski null
    @Obscura I have read your thread and it's awesome. I would like to ask you a detail. I'm working with Blender 2.77 and following your workflow the results with a beveled cube is perfect.





    Now i'm trying with curve surfaces . . . like a cylinder. Now i watched the images you uploaded but the images were too small and I can't still understand how the normals are directed. With this cylinder below, how the normals should be directed? If it is possible , can you upload a big picture with your normals? Because for now i really can't have a decent result.

  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Please go through the other pages. There were some Blender stuff as well.
  • Fuboski
    Offline / Send Message
    Fuboski null
    Oh . . . thank you anyway
  • mettigel
    Offline / Send Message
    mettigel polycounter lvl 6
    Not shure if this looks right ? :smile: . Trying to wrap my head around this in maya with AM Tools for the Vertex Normals.

    The Workflow is like:

    Selecting Hard Edges (30°)
    Bevel
    Selecting Faces by Angle
    Applying the Vertex Normal Script

  • mettigel
    Offline / Send Message
    mettigel polycounter lvl 6
    So and a friend and i tried it again. We changed the Edgeflow, but hmm... Not very satisfied with the resulst. Maybe we did something wrong ?


  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    What? :D It looks exactly like it should. If you don't like the fat bevels, make it smaller. Other than that it looks perfect.
  • mettigel
    Offline / Send Message
    mettigel polycounter lvl 6
    Obscura said:
    What? :D It looks exactly like it should. If you don't like the fat bevels, make it smaller. Other than that it looks perfect.
    Oh ok :D The Corners are looking strange to me.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    The corners shade weird because of how triangulation works with the normals. 
  • BaekIronFist
    So, has anybody the correct script for 3ds Max for a good weighted normals? I'm using so far the FWN script from scriptspot but isn't good at all :'( !!
  • bac9-flcl
    Offline / Send Message
    bac9-flcl polycounter lvl 10
    As far as I understand, there is no plugin so far capable of producing good FWN results without manual input. Every single one I've looked at can't distinguish chamfer surfaces from main surfaces and therefore distorts the normals of main surfaces. In principle it might be possible to write a plugin that can filter the chamfers out and would perform an equivalent of manual averaging across remaining surfaces (which is the main way to get clean FWN now), but I don't know any that implement that sort of thing. The least time consuming (but still time consuming) method of authoring FWN right now is to use syncview plugin that requires you to manually select faces and press a hotkey every time to snap the normals around the selection.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    It definitely seems like something that a plugin could address, I wonder if you could "bake" vertex normals from a high poly model, and if that would work.
  • AtticusMars
    Offline / Send Message
    AtticusMars greentooth
    ZacD said:
    It definitely seems like something that a plugin could address, I wonder if you could "bake" vertex normals from a high poly model, and if that would work.
    Definitely possible, Blender can do this and as I recall this is the way the normalthief script for 3DS Max works.
  • bac9-flcl
    Offline / Send Message
    bac9-flcl polycounter lvl 10
    I think a plugin that reprojects normals already exists, but it's not a very fast workflow either. Ideally a plugin would go through a hard-edged mesh, group vertices by similar direction, discard groups where vertices are closer than a certain threshold (indicating they are probably part of a chamfer), merge all vertices with the same position, apply averaged normals to all vertices, then would iterate through all the previously collected groups that survived the filtering by distance, applying cached perpendicular normal (or a normal generated from the surface group vertices make up) to every set of vertices like that. Result of an operation like that should be a mesh where normals are completely dominated by big surfaces while chamfers have zero influence on shading and simply bridge between them with smooth normal gradients.
  • shogunato
    Offline / Send Message
    shogunato polycounter lvl 12
    This thread is amazing. Thanks guys.
    Do you have an idea to fix this kind of bad shading when you have a hole on a curved surface ?
    Unfortunately, flipping edge orientation or vertex normal do not really work.

  • ZacD
    Offline / Send Message
    ZacD ngon master
    Face weighting the flat-ish surface and the inner loop should work well. 
  • shogunato
    Offline / Send Message
    shogunato polycounter lvl 12
    arf, still not work. thanks anyway ;)
  • AndrewAlexArt
    Offline / Send Message
    AndrewAlexArt polycounter lvl 7
    shogunato said:
    This thread is amazing. Thanks guys.
    Do you have an idea to fix this kind of bad shading when you have a hole on a curved surface ?
    Unfortunately, flipping edge orientation or vertex normal do not really work.
    You should use custom normals if  won't bake a geometry.
    Here is a very good script for that: http://www.scriptspot.com/3ds-max/scripts/average-normals
    I hope it will help you!


124
Sign In or Register to comment.