Home Technical Talk

A short explanation about custom vertex normals (tutorial)

245

Replies

  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    You are right, not everything is super shiny, and not everything will be seen from close, but this determines the diffuse shading too, so you can see the errors even when the material isn't shiny.I'll give an another try to the syncview one. So basically you say the results are ok, and you wouldn't care about small errors?Even if it can change the distortion of the reflection on the surface? Even if its changing the diffuse shading? With shiny materials, you would get visibly "bending" reflections.With non metals too, it would be just less visible. And the same would happen with the diffuse.
  • Joost
    Offline / Send Message
    Joost polycount sponsor
    I'm just presenting all the workflows I know of. Syncviews srcipt is the same as manually averaging faces but you can do it in 1 go. So if you store your face selection you save a lot of time. Unless you have custom tools I'd personally recommend this workflow over anything else.

    The other scripts are definitely usable though. In a production environment I'd probably be okay with some minor imperfections if it saves me from hand editing the normals. You can always spot correct with syncview's script if anything stands out.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Okay I could get it working. Now this is much better, it gives the result that I would also accept :D Thanks!

    Why you are still using supportloops on the spaceships, if you could use this script with chamfers?
  • Joost
    Offline / Send Message
    Joost polycount sponsor
    If it were up to me we'd use some workflow with FWVN. But the main reason is that modelling and unwrapping with support loops is just much easier than using chamfers.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Ah okay I got it. Thats very true. Even lod making is easier. But its a lot more geo.
  • feracon
    If anyone can lend any insight toward how best to do this in Maya, I'd appreciate it.

    I've found the edit vertex normal tool in maya but there's got to be a better way than manually rotating the angle of the vertex normals.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    We had an old thread here where the author referred to this as "Superspecular". I remember that was the first time I learned about this.

    For Maya I used to use a tool called migNormalTools. I don't think the original author is still hosting this, but you can get it from here:
    http://www.polycount.com/forum/showthread.php?t=93666
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Might have already been mentioned here, but I think the biggest gain if you use this in conjunction with normal maps is that you can reduce/eliminate gradients in the normal map texture on surface where both the lowpoly and highpoly are flat. Thereby getting less artifacts and a more clean result when you compress your normal map texture and use it in a game engine. Potentially no banding on these surfaces at all.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Oh and in newer versions of Maya, 2013(or is it 2014)? and newer has options on how to average the normals on the Shape node under Mesh Control. This gives you some options on what baseline of vertex normal averaging you want. The default setting in these newer versions is a lot better than the previous setting for getting the effect discussed here.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    kodde wrote: »
    Might have already been mentioned here, but I think the biggest gain if you use this in conjunction with normal maps is that you can reduce/eliminate gradients in the normal map texture on surface where both the lowpoly and highpoly are flat. Thereby getting less artifacts and a more clean result when you compress your normal map texture and use it in a game engine. Potentially no banding on these surfaces at all.

    Yes, it was mentioned in the first post I think. Also, if you have less geo limitations, it can work alone, without baking.

    I'll have to collect the different tools , and post them into the first post soon.
  • Eric Chadwick
    If you like, feel free to add this to our wiki. Always good to upload it there, to combat link rot (web images disappear quick if not saved), and keep it all in one spot.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Oh thank you Eric, I'll do that, but then I definitely need to make some adjustments first.
  • Eric Chadwick
    Please feel free to upload unfinished content and refine it as you like, you can't break the wiki. Everything there has a history so it's trivial to revert anything, at any time.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Okay, I'll add it soon!
  • Eric Chadwick
    I added a section here.
    http://wiki.polycount.com/wiki/VertexNormal#Smoother_Shading

    It would be great to add some images from this thread.

    You could also cut that section out and make a new wiki page.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Thank you Eric!
  • a3D
    I tried the script, in both 2014 and 2016, but no window is displayed. What could I possibly be doing wrong?

    I copypasted the whole text into the MEL box of a new shelf button. I tried running it (I tried the command line as well)
    Python returns incorrect syntax
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    a3D> migNormalTools?

    It's a MEL script. Just running the script will only register(correct term?) the function. You run the main function by typing migNormalTools() afterwards. Which in turn adds the "Normal Tools" menu at the top of Maya where you have File, Edit, Modify, etc.

    There's a few ways to run scripts, personally I'd probably use a userSetup.mel file which would load and run the script at startup of Maya. So I'd always have that Normal Tools menu available without any shelf button clicks and what not.

    I'm using Maya 2015 here at home. Seems to run fine. At least the interface appears and no error messages.
  • a3D
    I was considering this:
    A chamfered cube has the same vertex count as a cube with split normals (24 in both cases) but the chamfered one has a better shape and better shading [note: no matter how small, every thing in the world is chamfered. 90° angles just don't exist].
    With the same vertex count, one would think the chamfered one is always better.
    Yet, I heard small long triangles have a big impact on performance.
    How much does this matter?
    Should I chamfer everything forever because "lol vertex count is the same", or be wary of long traingles and never chamfer long stuff like room corners?
  • m4dcow
    Offline / Send Message
    m4dcow interpolator
    a3D wrote: »
    I was considering this:
    A chamfered cube has the same vertex count as a cube with split normals (24 in both cases)

    Not a terribly useful reply, but a chamfered cube still needs UVs and those introduce splits, so depending on how you map it approaches 40 verts.

    As for performance, I have heard the thing about thin triangles, back in the day it was an issue for vertex lighting, nowadays it can be an issue for quad overshading (http://blog.selfshadow.com/2012/11/12/counting-quads/).

    I guess try to avoid them, test throw a few hundred in an engine, see how it reacts, add loops to split the tris and compare.

    For using the chamfers, use them where you need them. No need to go and bevel everything just because you can.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    m4dcow - What you say is true. But I think you can already get this very easily without chamfers, and it will be just worse in the future as we use more geo. I already did a test about this by the way.
    I threw this model 100 times in ue4, and actually I got almost the same speed as with one. Maybe not the best test, because no any texture, and scene complexity, so you still don't exactly know how it would behave in an actual environment. But what I always say is that people can use lods.
    VcqTJdZ.jpg
    This mesh is from 80k triangles, and every single corner has supportloops so this case is worse than a chamfer.
  • Hayden Zammit
    Offline / Send Message
    Hayden Zammit polycounter lvl 12
    Having a bit of trouble wrapping my head around this and was hoping I could get some help. I did these two tests.

    rlb9bnA.png

    With the cube, I didn't have to edit any vertex normals. I just beveled all the edges and it came out looking like the image in the first post of this thread. For the other mesh I just added support loops. It obviously doesn't look as good. What would you have to do to improve that?
  • Joost
    Offline / Send Message
    Joost polycount sponsor
    Maya is set to use FWVN by default now I think. Support loops and FWVN give you different visual results. You're missing a loop or 2 in that image though which is probably what you're seeing.
  • Hayden Zammit
    Offline / Send Message
    Hayden Zammit polycounter lvl 12
    Oh ok. Well, it'd be a lot nicer to be able to just do this instead of high poly bakes. I was under the impression that it wouldn't be worth it due to the extra loops being added.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Newer Maya has the default setting of "Vertex Normal Method: Angle And Area Weighted". This is a whole lot better than the older way they used to have. It does look similar to what you want to have, but it's not good enough imo. The Vertex Normals of the bigger faces on that beveled cube don't all have the same normal vector. They still point a bit outwards and are a bit averaged towards the smaller bevel faces. Not by much, but to some degree.

    This would give you artifacts and gradients in the normal map, etc.

    It would be really interesting to have a customizeable version of this way of doing it. I would love to have some threshold values of when to use this method and when to completely follow a big surfaces face normal.

    Here's probably what your right most cube looks like if you view the vertex normals:

    WLI4dO.jpg
  • WarrenM
    Obscura wrote: »
    m4dcow - What you say is true. But I think you can already get this very easily without chamfers, and it will be just worse in the future as we use more geo. I already did a test about this by the way.
    I threw this model 100 times in ue4, and actually I got almost the same speed as with one. Maybe not the best test, because no any texture, and scene complexity, so you still don't exactly know how it would behave in an actual environment. But what I always say is that people can use lods.
    VcqTJdZ.jpg
    This mesh is from 80k triangles, and every single corner has supportloops so this case is worse than a chamfer.

    This is interesting. I wonder how long it would take before this sort of mesh became acceptable to deliver. I mean, I get what you're saying and I support it - this mesh works fine and renders fast on modern hardware. But if I submitted to my lead, I'd be told to go back and bake it down ... once they stopped laughing.

    But my question is how much of that laughing is based in reality and facts, and how much in "we've always baked stuff down, so we will continue to bake stuff down".
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Interesting question Warren. I think the most of people would say what you quoted.
    I definitely see that the hardwares still isn't completely ready for a full game like this, but we and technology is going so fast. So maybe its already acceptable, but not on this level yet. We can't completely rely just on geo yet. Also a fully static thing from 80k is different than a skeletal thing from 80k .

    After all, I would say people should find the balance, and use these techniques in a bit more optimized way, combined with normal mapped details. We already have some actual examples for this. Also, for simple objects, it doesn't worth a time to mess with baking, when a few extra edges can give the same look, in less time. This is just my opinion though.
  • Joost
    Offline / Send Message
    Joost polycount sponsor
    That specific example seems a bit excessive to me, but I can't think of a reason why you shouldn't ever do it like that. Especially if it's going to be used in narrow corridors anyway, where not much geo will be rendered at any given time. In Elite we have spaceships that use support loops that are well over 200k tris.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Thanks for the info Joost! :thumbup: Any chance for some wireframes in the future?

    Anyways, here is an example, for how it can be used in a lot more optimized way. This door has much more simple shapes, but it shows well, how it can be used combined with normalmaps. If I remember correctly, this mesh made from less than 1000 tris...I made the normalmap with NDo.
    DknmGCm.jpg
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    Obscura wrote: »
    Hey guys!


    4nnbCuT.jpg
    yx4Z1sw.jpg



    Hey, thats Doom 3 is it not? Did you work on D3? Loved that game. Played through like 3 times. Looks like that first main area just after where you get scanned :)

    As far as performance goes. I think normal maps will be around for a long time, but hopefully more automated and intuitive. Resolution, AA, Field of view, framerate, especially the 90fps VR (requirement?) view distances and other geometry will be competing for GPU attention for a long time.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Thane - Yeah it was that place, but it was just a fan art I started making in the last year. Unfortunately I didn't work on the original one. But this was the first environment scene where I started applying this supportloops heavily.
    These images are showing where I used them (reds are hard edges and greens are edges with support loops.
    H2BDnGm.jpg
    CMyDnzA.jpg
  • Joost
    Offline / Send Message
    Joost polycount sponsor
    Obscura wrote: »
    Thanks for the info Joost! :thumbup: Any chance for some wireframes in the future?

    I don't think there's much of a chance of that happening unfortunately :(
    I'll look into it though.

    The door is a great example. Much quicker than baking it and you can just reuse the same trim textures.
  • marks
    Offline / Send Message
    marks greentooth
    WarrenM wrote: »
    This is interesting. I wonder how long it would take before this sort of mesh became acceptable to deliver. I mean, I get what you're saying and I support it - this mesh works fine and renders fast on modern hardware. But if I submitted to my lead, I'd be told to go back and bake it down ... once they stopped laughing.

    But my question is how much of that laughing is based in reality and facts, and how much in "we've always baked stuff down, so we will continue to bake stuff down".

    Probably not as far off as you think, we used this technique for probably 95% of meshes on Alien:Isolation and I know that it's being employed heavily on Star Citizen aswell.

    If I'm honest though, I'm on the fence as to whether I can endorse it as a content creation methodology though, it caused a lot of new and unique problems on Alien in terms of performance.
  • WarrenM
    Right, that's part of what I was asking. I guess it depends on whenever we cross that performance line where we can throw more geo around instead of baking stuff down ... probably not for a few more years yet.
  • marks
    Offline / Send Message
    marks greentooth
    Its less about the number of triangles tbh, and more about the size of the triangles in screenspace right? Because having tons of triangles is totally fine if they're efficient to draw, but once your triangles start getting down to single digits of pixel coverage, you start getting crazy amounts of pixel shader quad overdraw which just takes a wrecking ball to your framerate.

    (info on pixel shader quad overdraw here : http://blog.selfshadow.com/2012/11/12/counting-quads/)

    Like, the biggest thing here is that going heavy on your mesh verts, or going heavy in your texture usage to get detail into a scene have very, very different advantages and disadvantages. It's not a case of "more triangles is better" its more like "more triangles has some unique advantages, but also some unique disadvantages that we haven't really encountered often enough in the past to have developed robust solutions to".
  • WarrenM
    Interesting. Yeah, I guess there will always be a place for that balance ... I suppose we're headed towards, or rather have arrived, at a mixed workflow. Some geo, some normals, combine for profit.
  • Trevelyan .
    Offline / Send Message
    Trevelyan . polycounter lvl 11
    Hey all, great to see this workflow getting the coverage it deserves, nice write ups obscura! I thought I'd add my experience to the mix, as working on Star Citizen this has been integral to our pipeline for a while now. It was pushed through by my good buddy/mentor/3D guruwizard Nathan Dearsley who'd also been the driving force for its use on Alien Isolation as well.

    So on SC when creating the ships we're modelling with custom normals in mind, chamfering edges and using one smoothing group where possible. Though, in some cases we use smoothing groups on very small details where you might not want to model a chamfer that will alias like mad at a distance. At first we were using a script available on script spot (probably linked earlier) in order to make face selections by hand and then adjust the normals at the click of a button, but our tech whizzes have subsequently given us a great set of tools that automates the process in around 90% of cases - life saver!

    On top of our mesh we then use tiling textures, overlayed decals and blend shaders to achieve further detailing.

    Custom normals have their quirks but from a personal stand point, they're worth exploring and adding their use to your arsenal of art weapons as I'm sure they'll start to become much more common place very soon!

    I don't want to highjack the thread with loads of pics, so i'll just post a link to a gallery here of my SC ship (mentioned at the start of this thread) that employed full use of custom normals for anyone interested:

    https://www.artstation.com/artwork/the-gladiator-exterior-shots
  • Joost
    Offline / Send Message
    Joost polycount sponsor
    At first we were using a script available on script spot (probably linked earlier) in order to make face selections by hand and then adjust the normals at the click of a button, but our tech whizzes have subsequently given us a great set of tools that automates the process in around 90% of cases - life saver!

    I presume you mean Syncview's script? I see you posted in that thread as well. Here's the link again if anyone's curious http://www.polycount.com/forum/showthread.php?t=75753&page=2

    Are you able to share any details about the custom tools? Does it work on face angles and surface area or is there some other process involved?
    If anyone else wants it, here's a link to a script that uses face angles
    http://www.polycount.com/forum/showpost.php?p=2230470&postcount=28
    Doesn't give perfect results but you can spot correct with the other script. I'd prefer using Syncview's script combined with a robust way to store face selections. You can quite easily do this with material IDs for example, but then you're stuck with 1 material. Face selections are too buggy in Max.

    Also are you using the quad chamfer plugin at all? If nothing else it's great for quickly blocking shapes out and then refining further. Though in the workflow I developed (which no one uses :shifty:) I'd try to avoid collapsing it for as long as possible. [ame]https://www.youtube.com/watch?v=34giO1tu43M[/ame]
  • marks
    Offline / Send Message
    marks greentooth
    Hey all, great to see this workflow getting the coverage it deserves, nice write ups obscura! I thought I'd add my experience to the mix, as working on Star Citizen this has been integral to our pipeline for a while now. It was pushed through by my good buddy/mentor/3D guruwizard Nathan Dearsley who'd also been the driving force for its use on Alien Isolation as well.
    ...
    Custom normals have their quirks but from a personal stand point, they're worth exploring and adding their use to your arsenal of art weapons as I'm sure they'll start to become much more common place very soon!


    Hey man, say hi to Nath for me :) I have got so much time for that guy! I'm pretty sure I owe him a beer or two still :) And yeah he was the driving force of this workflow on Alien so all credit to him and he is an utter badass and you should all check out his work because nath is the fucking man.

    Yeah the way I would say to look at this, is it is another weapon in the arsenal, another string on the bow, it is not going to change everything forever in the way that normalmaps did. It is very useful, but it will not do everything for you, and you won't always want to use it.

    Like you said, aliasing is one of the downsides of the technique - it's something we had big issues with on Alien (which was mostly due to us being deferred and only able to use fullscreen postprocess AA's, if we were tiled forward or something we likely could have used more traditional AA solution like MSAA or something which would have mitigated that a lot more).
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    marks, Joost, Trevelyan .

    Special thanks to you guys for providing informations about these techniques! Its awesome to get them from the creators of the stuff that inspired this thread. I will write that wiki page soon, based on what we collected here.
  • a3D
    Obscura wrote: »
    I will write that wiki page soon, based on what we collected here.
    I casually found out this:
    http://www.polycount.com/forum/showpost.php?p=2329530&postcount=384
    It might prove useful in specific situations as it may allow to bake directly in 8 bits, avoiding the noise that comes with the conversion process.

    A subsequent exchange with Earthquake went on for a couple of posts, as I was trying to see if it was possible to use custom vertex normal as an alternate method to prevent skewing.
    I didn't have the time to make enough attempts, but the subject is in my opinion worthy of further examination.
  • endeffect
    Offline / Send Message
    endeffect polycounter lvl 8
    I used this technique for the first architectural environment i made with UE4.1, 1 year ago (as i remember). But almost with no manual editing of normals, only used quad chamfer plugin for every bevel with manual shading presets. I was really happy with the result, so my customer too because it was very cheap to make any model without baking process.

    Every bevel you see here is quad chamfer plugin bevel with custom shading. Only downside for me was to spend more time on uvs. Almost zero normal maps here on this scenes.

    CustomNormal.jpg
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    a3D wrote: »
    I casually found out this:
    http://www.polycount.com/forum/showpost.php?p=2329530&postcount=384
    It might prove useful in specific situations as it may allow to bake directly in 8 bits, avoiding the noise that comes with the conversion process.

    A subsequent exchange with Earthquake went on for a couple of posts, as I was trying to see if it was possible to use custom vertex normal as an alternate method to prevent skewing.
    I didn't have the time to make enough attempts, but the subject is in my opinion worthy of further examination.

    Now I did a test about this. Unfortunately fwvns doesn't work as expected with the cage. But!... It looks like supportloops works completely fine, and they should remove the skewing issues.
    jcG5Z7W.jpg
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Obscura wrote: »
    Okay I did an another, complete test, but it gave the result I expected after seeing the cage in the first test. We need a cage that gets pushed based on vertex normal direction, and not based on average direction :D.
    -At least this happens in Max.

    73s693t.jpg
    wUZEeKi.jpg

    From EQ's thread.
  • a3D
    EQ said that if you bake using as a match method "surface normals" instead of "geometry normals" you get exactly that - a projection based on vertex normal direction.
    He also pointed out that you can't have hard edges there, otherwise you'd get gaps (because the cage would open)
    I tried that and yes it solved the skewing, but at the same times it gave me artifacts in other places, and I didn't have hard edges there.
    EQ asked to post screens but I've been busy with work and couldn't keep experimenting.
  • Trevelyan .
    Offline / Send Message
    Trevelyan . polycounter lvl 11
    Joost wrote: »
    I presume you mean Syncview's script?

    Are you able to share any details about the custom tools? Does it work on face angles and surface area or is there some other process involved?

    Also are you using the quad chamfer plugin at all?

    Hey Joost, I wasn't actually referring to that script, there are a bunch of scripts to work this pipeline on scriptspot though. As for our custom tools, unfortunately I'm not in-the-know as to how it works exactly, but I think it does take into consideration both face angles and surface area when analyzing a mesh, we're then presented with three different options, each with varying effects on the normals of the meshes, we just choose the most appropriate and then clean up afterwards manually if necessary. Finally none of us are actually using quad chamfer as far as i'm aware, the custom normal workflow works great for us just using the standard chamfer tools within Max.
    marks wrote: »
    Hey man, say hi to Nath for me :) you should all check out his work because nath is the fucking man.

    Ha, agreed :) ^ ^
    marks wrote: »
    Like you said, aliasing is one of the downsides of the technique - it's something we had big issues with on Alien.

    Yeh as well as our anti aliasing tech, as artists, we generally put a lot of thought into prototyping our chamfer widths and which work best with certain materials before going ahead and chamfering an entire model. So if I know i'm going to use a shiny metal on a particular part of a ship then i'll make an effort to widen my chamfers slightly so that the highlight isn't quite so tight on that particular area, similarly areas that are using a material with less gloss can get away with slightly tighter chamfers. So I guess that might be something for artists trying these workflows to consider :)
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    Is that in Max? Are you running into this problem with the cage:?

    Thane- wrote: »
    Noticed something weird with the cage in Max (or maybe its a feature?). I had made an inset in the faces in the model in the image i posted two posts before this one to capture all the floating geo you see. It worked great the first time i tried to render and i thought i'd never have to revisit that situation again however i noticed that in redoing the cage and rebaking i got some distortion, which once again shook my quite vague understanding of how all this works and in trying to figure it out i noticed that when pushing the cage out in Max:

    1. The first push of the vertices of the newly inset faces follows the normal direction of straight away from the surface following its normal.

    2. The second time you use push, those vertices suddenly start to expand in an outward direction, not following the normal and obviously creating distortion in the final projection....

    EDIT2: I've replicated the behavior on a simple box with insets.


    Really? Is that a feature i don't know about. I didn't see anything about it in the 3DS max doc in the projection mod section.


    EDIT: btw, i initially thought it might be because i didn't set the new faces from the inset operation with a smoothing group, something related to that, but giving that whole back plane its own group didn't help. I also thought it might stem from the fact that i forgot to triangulate those new faces from the inset, but nope.

    Here is a photo to help visualize:
    cage_issue.jpg
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Here is something that I'm working on and testing at the moment. Its the decal method from Star Citizen. I'm trying to take full advantage of them in combination with baked elements and custom vertex normals.

    On the picture, the flat buttons, the screw, the circle bevels,the rectangle bevels, the arrows, and the texts are decals. The boxes use custom normals, and the switch buttons are baked from hp to lp.
    HDuW99x.jpg
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    I'm experimenting with different chamfer widths. Here is a comparison. The left use 1 as chamfer amount, and the right use 2. Amount of 2 looks very rounded from close, but looks MUCH better from far even without anti-aliasing(2 left ones on the bottom picture).
    ibU0sKC.jpg
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    Thane- wrote: »
    Is that in Max? Are you running into this problem with the cage:?
    I guess thats a "no"?
245
Sign In or Register to comment.