A short explanation about custom vertex normals (tutorial)
grand marshal polycounter
Hello everyone! I already made a few examples about this to a few of you, but I thought it would be more useful to make a tutorial from it, so I won't have to write it down every time when it comes into the topic. Also this technique is becoming more and more popular as the next gen limitations allows us to use more geo, and less normalmap hackery. You can see this in action, and it was heavily used in Star Citize (as I see, their entire workflow built around this technique),Alien Isolation, and in many racing game, even from the previous generation. I would bet the cars in GTA4/5 made similarly! Probably there are many other real examples too, but I know only these.
So...What are vertex normals?... They are used to light/shade the geometry, every vertex has one. How they work?... Its pretty simple, there is a dot product calculation between the light vector, and the vertex normal. If its exactly facing the light direction, the you will get completely lit vertex/pixels. The more direction difference between them will cause more shadowing.
So basically they are determining how the model should shade under light.
If we take an example of a box, which has smoothing groups, and light to it from the top, we will get the top side being lit, and the other sides will be dark, or black, if we have black ambient color, or no any image based lighting, light environment, or such.
It works like this. But the problems are coming, when you don't want the "hard edge" look. People are solving this with baking a normalmap from a round edged object, and do all the hack that needed then. This worked, works, and it will work in the future too, but there are other, probably easier and less time consuming solutions too. The problem is that if you set the smoothing to use only one smoothing group, you it will make the shading drastically worse, if you don't have enough geometry. Here comes what the tutorial will explain, custom vertex normals. An another solution is to simply add some support edgeloops. Maybe I'll explain those too in an another tutorial later.
So lets get started.
Here is a cube, with a single chamfer on the edges. You would think it should shade almost good. It won't, because of the average normal direction between the faces.This is the downside of the default smoothing. It won't represent the flat sides well, it still has very visible gradients.
But from here, the only thing you need to do, is to fix that average direction to be a proper direction, which is exactly perpendicular to the box's sides in this case.
As you can see, after making them being perpendicular to the sides the gradients are completely gone. The model could be used even without a normalmap, it has smooth edges, and there is no any ugly shading.
I won't go into details about how to edit them in this and that app, maybe also in the future, if there will be a request.
Here are a few, bit more complex examples of the proper usage of this. In some of the examples, I'm showing support looping too.
This whole thing is a big game changer in my opinion, I don't always bake now, rather I decide if its needed at all, or if a chamfer with edited normals is enough. I would highly recommend at least trying it out to everyone, who do a lot of normalmapped object, because you can save many hours while still getting good quality at the end. With doing this, you can skip the "making a whole highpoly model" part of a creation process, and bake only the details that are not some simple rounded corners. Its also very useful to reduce gradients when you are baking!It can completely remove them if you are doing it good, so you can use less smoothing/uv splits, and texture easier.
I hope you found this thread useful! Similar explanation about support edgeloops , and advanced usage of similar tricks will probably come soon.
PS: sorry for the poor english
Also, maybe its a bit chaotic atm, I just sat down and wrote it down in one run. I'll try to make it more organized soon.
So...What are vertex normals?... They are used to light/shade the geometry, every vertex has one. How they work?... Its pretty simple, there is a dot product calculation between the light vector, and the vertex normal. If its exactly facing the light direction, the you will get completely lit vertex/pixels. The more direction difference between them will cause more shadowing.
So basically they are determining how the model should shade under light.
If we take an example of a box, which has smoothing groups, and light to it from the top, we will get the top side being lit, and the other sides will be dark, or black, if we have black ambient color, or no any image based lighting, light environment, or such.
It works like this. But the problems are coming, when you don't want the "hard edge" look. People are solving this with baking a normalmap from a round edged object, and do all the hack that needed then. This worked, works, and it will work in the future too, but there are other, probably easier and less time consuming solutions too. The problem is that if you set the smoothing to use only one smoothing group, you it will make the shading drastically worse, if you don't have enough geometry. Here comes what the tutorial will explain, custom vertex normals. An another solution is to simply add some support edgeloops. Maybe I'll explain those too in an another tutorial later.
So lets get started.
Here is a cube, with a single chamfer on the edges. You would think it should shade almost good. It won't, because of the average normal direction between the faces.This is the downside of the default smoothing. It won't represent the flat sides well, it still has very visible gradients.
But from here, the only thing you need to do, is to fix that average direction to be a proper direction, which is exactly perpendicular to the box's sides in this case.
As you can see, after making them being perpendicular to the sides the gradients are completely gone. The model could be used even without a normalmap, it has smooth edges, and there is no any ugly shading.
I won't go into details about how to edit them in this and that app, maybe also in the future, if there will be a request.
Here are a few, bit more complex examples of the proper usage of this. In some of the examples, I'm showing support looping too.
This whole thing is a big game changer in my opinion, I don't always bake now, rather I decide if its needed at all, or if a chamfer with edited normals is enough. I would highly recommend at least trying it out to everyone, who do a lot of normalmapped object, because you can save many hours while still getting good quality at the end. With doing this, you can skip the "making a whole highpoly model" part of a creation process, and bake only the details that are not some simple rounded corners. Its also very useful to reduce gradients when you are baking!It can completely remove them if you are doing it good, so you can use less smoothing/uv splits, and texture easier.
I hope you found this thread useful! Similar explanation about support edgeloops , and advanced usage of similar tricks will probably come soon.
PS: sorry for the poor english
Also, maybe its a bit chaotic atm, I just sat down and wrote it down in one run. I'll try to make it more organized soon.
Replies
I did my set for the september 2014 polycount challenge just with custom normals. No highpolys were hurt during the creation of this set.
http://www.polycount.com/forum/showpost.php?p=2146316&postcount=143
Heres a gif I just made! Awesome. I only know the basics, but are any of those other parameters useful? It'd be nice if some of the pros on polycount posted their tips and tricks but I guess they don't want to give away their secrets to competitors in this industry
The biggest downside is that vertex normal editing is very finicky, in Max at least.
some scripts for Max:
http://www.polycount.com/forum/showpost.php?p=1199836&postcount=41 This one is fully manual. Slower results but much cleaner.
http://www.polycount.com/forum/showpost.php?p=2230470&postcount=28
https://github.com/ByteHazard/wnormals
ALso more here http://wiki.polycount.com/wiki/Vertex_normal
The quad chamfer plugin also supports this and it's very powerful if you're using this kind of workflow.
[ame]
Took me a decade to plead Blender and Modo devs in every possible forum for editable vertex normals.
One weird thing is that it have always been so underestimated by 3d soft developers. I remember one Light Wave guy even asked me why I need edited normals at all.
Wiki - I'm also using this for a long while now, when I don't have exact limits. I use this, and many support loops with my personal projects, and to be honest, now I'm not trying to get it good with smoothing splits... Your results are showing the potentials very will. This month's challenge seem to fit this method, so I'll do it as a bigger experiment. Hopefully I'll find enough free time to finish it. But because of the timesaving with this method, I'll probably be able to finish it in time. Especially if I'll use simple support loops too.
tynew - No, actually I'm testing and using this for a long while now, that thread just made me sure, its an actually working thing in production work, and it has real potentials in the game industry. Well, It can be any complex if you are using enough geo. Just take a look at that Star Citizen Ship. Of course, he is using cheats like decals a lot, but well, it actually works, you can't notice if you are just a player, and not someone who meet such things every day. Here is one "complex" model's example, that I made using supportloops. Not chamfering, but actually these are almost the same, at least in result. The support looping gives more polygons, but it also gives even better shading in the corners. So here is a picture. It was a wip to my client and unfortunately I'm not allowed to show more at the moment. No any highpoly modeling, just decent amount of geo overall, and supported edges. I think this prop would fit nicely to the vr requirements.
As for the parameters and such, I also use this average selected, as a starting point. This won't always work, especially with complex surfaces and shapes, but still it can be a good start. Then I simply move/rotate the normals manually to make them perfect. If the model allows, I do quick selections from ortho view, and manipulate multiple normals at once. Sorry, I won't go into more details this time, as the first post says, maybe later ^^ But what I wrote can completely work alone, however its a bit slow...
Joost - Actually I was interested about your opinion, because you are one guy who can utilize the potentials very well. So Elite use supportloops instead. Nice! As I wrote above, I like supportloops more, because they gives even better shading in the corners. And to be honest, I don't give a f about the extra geo, because there are actual examples for doing this in the game industry, and it looks like its completely ok, it it makes it looking better. And also geo is clearly not a problem nowadays. I think more people should know about these techniques, and the industry should go into this direction. Well, hopefully it will. We see more and more examples nowadays ^^ Also yeah, I've heard that Alien isolation used one of these, But I wasn't clearly sure. I add it into the first post. And thanks for the links!
kanga - Actually I have nothing to say, Joost told you everything that you need to know
gnoop - I completely agree, you can simply delete this sync and smoothing / uv split thing from your head, once you start using edited normals or support loops. Long triangles are just bad boys, but you know, you already use a lot of extra geo when using such a smoothing solution, and it won't make a difference, if you add a connect! Just for making triangles shorter. Supportloops gives even better shading, but I mentioned this at least 2 times in this post already
Well, you would be surprised, but its a thing that people ask why you need edited normals at all, but... Where I live, I got "what the hell is this synced workflow that you are talking about" question so much times from "professional" programmers here. I think I don't have to say more
Edit - Oh I found a few more , better examples in my sketchbook. Some of them use supportloops instead, but its almost the same thing.
-chair, cola machine, and the terminal used supportloops. Only "med-poly" geo, and use of Ndo, no any baking. Again its a very similar thing in functionality.
Unfortunately I couldn't ever finish this scene because of a reason...
This one used chamfers and supportloops too.
And here is my favorite
and, as the most complex, finished, and working example, I added some chamfering and supportloops to the lowpoly to this, and it looks like the normalmap of this works in every engine, because of the mostly reduced gradients. I used only one smoothing group. I tried it in UE, Unity,Toolbag (I know its easy there because of the choose-able tangent space
So is the only downside that you are going to have to use a ton more geo on everything with all the extra bevels and chamfers?
It actually has the same amount of verts as just using a UV split and/or smoothing split. Or potentially less if you account for tri-stripping? I think the main issue performance wise is the long and tiny triangles, though I suppose it's easy enough to LOD those out fairly quickly. And again there's definitely some workflow issues.
I personally don't think support loops are more aesthetically pleasing, you get a lot of black shading which isn't there with FWVN. And there's the silhouette. And I don't think the extra geo is negligible. On the other hand it can be significantly easier to edit the loops compared to chamfers. That's why I prefer using quad chamfer as in the video above with variable edge width by using edge creasing. I did make a little presentation a while back on this subject which I might share if anyone's interested? Another benefit of face weighted vertex normals is that apparently it looks better in VR as well, so it might gain popularity.
Also FWVN alone and FWVN combined with normal maps are 2 separate things. The obvious benefit for FWVN alone is that you don't need a normal map and for example when you have massive structures you can't really get away with just using a 2k normal map. As with anything it does have trade-offs though.
When you use FWVN with normal maps you get bigger UV islands(unless you're using 1 smoothing group), better silhouette and no gradients on big surfaces. Which shouldn't technically be an issue if you're using a synced workflow( a lot of people still don't!) but you do get better results with mipping if you don't have gradients and I think you get cleaner results with programs like DDO(not 100% sure on this one) The downside is that chamfers are a pain in the ass to model and unwrap.
edit: Another image. There's other factors to consider, like UV splits, having a lot of thin and small triangles at a sub pixel level and tri stripping ( which I don't fully understand tbh) but it might help clear some things up.
So what I understand is you are modeling an asset and editing the normals. To help with the normal editing you add support loops or breaks between surfaces to allow you to edit the normals freely, like in the examples you show above. For the high frequency details I guess you could just use ndo, or ddo or substance designer/painter to do the rust on the bolts etc. I suppose those texture apps support smoothing groups. So the advantage is not having to make a high poly and a low poly, just a mid poly and work the groups.
Above there is a pic of a little helicopter. The smooth cylinders on that model blow my mind. I would like to see a wire frame of that. You are also saying that the extra geometry is not a problem for modern game engines? With something as complex as a sci-fi character or say a mech, would this technique be advantageous?
This is a very interesting thread.
Back to the wiki
Edit: I see I posted the same time as Jooost who provides more info.
Joost - Yeah silhouette is an anther question. It also a lot faster to do than FWVN. I think it looks better beceuase you get the "fat" looking rounding, as your example also show it. I would say its becuse of the more geo, and you get better light interpolation (?..I'm not sure in this). Probably a face weighted double chamfer would give the same. I do use one sg when use either a FWVN or supportloops.
Here is a picture that is showing the vertex normals in the different cases. Basically you get "fwvn" by default with supportloops. I know its actually not that, but it looks like.
Anyways, I really don't like the hard edge look. But I keep them wen needed. Luckily its rarely needed. Also, people shouldn't go crazy and bevel every single corner, sometimes it better to simply use an individual element. But this thing is connected to an another topic.
Mant1k0re - Its face weighted vertex normals.
And about the challenge, I will do the exact same thing if I will have enough free time
I just tried doing FWVN and it's pretty awesome. I like how with this method you have a lot of control over which edges have chunkier edges + editing the vertex normals by hand. This seems like a step up from just hand chamfering edges and then doing vertex normals since that kind of seems destructive in comparison. Besides this one thing has got me wondering about the workflow. Say for example there are certain objects in your UVs that have wonky UV shells(all that hard surface organic zbrush stuff) because of being relaxed, that would be impossible to hand paint normals onto the surface. So I suppose hard surface objects that have organic shapes should have a proper high poly instead?
"hand paint normals" - wut?
Might be a bit hard to explain what I mean. I'm thinking that using this workflow would be very hard to do with objects like this:
The UV's would be warped so any kind of micro details like paneling and indents look like they must have an actual high poly model instead of painting the normals in ndo2 because they would be warped UVS.
Alternatively, you can make an "unified" med poly mesh with chamfers or supports on the corners thats visible on the silhouette, or important for shading, and do the "painting". That can also perfectly work. I don't know which would be faster, but the second would result less geometry.
Or... You can do the usual workflow, but add chamfering or supporting at the corners, so you don't have to be synced, don't have to add hard edges, etc, it will bake perfectly.
Also, cheat with the uvs to work better for you! you can make it work with "painting".
I wouldn't use this method for characters the vert count gets pretty high, and this technique doesn't LOD well.
Barely touched the stuff yet, just if anyone's interested
pc. Actually in many cases you even don't need to tessellate, just keep bevels with hard edges on reference object.
This method actually is not a complete solution. You just transfer those problem making normalmap gradients from being evenly distributed to those small bevels only. Inside those bevels it may still be nasty.
So in some places a hard edge could give you better result since with enough UV padding and UV islands/borders in same direction you could get more perfectly beveled looking result without any actual geometry bevels.
@tynew - Sorry, it works like that. You can do quick selections in ortho view, and manipulate multiple normals with hand then. This works mostly when an edge is parallel to an axis. Unfortunately averaging rarely works, probably because of how it does its job "AVERAGING". Lets say you have a corner with a chamfer on it. There will be one face, a chamfer, and one face again. If you would like to average the normals on one face , it won't make the normal exactly perpendicular to the face, because of the neighbour face's direction. So I rarely use it, if I do, I do it only for making them a bit closer to what I want, so I will have to do a bit less moving with hand.
I'd recommend using the first link I posted on the previous page.
A possible workflow would be to assign a different material ID to the main faces to run the script on. Could also use selection sets instead but they seem to break often. Alternatively the other 2 scripts do an automatic averaging for all faces based on face area. Which does a decent enough job in a lot of cases.
And then there's the quad chamfer method which works really well IMO.
There's no clear cut workflow though. You'll have to construct your own workflow if you want to use FWVN. And again I'd like to stress that there's definitely a lot of drawbacks to FWVN but if done right it can produce some very nice results.
http://www.polycount.com/forum/showpost.php?p=1199836&postcount=41 This one is fully manual. Slower results but much cleaner.
http://www.polycount.com/forum/showpost.php?p=2230470&postcount=28
https://github.com/ByteHazard/wnormals
I tried slide normal thief with a low poly cage and it worked pretty well giving me the same top result as my below image. Although now I can't replicate that anymore for some reason.
@Obscura
What method do you use to adjust your normals, just moving the handles by hand?
@Joost
I have max 2012 I tried running the first link script in maxscript editor/dragging into viewport but nothing happens. Could you explain your process for using the quad chamfer method? I tried doing the same thing as your video but It still gives me the flat shading problems?
I don't really know what I'm doing but here is what i tried anyway:
The script works on face selections. So you make a face selection and then press a hotkey to trigger the script. After dragging the script into max it will appear in the Normal Tools category.
One final question is what should the geometry/edge flow be like using this? The overall shading is great (dont mind the really dark parts down the bottom that's the cubemap horizon/grass) but I seem to get some artifacts where edge loops are.
You make all the edge bevels in modifier . Blender allows to make every beveled edge having its own chamfer width by "weight" adjusting in real time. Then make an object copy and turn bevels/chamfers into multi-segment curved ones (one click). Rarely a few extra sudivision/edge loops might be also necessary. Then just transfer normals or export both in Max and use normal theft script there.
I had been always doing the transfer in Max before but right now Blender seems like doing correct fbx export of normals and its data transfer modifier works no worse than theft script. I would say even better since it transfers in real time like Maya.
Are you referring to the sharp vertical lines?
That's just what happens when you have 2 edges close to each other. Not a bug, just a feature?
If yes, are there any workflow adjustments and/or import/export settings that need to be done in a specific fashion?
After making the edit on the normals, the only thing you need is to export the normals too with the model, and on the import, enable import normals. Nothing more needed.
1- model lowpoly
2- bevel edges
3- manually align vertex normals as desired
4- make UVs
5- triangulate and flip edges as needed for nice shading
6- export phase! FBX settings:
-Smoothing Groups -> YES
-Split per-vertex Normals -> ??
-Tangents and Binormals -> ??
7- Open NDO or similar software, create new flat normal map (right?), add detail where needed
8- Import model in your engine. Import settings?