Do you need smoothing groups if you have normal map?
node
Hello, I wonder if I need smoothing groups/hard edges if I will apply normal map on model? Do I need smoothing groups/hard edges if I am baking with cage? Without a cage?
Replies
Any examples? Answer is mostly yes.
is it just a generic tiling normal map? the underlying hard/soft edges could be anything - up to your visual discretion.
is it a baked unique normal map? Can't just throw hard/soft edges around as it'll break the normal map (normal maps basically are baked out based on your low poly's shading/hard and soft edge locations).
it is a model of the tank, uniquely baked from high-poly to low-poly. So based on your info, I can just put one smoothing group on everything
yes you can, but likely will need quite a bit of support geometry to reduce the lowpoly shading, otherwise the compressed 8bit normalmap might break on too steep angles.
there are plenty game productions which do not use hard edges / smoothinggroups, but tom me personally i would prefer using those any day over all the struggles one goes through to make an all smooth hard surface mesh work.
One smoothing group for everything will give you drastically ugly and harsh shading, which is not what the normal map of the tank was originally referencing when being baked out. Neox's suggestion here would work, adding some additional edges/loops will alleviate the bad shading.
I have an option in mood "Weight polygons by area" (in regard of shading), so I suppose this option compensates shading relative to polygon area? And also there is an option "Weight polygon by angle"
That still requires vertices/normal tangents to be 'weighted', ergo more loops/geometry
The process I usually use for props and models, like a tank, is to bake the model with hard edges on the UV splits and look for smoothing issues. I'll then go and selectively add hard edges where I want to fix the issues. I don't use a lot of hard edges. I also don't use a lot of UV splits.
there are several reasons I work this way:
1) polygon edges usually look aliased - so you can see obvious stair stepping on the model and visual noise when you look at the scene. I try to avoid putting hard edges on the outside edge of a model for this exact reason.
2) hard edges increase the vertex count of the model as each split doubles the polygons. For most models I try to get between a 1:1 and a 2:1 triangle to vertex ratio. So there should be twice as many triangles as vertexes. This is very important for the memory footprint of your model.
Thanks. How do you check the number of verts? I mean the 3d software doesn't account verts generated by hard edges. By the way, should you necessarily make a UV split on hard edges? - I've heard an opinion that you should to do so
2. isnt true tho? at least the vertex part isn't as you do the hard edges only on UV seams it will not add more verts
it only would add more verts if you place them where no UV seams are, which would not bake correctly
what if I bake with cage?
I have no idea what you mean. what is your concern using a cage?
you said that there would be a problem with baking if placed hard edges without a UV split. Is this also true if I bake with a cage?
a cage affects the normals used for projection, not the result - cage baking does nothing useful unless you need to control ray distance on a per vertex basis or induce skewing
if you use a single smoothing group on a mesh with sharp angles you are asking the normal map to do work that it simply doesn't have enough precision to cope with.
a normal map only has a range of 180 degrees at full precision. A 90 degree outside corner covers 270 degrees. anything more than about 50/60 degrees is going to suffer somewhat and should be considered a potential candidate for a hard edge.
Using support loops to control the normals is all well and good if you are not lodding your mesh. it is a terrible way to deal with the problem if you are using lods and - in the case of edge bevels - as expensive as just modelling the detail in properly.
just ignore the vertex count stuff for now. unless you're doing something utterly stupid it will have no practical effect on performance.
ok, thank you for your informative answer. So then as I understand if I decide to use smoothing groups I don't need supportive geometry, right?
generally no - but that doesn't mean you'll never need to do it
I have multiple OOM bugs, on multiple titles, that say 2 is absolutely true.
It's not a performance issue it's a memory issue - vertex count is the literal size of a mesh in memory. The goal is to get the most amount of triangles for the least amount of vertexes. Maybe if you only develop for PC it doesn't matter but I've worked on some pretty tight memory budgets.
I have never noticed bad baking from not having a UV split on a hard edge. I use substance designer/painter to bake and triangulate my mesh before baking. What software have you noticed this in?
This is specific to substance painter.
This might be why @Neox thinks you need to split UVs for normals to work.
In the substance painter baking dialog box there's a checkbox "Average Normals" - if you have this checked it will ignore the smoothing information on your lowpoly bake mesh. If you want to use custom hard edges or smoothing groups you need to uncheck this box.
Here's the official documentation that talks about this
https://substance3d.adobe.com/documentation/bake/common-parameters-186975292.html
The average normals tick box is to average the cages normals not those on the mesh. (ie you want to average the cage if you use hard edges as you'll get gaps in projection where the hard edges are otherwise, at the expence of having to deal with skew from the averaging of the raycast)
https://substance3d.adobe.com/tutorials/courses/Substance-3D-Painter-Baking-Parameters
I'm pretty sure that just applies to the rays - Your hard edges/smoothing groups are preserved.
An extra vertex is a very small amount of memory - in the order of a few bytes.
duplicating all of them is bad but if not using hard edges solves your problem - you have a different problem.
Anyway. The best reason for using hard edges is so you can lod without your lods looking shit and being expensive.
And yes. The above posts about the average normals tickybox are correct. It is a poorly named tickybox
Yes - this is what I mean - it uses the wrong normals to project rays when baking your model. It doesn't change the model you see in the viewport.
It's never just an extra vertex - it's thousands of extra vertexes across thousands of assets. It's a bad habit and sloppy modeling.
Like I mentioned before - I've had problems where artists I was working with have used hard edges and UV splits without paying any attention to the vertex count and ended up with meshes that caused out of memory crashes.
If you bake a hard edge without a UV seam you will ultimately have an artifact on that area as a line between two faces is more precise than the amount of pixels you have in your texture. it has nothing to do with the cage. You might get away without an artifact if your UVs are perfectly horizontal or vertical but thats not a given.
so when you are baking highpoly data down to your lowpoly and have a hard edge you always need a UV seam and some space for padding, but if you have a UV seam you do not always need a hard edge, you can tho and they do not add to the vert count, the UV seam already did that for you. This might not apply when using trim sheets or flat normals with painted details etc. but for baking it will cause issues.
I don't have unity here to test if it shows render verts differently than unreal. but the same mesh once with hard edges on UV seams and once without has the same render vertex count inside the engine.
okay here we go. we have a lowpoly with an hard edge and uvs without a UV cut and a highpoly that is all smooth
for the sake of the argzument i rotated the UVs to show the issues between mesh resolution and texel density
here is the baked result in 2048x2048px in marmoset toolbag, the cage is averaged. so the split doesnt happen because of that
the same at 4096 resolution. while a lot more faint, it is still there
or a more realistic usecase at 512 resolution
here is why it happens
the resolution of the mesh is just a lot higher than the resolution of the texture
now lets add a UV seam and some space for padding
the issue is gone at 512px
at 256 px it slowly creeps back in
at 128px it becomes more visible
just because there arent enough pixels to compensate for the lowpoly shading here.
I mean this is just a simple usecase and on a sphere you wouldn't use or need a hard edge because the angles in the lowpoly are so flat that the shading doesnt need that compensation.
and just for the sake of it. same thing in substance painter
Average normals is turned on in the cage settings
and the same with a UV split
this just averages the normals of the cage? meaning it doesnt tear the cage apart on UV seams?
if you bake with a different set of normals than your actual lowpoly will have, you will have a ton of issues, pretty much everywhere where the shading isnt averaged the same way as your baker would do it.
"I have multiple OOM bugs, on multiple titles, that say 2 is absolutely true." i never said it doesnt add to memory, but as far as i can tell in any engine i have access to that shows the render vertcount. it doesnt add extra verts, the vertcount stays the same as long as the hard edges are on UV splits, the extra set of vertex data is already there thanks to the UVcut
if you add them just so without the UV cut, yes it will add new verts.
I've tried to bake a cube. As you see on the left edge there was no uv split so the artifact had appeared, the right edge was splitted so the normal effect nice and smooth. Thank you, guys, I will make uv splits on my current tank model - otherwise I would have a problem
do you think guys this stripe should be as separate smoothing group? (and a uv island respectively
What about this semi-circle arch? Should I:
a)set one smoothing group to it
b)add one more segment to it and set one smoothing group to it
b)set a separate smoothing group to each of 3 polygons
?
before we suggest something for all of these cases. how about you test what works and what doesnt? i mean its not taking forever to do that. one thing i would suggest tho, straighten your UVs. your UVs are needlessly wonky and at odd angles, which result in worse quality bakes usually.
I think to test everything by myself is a good idea, but tomorrow I may go to army and couldn't finish my project in time
Well what it comes down to is a chicken and egg problem. UV splits add vertexes and so do hard edges. I've outlined my process for avoiding making a model that takes up too much memory. But in what order do you do it?
I can see that the sphere example isn't something I would've encountered since I wouldn't be adding a hard edge, without a UV split, to something that didn't already have a line in the pixels of the normalmap. For example adding a hard edge where there is already 90 degree corner on the hipoly - such as the intersection of two boxes.
I don't have one specific order, i place UV cuts where i need them, harden edges that i need hardened. But if i need to harden an edge that has no UV cut, i also make sure it gets one.
I am no big fan of completely automatic solutions that just convert UV seams to hard edges, because that will create hard edges in places where I wouldn't need any. Using a script like that as a base is fine tho.
Your example with the stuck together boxes, might be one of those edge cases where it works, but ultimately the issue remains. As shown on the Box by OP. Pixels are just a lot less precise than Edges, and even on 90° angles with straight UVs you'd have to snap the UVs to the pixelborders for a perfect result?
Big thread on this here: https://polycount.com/discussion/107196/making-sense-of-hard-edges-uvs-normal-maps-and-vertex-counts/p1
I thought this was stickied at some point but saw that it wasn't. Well, it is now!
Also generally everything Neox has posted here is good advice.
thanks i couldnt find that one anymore!
yeah we were getting too many stickies, a lot to scroll thru to get to new threads. so i put them here. Sticky Long-Running Technical Talk Threads