Quick question about Smoothing Groups

DustyShinigami

Hi

I have a couple of small things I want to check regarding Smoothing Groups. I'm guessing I'm correct about this, but just want to check.

For something low poly, where no baking/Normal map is needed (only a Diffuse and Roughness map), should a mesh still have Smoothing Groups added before it's taken into Unreal? And related to that - if you are free to use Smoothing Groups to highlight sharper/angled surfaces, rather than the whole thing being faceted, does it matter too much how they're assigned? What I mean is - I have a simple cushion mesh that's just a rectangle. I did cut it up into 3 UV islands - left and right sides and a ring of faces running through the middle. So because of the three separate UVs, and that the mesh has 90 degree angles, does it not matter if a different Smoothing Group is not added? Initially, I just gave the entire mesh a single Smoothing Group, or set it to Soften in Maya, as I figured, means it's a cushion, it should be softer/rounder. Though it does mean it has weird/darker surfaces like this:


Thanks

poopipe

you only need to split normals at UV seams if you're baking a normal map to the mesh. 

Those dark bits are caused by you not managing the surface smoothing, it will look like that wherever you render it. 
 
you manage the surface smoothing by using hard/soft edges and/or adding geometry - if you're not using a baked normal map, you need to put more work into the low poly model smoothing to make it look nice. 

Thanez

If you're adamant about not baking a normal map, here's my thoughts.
Unless you do smoothing group changes (hard edges) like you did on the base of the chair,

You're stuck with either a) supporting loops which would look something like this.

The model at the top has the bottom of the cushion intact, and the bottom model has the bottom of the cushion deleted. 

OR b) custom vertex normals / face weighted normals: https://youtu.be/CYQTnhiQOW4
OR a combination of the two. Neither of them look very good unless you have the geometry needed to carry the smoothing properly. 

DustyShinigami

Thanks for the suggestions, guys. In the end, I did add a slight Chamfer/Bevel so it had supporting geometry. For the main seat cushion, I believe I did delete the bottom face.



In regards to not baking Normal maps, I take it (unless it was a chosen style) some kind of Smoothing would still need to be added? Not just to show where the transitions between hard edges are, but because without any, the mesh would be faceted.

I have come across Face Weighted Normals, but wasn't quite sure what they were or how to achieve them. But that video in Maya has cleared things up a bit and it's definitely something I'll have to try sometime.

Thanez

DustyShinigami said:

In regards to not baking Normal maps, I take it (unless it was a chosen style) some kind of Smoothing would still need to be added? Not just to show where the transitions between hard edges are, but because without any, the mesh would be faceted.

Yes, there should always be smoothing IMO unless you're doing a very specific art style where you want every triangle to be shown. There are also specific cases where style isn't the issue, but you want something like an icosahedron with infinitely sharp edges, and Antialiasing will always be on. No normalmap needed and no smoothing groups would be the right decision.

For anything else, I'd start with everything in 1 smoothing group, and add hard edges, chamfers and edgeloops to fix where the capabilities of the smoothing algorithm looked to be pushed beyond it's limits.
Here's a good example of smoothing group and supporting edges on a silencer made by Twinke Masta back in the day before CS:S modders baked normalmaps. There are only 2 smoothing groups.
Before we start, here's how the model would look without smoothing at all. Ew, gross. get. it. away. from. me.

Wipe that trash from your memory.

First you see the smoothed model as he intended.


Second, the chamfered edges according to how detailed he wanted the transition between cylinder and flat front to be.


Third, he chose to add supporting edgeloops. This makes the round part of the cylinder render like a perfect cylinder, and the flat front render perfectly flat. 

A note on the supporting edgeloops in the previous pic, they're usually not strictly needed nowadays. He used them mainly for perfectionism, and because the engine we were modding didn't support custom vertex normals.
Had the engine supported custom vertex normals, in order to reduce triangle count, he wouldn't have used those supporting edge loops and would have modified these vertex normals instead, pushing the effect of the smoothing algorithm to the polygons of the chamfered area, leaving the cylinder cylindrical and the front faces flat:

Lastly we see the smoothing groups. Group 2 is selected here, the rest is in group 1:

The back of the silencer receives absolutely no love because it's mostly hidden by the rails of the rifle.
And here's a comparison between unedited normals...

...And edited normals so you get a better idea what they do.

And here's an OLD vid on how you can edit them in max at least. There's some learny stuff here as well https://youtu.be/_kONIPr_gBM

DustyShinigami

Thanks for the breakdown and explanation. Would you recommend Custom Vertex Normals for all high > low workflows, even if Normal maps are to be baked? Or are they better used in cases like this where there's no baking?

Thanez

Yes, I would recommend custom vertex normals for baking normals to any engine that supports them. Whenever there is geometry in the transition between two flat faces on the lowpoly, like a chamfer or a curved surface, like that chamferbox to the right at 6 minutes in the vid: ( https://youtu.be/_kONIPr_gBM?t=362 ), you can do the custom vertex normals trick.
If the highpoly and lowpoly are both flat in those areas, then the angular difference between the highpoly and lowpoly at those flat faces will be 0. That means 0 information is written to the normal map on those faces. That in turn means that the normal map stands up way better to fidelity loss when it gets compressed, or when mipmapping occurs. These are two scenarios that will regularly break the illusion of a baked normal map, especially so if the baked normal map is stretched to it's limits. Back in the day we would release mods with UNCOMPRESSED NORMALS and with mipmapping clamped. That is an insane cost to push on the GPU just because the engine doesn't support custom vertex normals. Ask anyone. It's insane.

There is also an argument of there being infinite amounts of angular variations between those vertex normals on the leftmost chamferbox in the video at 6 minutes, and if the normal map has to compensate for that on all flat faces, it needs to store that infinity on what, 500x500 pixels? Yeah you can't store infinity on 500x500. Zoom in and see your flat faces turn to mush. This effect you will see best if you look at flat faces and do test-bakes at 1024², 512², 256², 128², 64² and 32² and find where it breaks due to lacking pixels. For a simple chamferbox, the flat faces will not be flat at 32x32, but if you edit the vertex normals, they will be, and only the edge definition and details baked will be affected by the low resolution.

The last argument I can think of is that there is infinite fidelity between -90° to 0° to 90°, which is the limit of the variation you can bake from a highpoly to a lowpoly. The closer to the surface you look, the more decimals you'll have to add to the difference between the highpoly and lowpoly. The normalmap has to store that compensation. In an 8bit bitmap that leaves you with 256 possible variations in each axis in UV space. 

If I made any mistakes, please correct me, my face is too tired for accuracy
https://youtu.be/5b6qH48bslA

Eric Chadwick

3ds Max now has a Weighted Normals modifier, which automates the alignment of face-weighted normals.