The good ol' baking question

monitorhero

Hey guys,

I am a little lost when it comes to baking even though I read up on all the threads by @EarthQuake and the Toolbag Baking Tutorials. Also I was watching the good ol' Handplane Baker videos. Especially this one which still leaves me confused to this day.
https://www.youtube.com/watch?v=ciXTyOOnBZQ

He shows a simple cube with hard edges and the baked result with UV splits along the hard edges and what happens if you don't split it.
However if I try the same thing in Blender I still see some edges on my cube using Eevee. I also had to use a cage with smoothed normals/one smoothing group to get closer to the result from that video. In the Handplane video it looks pretty perfect for baking down a beveled edge onto a hard cube.
In the Toolbag Baking Tutorials I saw this picture however which shows split UVs and a carved in look. Why does it look so different and smooth in the handplane video on the other hand?
Now onto my own results as explained above:


In this picture you can still see the hard edge shining through a little. Also here is my .blend file to check it out.
https://we.tl/t-8aYOTnzm4k

Am I doing something wrong here? There is also this video where someone bakes down some sculpting details onto a cube in Blender and if you open his .blend file you can't even see any hard edge on the low poly cube no matter how far I zoom in https://www.youtube.com/watch?v=pyOWzpKPDjQ&list=WL&index=57&t=0s
If I replicate his scene 100% but with my own sculpted details I always see a minor seam.

So the big question is. Am I doing something wrong? What am I missing?

Thank you for any help in advance.

FourtyNights

Firstly, there's this rule "All of your sharp edges are always UV seams, but not all of your UV seams need to be sharp edges.". So, whenever you use a sharp edge, which is usually a continuous loop to break the shading, you UV unwrap it to its own UV island with a decent padding between other UV islands. In general, you break the shading with sharp edges if you really find it useful for fixing some nasty gradients on the low poly's shading, so that the baked normal map doesn't need to compensate the low poly shading too much. I usually try to avoid sharp edges, and bevel my low poly models, or use custom face weighted vertex normals instead.

To be honest, I also get some seams for my bakes sometimes, which are barely noticeable, but I usually either mark seams behind or under something if possible to minimize the visibility of them. So, it's not an error per se, it's just how this works.

Here are my thoughts after I checked out your .blend file:

I noticed that you're using the default "sharp shading" instead of sharp edges + Auto Smooth. It's a good practice to "shade smooth" everything on your low poly models, then turn on the Auto Smooth from the "Object Data" tab of the object, and crank the 30° to the max 180°, and then mark your sharp edges manually to break the shading. For fun, you can still try to bake the "Cube_continuous_low" without sharpness, to smooth it completely and see how that works out for it.

In this case, the "split bake" of yours is correct. The visibility of seams is reduced almost to barely noticeable, so it's not a problem. Of course you can see them if you zoom so close, but that's not practical thing to do anyway. With great texturing on top of it, seams are even less noticeable, I assure you.

It also seems like you used averaged normals on your cages for baking, which I haven't done in Blender, I've always used the cage duplicated from the low poly as is. I use Marmoset for baking, and there's this setting "Smooth cage", which determines whether to average the cage normals (enabled) or use the low poly’s vertex normals for the projection direction (disabled). So I haven't tested baking with it turned off though.

monitorhero

FourtyNights said:

I noticed that you're using the default "sharp shading" instead of sharp edges + Auto Smooth. It's a good practice to "shade smooth" everything on your low poly models, then turn on the Auto Smooth from the "Object Data" tab of the object, and crank the 30° to the max 180°, and then mark your sharp edges manually to break the shading. For fun, you can still try to bake the "Cube_continuous_low" without sharpness, to smooth it completely and see how that works out for it.

Usually I just use one something group for my low poly but in the video I saw from Artem Tovbaz he said that's not a good way to go about it.
When smoothing the cube_continuous_low the normal map looks like this:

It looks weird since my high poly is using face weighted vertex normals and now you get this gradient that isnt really there in the high poly. I assume its because the normal map has to compensate for the smooth shading of the low poly. I also now noticed that the cube_continuous_low with hard edges needs to have seams on every hard edge as well to avoid those gaps in the normal map. I knew that rule but I haven't internalized it properly. I have just seen some insanely good baking results like this one from Artem Tovbaz:

Just look how close I get and almost no seam is visible. I can't achieve that with my own bakes.

FourtyNights

Yep, no visible seams, but horrible gradients, that's how it is, and yes those gradients are from the low poly indeed from the compensation, not from the high poly. Also, I just realized that you didn't triangulate your meshes before baking. That's also crucial to get the normal map to work properly.

I've never used face weighted vertex normal on a hight poly. I traditionally make high poly with subdivision surfaces, support loops and so on.

Artem Towbaz doesn't talk about triangulation either, which is weird, since it's very important. And he inflates his cages too much. You only need to inflate them until the cage is JUST ABOUT encompassing the high poly without intersections. Too inflated cage can also cause  projection problems.

monitorhero said:

I also now noticed that the cube_continuous_low with hard edges needs to have seams on every hard edge as well to avoid those gaps in the normal map. I knew that rule but I haven't internalized it properly.

Exactly, sharp edge = UV seam. That's why the seam pops up because those sharp edges weren't separated in the UVs. Simple as that.

Using sharp edges is generally ideal for 90° angles and above between edges. There are still few things I recommend you to practise.

- Always triangulate your low poly before baking

- Try find a solution to >90° angles with beveling you low poly models instead of sharp edges

- Additionally experience with face weighted normals on low poly models after bevels to improve gradients even further

monitorhero

FourtyNights said:

Yep, no visible seams, but horrible gradients, that's how it is, and yes those gradients are from the low poly indeed from the compensation, not from the high poly. Also, I just realized that you didn't triangulate your meshes before baking. That's also crucial to get the normal map to work properly.

I've never used face weighted vertex normal on a hight poly. I traditionally make high poly with subdivision surfaces, support loops and so on.

Artem Towbaz doesn't talk about triangulation either, which is weird, since it's very important. And he inflates his cages too much. You only need to inflate them until the cage is JUST ABOUT encompassing the high poly without intersections. Too inflated cage can also cause  projection problems.

monitorhero said:

I also now noticed that the cube_continuous_low with hard edges needs to have seams on every hard edge as well to avoid those gaps in the normal map. I knew that rule but I haven't internalized it properly.

Exactly, sharp edge = UV seam. That's why the seam pops up because those sharp edges weren't separated in the UVs. Simple as that.

Using sharp edges is generally ideal for 90° angles and above between edges. There are still few things I recommend you to practise.

- Always triangulate your low poly before baking

- Try find a solution to >90° angles with beveling you low poly models instead of sharp edges

- Additionally experience with face weighted normals on low poly models after bevels to improve gradients even further

Thanks. I just wanted to get reassurance that I am not doing something wrong. I always triangulate my meshes if I switch between software and beveling is indeed a good practice like it is explained in EarthQuakes posts. My example was more a mid to low poly thing that's why I used he face weighted normals on the high poly to avoid banding in reflections.

FourtyNights

Hey, I tested bakes in Blender, but with 4096x4096 resolution, and seams are pretty much gone. You baked with 1024x1024. It's good to bake with higher resolution, and also work with those high res bakes when texturing, and only exporting lower res if needed.

EDIT: If you get banding in reflections, that's a dithering issue. It's always good to dither final 16-bit normal maps (or 32-bit, as Blender does) when converting them to 8-bit, which is what game engines understand the best.

monitorhero

Hey FourtyNights, thanks again for the response. Yeah that might have been the reason for the differences between the bakes. And I always use dithering for 8bit maps. With banding I actually meant this effect you see in reflections when not using face weighted vertex normals since the reflections follow the normal orientation: