Baking/Shading Artifact - How Can I Solve This? I'm desperate...

Alander787

I have unwrap my lowpoly, everything is alright, no stretching, overlapping etc.
I have my highpoly and lowpoly at the same coordinates.

When I try to bake my textures in Substance Painter or even in Blender, it does this sort of a weird thing when it aside from the highpoly, bakes the low poly shading. When I tried to fix it with turning on Autosmooth in Blender and making it alright on every piece, it didnt work. 
When I bake it with Autosmooth, the whole texture and every detail is totally blurred out and all sharp edges are visible, its just not good.

Can you please help me solve this, I'm desperate, I'm running out of time here!

FrankPolygon

@Alander787

The strong gradation in the baked normal texture suggests an issue related to mesh smoothing, triangulation or noncoplanar geometry in these areas. Additional images of the low poly wire frame, hard edges, UV splits and UV map would help diagnose the issue.

Some general suggestions for resolving these issues:

Triangulate the low poly mesh before exporting and baking. Different applications can use different triangulation methodology which can result in a mismatch between the triangulation order and the baked normal texture.

Use hard edges (backed up with corresponding UV splits) to control the mesh shading and reduce the normal gradation on flat surfaces.

Check that the flat areas on the low poly mesh are actually flat and check for any stray or unmerged geometry.

Here's some links to additional resources that cover baking errors, hard edges and mesh smoothing:

https://polycount.com/discussion/comment/2727831#Comment_2727831

https://polycount.com/discussion/comment/2728365#Comment_2728365

https://polycount.com/discussion/comment/2723853#Comment_2723853

Alander787

FrankPolygon said:

@Alander787

The strong gradation in the baked normal texture suggests an issue related to mesh smoothing, triangulation or noncoplanar geometry in these areas. Additional images of the low poly wire frame, hard edges, UV splits and UV map would help diagnose the issue.

Some general suggestions for resolving these issues:

Triangulate the low poly mesh before exporting and baking. Different applications can use different triangulation methodology which can result in a mismatch between the triangulation order and the baked normal texture.

Use hard edges (backed up with corresponding UV splits) to control the mesh shading and reduce the normal gradation on flat surfaces.

Check that the flat areas on the low poly mesh are actually flat and check for any stray or unmerged geometry.

Here's some links to additional resources that cover baking errors, hard edges and mesh smoothing:

https://polycount.com/discussion/comment/2727831#Comment_2727831

https://polycount.com/discussion/comment/2728365#Comment_2728365

https://polycount.com/discussion/comment/2723853#Comment_2723853

I'm trying to better adjust my smoothing and seams. I hope it will fix the problem, will post about it after it's done.

Alander787

So I've been unwrapping stuff and I came to this part:


Should I go with option 1 or 2? Does it even matter?

FrankPolygon

Placement of the hard edges would determine the location of the UV seams but if there aren't any hard edges in this area then placing the seam along the second option would match the seam placement on the other side of this surface. Another thing to consider is the viewing angle and any texture details that may need to carry through from one side or the other.

Moving this seam one way or the other seems like it's a minor detail (once the main shape is unwrapped and pinned) so maybe it's something you could do each way and test to see which looks best? Blender can generate a UV grid texture which would help compare the distortion of each unwrap. https://docs.blender.org/manual/en/latest/editors/image/image_settings.html#generated

Alander787

I tried to bake the normals but they are just blurry, jagged etc.
I don't know what am I doing wrong.

Should I upload the .blend file?

The normals on parts like i posted here (the last one) are alright, but on other parts, where the sharp edges should be, it's just jagged, blurry and bad.

Alander787

here are some examples of my issue:

Alander787

I marked those edges hard and I turned on the Autosmooth, marked seams as they should be.

FrankPolygon

The seams around the outer edges look like ray misses caused by hard edges without matching UV seams but the pixelated areas look like an issue caused by the bake settings.

What texture size, antialiasing sample and dilation value were used for this bake? If these parameters are still using the defualt settings then try increasing the values and re-bake the mesh to see if that resolves the issues.

Here's some Substance Painter documentation that covers common baking parameters:

https://docs.substance3d.com/bake/common-parameters-186975292.html

https://docs.substance3d.com/sddoc/bakers-102400029.html

If you upload the blend file I can take a look at the models and run a test bake.

Alander787

FrankPolygon said:

The seams around the outer edges look like a ray miss related to hard edges but the pixelated areas look like an issue with the baking settings. What's the texture size, antialiasing and dilation values in Substance Painter? If you upload the blend file I can take a look at the models and run a test bake.

Here's some additional documentation on baking and common parameters in Substance painter:

https://docs.substance3d.com/bake/common-parameters-186975292.html

https://docs.substance3d.com/sddoc/bakers-102400029.html

Thank you, here's the link: https://fromsmash.com/polycountblender

Alander787

I tried both 2K and 4K, AA 2x2 and 4x4, dilation value was default and frontal distance was 0,005

FrankPolygon

The hard edges all appear to have matching UV splits so that doesn't seem to be the issue. Increasing the texture size captured more details but even at 8k there were significant baking artifacts. It appears that the pixel density is quite low in some areas and this means there's a limited ability to capture the relatively sharp edges of the high poly mesh.

Softening the edges on the high poly mesh removed most of the baking artifacts at a 4k resolution. Increasing the antialiasing samples had a minimal effect on the baking results. There are some areas where the low poly mesh appears to fall inside the high poly mesh and this does have an effect on the quality of the baking results.

Overall it seems that these artifacts are a combination of issues related to the texture density and the sharpness of the high poly mesh. Here's a comparison of the sharper (left column) and softer (right column) high poly meshes baked to the same low poly mesh with 2k, 4k and 8k texture sizes.

Here's a closeup of the low poly with 4k normal textures baked from the sharp high poly mesh. The pixelated normal artifacts are still visible along some of the edge segments.

Here's a closeup of the low poly with 4k normal textures baked from the soft high poly mesh. Most of the pixelated normal artifacts are resolved and the edge segments appear to be relatively clean.

Where to go from here really depends on the scope of the project: time deadline, remaining budget, intended use, quality level and technical limitations. If the edges must remain this sharp then you'll have to find some way to balance the texture size and the ability to capture the minimal edge width on the high poly. Softening the edges on the high poly will translate to a better bake at a lower texture resolution.

Wasted space in the UV layout could be minimized by straightening some of the curved UV islands and using one of the advanced UV packing add-ons to generate a more efficient UV pack. Another option is to remove any internal faces that won't be seen. All of this would help increase the pixel density without increasing the texture size.

If the asset's resource footprint is a concern then it might also be worth dissolving some of the excess geometry on the flat areas. This could be done automatically with the limited dissolve operation or manually with the other dissolve operations. It also looked like there were a few spots with unwelded vertices and flipped normals so try running a merge by distance operation and recalculate the normals outwards.

Try increasing the texture size, sampling count and dilation value. If that doesn't resolve the issue then try adjusting the softness of the high poly mesh and see if that improves the bake.

Alander787

So I should soften the edges on my highpoly, remove some faces that wont be visible, straighten some UVs, adjust UV island size and I could also try to bake 8k textures and then downscale them to 4K.
You can also send me the .blend file back, if you made some useful changes. I have a pretty short deadline. The model will be used for AR. Do you think that increasing the poly count on my low poly could also help?

Alander787

I tried a quick test with applying limited dissolve, deleting some faces and adjusting UVs and it looks better, but I also need to soften those edges a bit too.

gnoop

 For such object you don't need Hi poly at all.  Just "rounded corners', "bevel"  or whatever it may be called shader.  Almost evry 3d package or offline renderer has one.  Arnold  does for example.    No cage necessary, no troubles with ray missing .

Alander787

gnoop said:

 For such object you don't need Hi poly at all.  Just "rounded corners', "bevel"  or whatever it may be called shader.  Almost evry 3d package or offline renderer has one.  Arnold  does for example.    No cage necessary, no troubles with ray missing .

Well, that's good tip, but I'm a bit worried about the parts where I need smooth surfaces with shading problems. I could just add more polygons to those parts, that could work too.

gnoop

Usually face weighted normals  could fix it without any extra geometry.  Blender has  a modifier for that for example