Baking normal maps on hard-surface models

Basajaun

Hi,

I know this has been asked a million times but after going over several threads in this and other forums am unabIe to figure it out. I am making a real-time HK416 weapon to be used in a FPS game. I have my high-poly and low-poly models completed and unwrapped and now I want to bake the normal map using Substance Painter.


High-poly model.


Low-poly model.


UVs of the body of the weapon.

As you can se what I want to accomplish is rather simple, something you see done in video games all the time, and yet the results I'm getting are far from ideal as you can see here:





The resulting normal map is full of seams al over the place. The low-poly model has smoothing groups that coincide with the UV shells and the high-poly model has floating geometry which is baked flawlessly. These are the settings I used to export the low-poly model:



And here the settings used to bake the normal map:



I did a test using averaged normals and this is what I got:




This time the issue with the seams was fixed but there were some annoying side effects. I'm not sure what I'm doing wrong but I'm stuck and very frustrated so any help would be highly appreciated.

Thanks in advance.

Gazu

For the Distortion : Bake a NON-Average Map, just uncheck AVERAGE NORMALS in Painter.
And later on cut the details in Photoshop and paste it into your "Average" Normal Map.
So you have to bake two times, yes. One time your "real" normalmap, and one time your non-averaged normal just for the distorted details.

For the picatinny rails: check your hp, lp names. give attention that the meshes are at the same position in 3d space before export. X-Ray them. Check UVs again.

seams: did you set your smoothing groups from UV Shells before export? Does your highpoly model has seams? -> Edit Normals -> Unify normals.

EarthQuake

I think all of your issues are covered here: https://www.marmoset.co/posts/toolbag-baking-tutorial/

pior

A few points to understand : 

• Some baking environments split the projected rays according to the hard edges of the lowpoly model. This can be useful in some cases, but probably not what you need here. Test this out with a very simple model to be 100% certain of what Painter does when baking.

•  "The low-poly model has smoothing groups that coincide with the UV shells"
This is absolutely *not* needed, despite a widespread misconception. The necessary and sufficient condition is : every hard edge needs to also be split in the UVs. There is no reason to introduce hard edges where they are not needed (for instance you may have a UV seam over a smooth surface. Setting this edge to hard would be a bad idea). Of course there are cases where a UV seams <<>> hard edges parity is fine, but again keep in mind that this is *not* the necessary and sufficient condition. The rule is hard edges >> UV seams.

• You probably want to use Toolbag3 for baking as opposed to Painter. It is vastly superior features-wise, and will give you instant visual feedback on everything.

musashidan

pior said:

• You probably want to use Toolbag3 for baking as opposed to Painter. It is vastly superior features-wise, and will give you instant visual feedback on everything.

Is this really the case in such strong terms? I still bake in SP. Haven't tried TB yet for baking. This paragraph just might be the reason to give it a try.

As @pior says people confuse 'smoothing group splits/UV splits' with UV splits on surface angle changes.

OP - what padding are you using? It looks like you only have a pixel or 2?

If you're using SP then, imo, floaters are obsolete. You can just stamp those details in SP.

Basajaun

Thank you guys for your prompt responses

I've been doing some tests with the Edit Normals modifier and I've managed to get rid of most of the problem I had in the picatinny rails although there are still some odd artifacts:





I have no idea where those annoying artifacts are coming from

musashidan said:

If you're using SP then, imo, floaters are obsolete. You can just stamp those details in SP.

Really? I hadn't considered that... I guess I should be able to get the same results by simply painting those extra details. I'll give it a try!

Thanks guys!!!

Basajaun

Hi guys,

Yesterday I spent all day doing tests and I think I got it.

For the weird artifacts in the rails I noticed that while editing the normals helped, it didn't completely fix the issue (see post above) so I decided to used a "brute force" approach and instead, I used a different smoothing group for the faces showing the artifacts (without generating individual UV shells for them). These are the settings I then used for the baking process in Painter:
- Dilation width (padding): 16 pixels (for 4K resolution)
- Max frontal distance: 0.005
- Max rear distance: 0.2
- Average normals: ON

This is the result:




Problem solved!

For the distotion I was getting in some of the floater objects what I've done is, I got rid of the ones that were getting distorted (I will paint them in SP) and left the rest. Hopefully this will get me a clean normal (and AO) map.

Thank you for all your advice and suggestions, I doubt I would have been able to fix the problem without your assistance.

musashidan

Glad you got it sorted and great to see you put the work in by experimenting and working at it yourself. Best way to learn once you're pointed in the right direction. I see a lot of lazy posting on these forums by those looking for an easy fix without putting in the work.