Hello, I am very new to Modo but I am trying to make a switch to it from Maya, because last summer I used modo to do some modelling and I really took a liking to the program! So right now I am trying to learn to bake normal maps with rounded edge shading (with various edge sizes using sets) since that would be super usefull!
I followed alot of different tutorials. However I keep getting normal map artifacts that looks like flipped faces/ normals, as if the cage wasn't big enough or as if I had stacked uvs. I usually don't have any problems when baking normal maps in Xnormal, Marmouset or Maya I don't know what is causing it but it has to be something I missed in Modo.
This is what I have done so far to try to fix the problem.
-I have increased the size of the mesh to make baking easier.
-I have runned mesh cleanup on all meshes.
-I have removed the uv map for the high poly since it wont need it.
-I have made a cage and an explode using morph maps.
-I have softed all the edges on the low poly except the uv seem edges they are hardend.
-I tried with a test scene and got fairly good results, so something has to be wrong with the scene I am baking from?
Yet I keep getting artifacts and it doesn't even look like it is baking the rounded edges like before. If anyone know what might be causing it I would really appreciate the help, thank you!
Replies
I take a look at your file. The main problem was that you didn't have the highpoly material and in that material group have different selection sets for rounded shader. Also, there was none hard edges assignment and vertex normal on lowpoly ( maybe lost on import or something ).
And you don't need to apply morph influence on cage as you are using the baking wizard ( it automatically or you assign cage if you have any cage created ), only if you want export cage and bake in different software. Personally, I don't triangulate mesh before and after baking, only if I have some mesh cracking on baking, add extra support ( but I add them on the fly during modeling ). Because it's very hard to make any changes on the cage or model. On exporting mesh, set in the preferences to triangulate the mesh.
You don't need to create a tangent normal in the mesh list as you have a texture that is set to normal map ( so, it's basically the same thing ). Also, set textures that you want to bake for every lowpoly model material. That way you have a clean shader tree to select in backing wizard or add extra textures to bake.
Always delete UVs in high poly if you don't need, it's lower file size and saves project way quicker.
You don't need so high AA on render settings for backing, 8 or max 16 is more than enough and UV edge padding depends on texture size in preferences.
Here is the download link of the new file - https://drive.google.com/open?id=1O60F3jWdWu-HF6gLmvtJhBGhz5r6oyHc
Also, I increase a rounded shader width by double for better looking model in the game, otherwise looks to sharp and lowpoly.
What problems do you have with the baking ?
The best way is to create materials for model ( metal, plastic, chrome, etc. ) and in each material have edge roundness selection setup. That way you can control edge roundness and add extra textures, occlusions etc., that is only for example. metal.
Constants are added ( there are set to 1% opacity, so they don't show in render ) to each selection set in different color. That way you can see in the viewport is set to large, medium, small or noone selection for your rounded shader. As you can see in your highpoly model.
Don't need to apply morph influence on cage if you are bake in Modo. Only on exploding mesh you need to apply morph influence. Not need on the cage, only when you going to export cage to use in different software to bake ( quixel, substance, xnormal etc. ).
Same as for the highpoly materials, create lowpoly materials for each model. In that material, set textures that you want to bake. Just like in updated file. That way you can easily select in backing wizard what to bake.