Everytime I bake normal maps in xnormal I get extremely ugly results. I made this model to test out smoothing groups and perfect normal baking.
In this image, my cylindrical bases will be divided like so:
The cylindrical part will get a cylindrical map with one smoothing group on one UVW island.
The top of the cylinder will be a unique island with a different smoothing group.
As for the cannon part, I have written the smoothing groups.
My question is, how can I get a nice, clean bake using xnormal without ugly hard edges or weird looking normals?
Replies
And here's what it looks like with materials applied:
here's what xnormal baked for my maps:
This thread Goes more in-depth then I have time to do right now.
The main problem you are facing is that you don't have enough geometry on your lowpoly. Try adding edge bevels to reduce the number of 90 degree angles between faces, which allows you to use less unique smoothing groups and thus have less UV islands since you don't need to split them. Any circular shapes you have in the lowpoly needs to have more geometry so it more closely matches the highpoly.
The whole "add moar geo" thing is a bit confusing. WHERE exactly am I adding them? and where does that newly created bevel go on the uvw map?
You'll have to re UV map the object if you add more geo.
Ok so for mapping that, would I still wrap the cylindrical part as a cylinder projection, and the top with the new bevel would just be a quick planar map?
And each island gets one smoothing group?
Your smoothing groups should be defined by your UV islands. There are [SCRIPTS] that will do this for you automatically. If you have a smoothing group break, you have to split that in the UV's otherwise your normals will break, which is exactly what is happening.
Also, judging by your ao bake, it looks like some of your face normals are inverted. Make sure you reset xforms on your final low poly model and flip any errant faces.
Pre bake checklist:
xForms reset
Model is triangulated
Smoothing groups set by UV shells
Overlapped UV shells are offset by 1
I think the main problem here is that xoliul shader doesn't work properly with max 2013+. I had similar problems some time ago when I tried it with max 2014 I think. http://www.laurenscorijn.com/future-xoliulshader-support.html
Try Marmoset2 or a standard material in max.
Also, what Quack! said.
Rendered it in marmoset and this is the result:
it's slowly getting a little better, still some issues (ive circled them)
But overall I am liking this
The edges you have highlighted are also not really errors. Those edges correlate with the soft edges in your high poly. So those look correct.
[edit] Oh, I see there is a very faint black edge there. That could be a number of problems.
Make sure you are using .fbx when you export this model. Also make sure when exporting you check the boxes that are for tangents and binormals and smoothing groups as Toolbag2 will read that data and display the best possible normal map.
I also agree that your view in Max may look weird because you are using Xouliul's shader. You should switch it to Nitrous and see if the issues lessen.