lowpoly smoothing groups

tonyd927

Everytime I bake normal maps in xnormal I get extremely ugly results. I made this model to test out smoothing groups and perfect normal baking.

In this image, my cylindrical bases will be divided like so:

The cylindrical part will get a cylindrical map with one smoothing group on one UVW island.
The top of the cylinder will be a unique island with a different smoothing group.

As for the cannon part, I have written the smoothing groups.
My question is, how can I get a nice, clean bake using xnormal without ugly hard edges or weird looking normals?

AdvisableRobin

Can you post an image of the high poly you are baking with?

tonyd927

AdvisableRobin wrote: »

Can you post an image of the high poly you are baking with?

Im going to use DDO to add some panels and stuff.



And here's what it looks like with materials applied:



here's what xnormal baked for my maps:

AdvisableRobin

So a general rule to follow, if you have a unique smoothing group then you need to have a UV split along it. The reason you are seeing seams is because you don't have UV splits along your smoothing groups.

This thread Goes more in-depth then I have time to do right now.

The main problem you are facing is that you don't have enough geometry on your lowpoly. Try adding edge bevels to reduce the number of 90 degree angles between faces, which allows you to use less unique smoothing groups and thus have less UV islands since you don't need to split them. Any circular shapes you have in the lowpoly needs to have more geometry so it more closely matches the highpoly.

tonyd927

AdvisableRobin wrote: »

So a general rule to follow, if you have a unique smoothing group then you need to have a UV split along it. The reason you are seeing seams is because you don't have UV splits along your smoothing groups.

This thread Goes more in-depth then I have time to do right now.

The main problem you are facing is that you don't have enough geometry on your lowpoly. Try adding edge bevels to reduce the number of 90 degree angles between faces, which allows you to use less unique smoothing groups and thus have less UV islands since you don't need to split them. Any circular shapes you have in the lowpoly needs to have more geometry so it more closely matches the highpoly.

The whole "add moar geo" thing is a bit confusing. WHERE exactly am I adding them? and where does that newly created bevel go on the uvw map?

AdvisableRobin

The idea behind adding more geo is to reduce the amount of sharp edges you have. So start with the base hexagon for example, notice how the highpoly has a smooth transition from the top of it to the side of it? Try to represent that in the lowpoly by giving the outer edge a bevel.

You'll have to re UV map the object if you add more geo.

tonyd927

AdvisableRobin wrote: »

The idea behind adding more geo is to reduce the amount of sharp edges you have. So start with the base hexagon for example, notice how the highpoly has a smooth transition from the top of it to the side of it? Try to represent that in the lowpoly by giving the outer edge a bevel.

You'll have to re UV map the object if you add more geo.

Ok so for mapping that, would I still wrap the cylindrical part as a cylinder projection, and the top with the new bevel would just be a quick planar map?
And each island gets one smoothing group?

Quack!

You don't need to add any more geo to get an OK bake out of this.

Your smoothing groups should be defined by your UV islands. There are [SCRIPTS] that will do this for you automatically. If you have a smoothing group break, you have to split that in the UV's otherwise your normals will break, which is exactly what is happening.

Also, judging by your ao bake, it looks like some of your face normals are inverted. Make sure you reset xforms on your final low poly model and flip any errant faces.

Pre bake checklist:

xForms reset

Model is triangulated

Smoothing groups set by UV shells

Overlapped UV shells are offset by 1

airage

Your uv and smoothing group setup looks fine to me. So does your normal bake.

I think the main problem here is that xoliul shader doesn't work properly with max 2013+. I had similar problems some time ago when I tried it with max 2014 I think. http://www.laurenscorijn.com/future-xoliulshader-support.html

Try Marmoset2 or a standard material in max.

Also, what Quack! said.

tonyd927

Ok so I made one smoothing group per UV island using textools.
Rendered it in marmoset and this is the result:

it's slowly getting a little better, still some issues (ive circled them)
But overall I am liking this

Quack!

The muzzle cylinders are not really 'errors'. Those are just cases where you should have quite a bit more geometry to describe that shape if you want them to look a bit more accurate. At least double the amount of sides on those cylinders.

The edges you have highlighted are also not really errors. Those edges correlate with the soft edges in your high poly. So those look correct.

[edit] Oh, I see there is a very faint black edge there. That could be a number of problems.

Make sure you are using .fbx when you export this model. Also make sure when exporting you check the boxes that are for tangents and binormals and smoothing groups as Toolbag2 will read that data and display the best possible normal map.

mospheric

As a protip, the new versions of 3DSMax have a button that auto unwraps based on smoothing groups. that they added that.

I also agree that your view in Max may look weird because you are using Xouliul's shader. You should switch it to Nitrous and see if the issues lessen.

tonyd927

You guys rule! Thanks everyone!