How to avoid this issue?

ghaztehschmexeh

So I've had this issue before and wondered if there was a way to avoid this issue with my normal maps. The bottom seam happens at UV seams, and the top happens on certain curved areas where quads have bee triangulated. What is causing this and can I avoid it? Baked in xnormal, passed through handplane and rendered in marmoset.



Thanks for any help.

Amsterdam Hilton Hotel

would you mind dropping a little more info - is your triangulation locked (the same when baking and when viewing), are your smoothing groups split at UV islands? even with perfect technique you can get really slight, really small seams sorta like that which generally won't be noticed with textures and at normal view distance, but you wanna make sure you're doing those things to prevent the more overt problems.

ghaztehschmexeh

Amsterdam Hilton Hotel wrote: »

would you mind dropping a little more info - is your triangulation locked (the same when baking and when viewing), are your smoothing groups split at UV islands? even with perfect technique you can get really slight, really small seams sorta like that which generally won't be noticed with textures and at normal view distance, but you wanna make sure you're doing those things to prevent the more overt problems.

I manually triangulate my meshes before I bake normals, because I know that a) if an engine triangulates my mesh differently it will bake the normals b) when I create a cage from a untriangulated mesh often xnormal triangulates the two differently and throws an error.

As for smoothing groups, I use TexTools UV islands to smoothing groups script, and then I combine some smoothing groups into one where it seems appropriate. There should be no smoothing group splits without a UV split on this mesh.

ghaztehschmexeh

There's also this issue...

ZacD

Post your normal map. Marmoset isn't synced with any normal map tangent so you may always get minor errors.

kodde

First issue looks like geometry error? Lamina Faces(double faces)?

Second looks like lacking color space depth for your gradients in your normal map. Try to get less gradients in your normal map if possible.

ghaztehschmexeh

As far as I can see there aren't any double faces. As for the gradients, I assume the best way to have "less gradients" is to stop my normal map having to compensate for the low poly normals as much? Because I don't think that's really possible in this case.

I've experienced both these issues in previous models I've made.

For the first image, the normal map is:



For the second one:

kodde

Here's your second image with super-cranked levels. See the gradients? When it comes to sampling reflections for instance one direction might hold a certain color while the slightest change in direction can hold a whole other color resulting in apparent artifacts. On flat surfaces this can look really weird in those certain cases.

ghaztehschmexeh

Oooh, so is there any way to avoid/reduce this? Or am I just going to have to break up my smoothing groups more?

kodde

It's all about the lowpoly's vs the highpoly's differences in surface directions. They both have to be exactly flat(and matching surface direction) to get a solid color all over the place.

ghaztehschmexeh

Well, on the flat surface (minus the bolts) they should be totally flush, so I don't know why that would cause an issue.

Will this show up much after I've textured the object? Do I need to worry about it much?

kodde

With all other textures and things going on it won't be as apparent. But you might in certain circumstances still be able to see it. When being hit by strong specularity/reflections.

It's fairly common to see in commercial games as well if you go looking for it.

My guess is you have a very minor difference in vertex normal directions somewhere on that surface on the hp and/or lp. If you add DXT(DDS) compression on it as well you will probably get more banding artifacts on the gradient areas.

ghaztehschmexeh

Good to know. One of the issues solved then. I won't worry about it unless it's really apparent when I'm done. Thanks for the enlightening me

So what about the first image? Similar thing that it's a common issue that won't show up much when textured? Or something else?

kodde

No problem.

The first image, are those squares located along faces? How does the wireframe look?

ghaztehschmexeh

Yeah they are:

Eric Chadwick

For the issue at the left red arrow, compare the original baked map vs. the handplane output. Also it can help to put those normal maps on the model as fullbright color maps, so you can see them in-situ.

kodde

The right arrow doesn't read as normal map related errors to me. Only thing I can relate to is Lamina Faces in Maya. Any way to check the vertex normal directions in 3DSMax? Are they conforming with the rest of the surrounding vertex normals?

ghaztehschmexeh

There does seem to be a sudden change in colour, especially on the post handplane map...

ghaztehschmexeh

kodde wrote: »

The right arrow doesn't read as normal map related errors to me. Only thing I can relate to is Lamina Faces in Maya. Any way to check the vertex normal directions in 3DSMax? Are they conforming with the rest of the surrounding vertex normals?

Edit:

Think I found it (this is actually quite useful, because I often found myself wondering when I was adding in geo to support normal details. This will allow me to check my normal directions)

Eric Chadwick

The change in color on the OS normal map looks wrong. The vertex normals (aka smoothing groups) should not affect the OS map. But then again it looks like the OS map is not being shown fullbright (self-illumination should be 100% in Max).

ghaztehschmexeh

Ah see I didn't know that ;p

ghaztehschmexeh

Soooo. any suggestions on this issue or am I stuck with it?

EarthQuake

ghaztehschmexeh wrote: »

There's also this issue...

This is a precision issue, common 8bit-per-channel image formats can only hold so much information. Your baker has two options, pick the closest color value (which gives you stair-stepping artifacts like this) or dither (which adds noise). Unfortunately no normal map bakers that I am aware of give you an option.

One thing you can do, its bake 16 or 32 bit per channel, and then convert that to 8bit per channel in photoshop, this generally does a good job of dithering to get rid of the stairstepping.

Though seriously, what you should do is zoom out, texture your asset, and see how it really looks. Chances are you would never notice this in a real world situation.

As for your other issue, make sure your uvs are all welded, randomly unwelded uvs can cause some issues. Also, try converting to Maya format for Marmoset Toolbag, its not synced, but it will probably give you the best results. Toolbag 2.0 will have a few options for different tangent spaces, like Maya, Max and Xnormal.

ghaztehschmexeh

Awesome. Thank you for that! I thought it probably wouldn't show up (especially because the gloss value here is so high), but I wanted to find out if this was indicative of something I was doing wrong. Either way, I learnt something interesting.

Also I was wondering which format was closest to marmoset, so thanks for that too.

Noors

I've noticed that photoshop adds a slight noise when converting from 16bits to 8bits.
I know noise (dithering) is the way to counter 8bits banding, but somehow it looks better when saving directly a 16bits file to 8bits png, without converting in photoshop (atleast on my lightmaps)

But dds compression will trash out all your efforts anyway...

ghaztehschmexeh

Okay after finishing the rest of the low poly I've encountered something I think must be related to my issue. I baked a simple AO from my low poly and I have this black seam where all of my UV seams are. Any ideas on this? It looks like it could be a padding issue? I had the default setting on in max for this (which is 2 I believe). This wouldn't explain my normal map issue though, as I always have my normal map padding high.

WarrenM

Lack of edge padding?

ghaztehschmexeh

WarrenM wrote: »

Lack of edge padding?

I had the default setting on in max for this (which is 2 I believe). This wouldn't explain my normal map issue though, as I always have my normal map padding high. I'll try rebaking with a higher value though. It just seemed like it might be related to my original issue.

ghaztehschmexeh

Edge padding solved it. (But not my original issue). Also, for some reason whenever I bake a simple AO map it always break the appearance of my normal map. I know the combined AO map thing is a thing people use, so how can I avoid this issue also? Notice the obvious seams and lack of smoothing.

WarrenM

You're plugging the AO into the diffuse channel of your material? I don't see why it would be affecting the normals. It shouldn't, in any way.

ZacD

@WarrenM I think the seams are just in the AO, not the normals, what app are you baking with? Try another baking application?

ghaztehschmexeh

Yeah I am.

There are seams like in the original issue, but they aren't so visible.


Add in the AO map and they are much more obvious and it just seems to break the smooth look.



Here's my simple AO bake.

Eric Chadwick

You're baking the AO from the lowpoly model, right? Apply a Smooth modifier and set the model to a single smoothing group, bake the AO, then remove the modifier (otherwise your normal map won't work properly).

Obscura

Eric Chadwick wrote: »

You're baking the AO from the lowpoly model, right? Apply a Smooth modifier and set the model to a single smoothing group, bake the AO, then remove the modifier (otherwise your normal map won't work properly).

Why not from the highpoly?

Eric Chadwick

You can do both.
http://wiki.polycount.net/AmbientOcclusionMap#EarthQuake.27s_Baking_Method

Obscura

I know I just asked because I think baking the ao from the highpoly gives the best quality if you want to get the same result that you would see on the highpoly mesh. I feel this baking from the lowpoly stuff a little unnecessary.

Eric Chadwick

Yes, but not if using a lowpoly mesh as your final display. Did you read the link? Some good info in there from Earthquake about why mixing the two can be beneficial.

Obscura

Eric Chadwick wrote: »

Yes, but not if using a lowpoly mesh as your final display.

Huh? I dont agree with this. Yes I did but I still feel this unnecessary. Its just my opinion.

WarrenM

The reasoning, from what I understand, is that you're generally baking with an exploded mesh. So baking the AO from the high poly gives you some detail but you don't get all the shading where parts would be intersecting with each other. Baking from the low poly gives you that, so you bake both and mix and match to get the best looking (and most logical) AO map...

Obscura

I would recommend to use material ids because then you dont need to explode and to bake from both meshes to get good result.

Ruz

Huh? I dont agree with this. Yes I did but I still feel this unnecessary. Its just my opinion.

yet here you are asking for advice from seasoned pros:)

Obscura

This is why I said, its just my opinion And I think using material IDs is solving the problems.

WarrenM

Material IDs is functionally the same as exploding. The same issues and limitations are there.

Obscura

When you are baking with material IDs then you get ao with details from the highpoly and with the shading where the pieces are intersecting each other, so you dont need 2 bakes for this. This is what I meant.

.Wiki

With material ids you still get artifacts from intersecting parts. So you need to explode the mesh to get right ambient occlusion renderings so far...

tonyd927

excuse my noobness, but material ID's is referring to the smoothing groups that you assign correct?

Eric Chadwick

.Wiki wrote: »

With material ids you still get artifacts from intersecting parts. So you need to explode the mesh to get right ambient occlusion renderings so far...

How do Material IDs cause intersection artifacts?

tonyd927 wrote: »

excuse my noobness, but material ID's is referring to the smoothing groups that you assign correct?

No, two different things.

Edit: smoothing groups, material IDs

.Wiki

Eric Chadwick wrote: »

How do Material IDs cause intersection artifacts?

You will still get problems with the ambient occlusion calculation on intersecting material IDs. I guess it just works for simple diffuse projections and normalmap projections but not for the ao if you are using scanline or mental ray.

ghaztehschmexeh

The smoothing group thing worked! I had always assumed that AO was based just on the geometry itself and not normals. Thanks for that.

Obscura, material IDs do not solve the problem. To start with, my workflow doesn't bake anything in max because many times in the past max just seemed to render blank maps for me. So I purely use xnormal which doesn't support material IDs.

Secondly, there is great benefit in using a low poly bake, I get the feeling you didn't actually read the wiki (or experience these issues for yourself).

When using floating geometry, you'll often get AO artifacts that you need to paint out, if your mesh is exploded and you have separate layers for high and low frequency AO its easier and quicker to paint out these errors that it is when you have the small shadows along with the large shadows both in the same texture.

Matching the exact silhouette of the low-poly is actually a more desirable result than that of the high-poly. Example: You have a 32 sided cylinder intersecting with some other shape, if you bake the AO directly from the high-res you'll get a perfectly round shape. If you bake it from the low-res you'll get a shape that actually fits your model instead of just hurting the mesh by point out how jagged your low-res really is (via the contrast from the 12 sided guy casting a perfectly round shadow, not matching up correctly etc.).

Also, I'm not really sure how cages and material IDs would come into play with an unexploded bake. Surely you want the AO to be cast between pieces and not just each part onto itself?

ghaztehschmexeh

AO now seems to show the streaks that appear with one smoothing group though. I guess it shouldn't be too obvious when combined and overlayed.


I also have this issue that when two parts are close to each other or intersecting, the AO seems to suddenly stop or become very noisy. Any thoughts on this?

Thanks for all the help by the way guys

EarthQuake

Thoughts? Zoom out and view this asset from the distance/angle it will be viewed in game. Most of your issues with "disappear". =D