Of seams and shading artifacts/My brain is splitting at the seams

Ludvigs

Disclaimer: Yes, I've read every one of the great threads on the wiki, on normal maps and its baking and rendering (thank you for those by the way). This is where I get stuck hammering my head into a wall. Thus, I come out from my long lurking to ask for aid.

So, I was building a high res and low res version of a Warhammer Dreadnought, to learn the Zbrush Zmodeler brush (not what this is about). Fairly simple shapes, right? Mostly boxes and cylinders. Indeed, smooth modeling, and easily done UV's. I then think, "Hey, this should bake down pretty well, no?". Indeed, no was the answer. Fair enough, I'd skimped a lot on the polys in the low, so i rebuild it with some more support-loops, do the pretty UV shells again, and send it off to Xnormal. Yet again the gods of normal mapping laugh in my face and send me back to the drawing board.

In my frustration, I try to bake out a simple box, 6 sides, no supports, split the UV's at the smoothing groups. Hey, nay bad. Lets go more advanced. I go and build one of the contiguous box meshes that people seem to use for baking tests. I even put it in a church, hoping that the good lord would look upon my travails with favor (you can all guess how that went):

Yea, so looks good from a distance, right? But look here as we get up close and personal (open the picture in a new tab to really see the seams):

Meh. This is with a regular flatten by edge angles in 3ds Max, smoothing groups at every UV seam, as per recipe. 2048x2048. (Note: I also tried upping the edge-padding to something ridiculous, to no change.) Thus the problem rears its ugly head. Am i doing something wrong? Am i supposed to accept these seams, and go build in some bevels at every edge? Hmm. Alright, I'll have a go at welding some of the shells:


Ok, so we traded away some seams for crappy shading. But hey, that can be fixed with some object space mapping (indeed, before you ask, the seams look the same with an object space map). However, in most models (mine at least), there will be a seam at one edge or likely several, and as the model turns, those will be quite visible. Are these the best results one can expect? Can anyone do a better job than me? 

Update: Tried with beveled edges, exact same results on the seams.

Update 2:
I went and put my normal map into the albedo, lo and behold:

This can't be right? The colors should match across the seam, no? Have i gone insane? Is this just my brain letting off a giant fart? Am I actually a duck having nightmares about rendering normal maps? Time will hopefully tell.

Scruples

The color depends on the orientation of your UV's, if they were the same orientation from that edge they would be the same color. But that single smoothing group version by no means should look that terrible, you either didn't triangulate before baking/exporting or your tangent basis isn't matching with...Marmoset?

Ludvigs

Scruples said:

The color depends on the orientation of your UV's, if they were the same orientation from that edge they would be the same color.

Ah, yes. This is true, I should rephrase. There should be an even gradient going over the seam, as can be seen in this image where there is an even gradient where there are no seams present:

Scruples

No that doesn't make sense, having the color interpolated between the edges on a UV seam+Soft edge is possible but it just makes a bigger seam since the UV edge is infinitely sharp which is why on geometry where the UV edges change orientation drastically (and large shifts in tangent information) it's normally advised to also add a hard edge so there is no interpolation.

Bek

You don't mention the word "cage" once in your OP, and the problem you are having is solved by baking with an averaged cage. Check out the polycount wiki.

Ludvigs

Bek said:

You don't mention the word "cage" once in your OP, and the problem you are having is solved by baking with an averaged cage. Check out the polycount wiki.

Right, sorry. I do indeed have an averaged cage imported from 3ds Max on all the bakes presented here, and i could have sworn i have gotten better results using this method earlier. I have read the entire wiki on normal maps several times over the last years and again in the last week. Trying to figure out what it is I'm missing here  

Scruples said: No that doesn't make sense, having the color interpolated between the edges on a UV seam+Soft edge is possible but it just makes a bigger seam since the UV edge is infinitely sharp which is why on geometry where the UV edges change orientation drastically (and large shifts in tangent information) it's normally advised to also add a hard edge so there is no interpolation.

 How odd. Where the edge is soft and I don't have a seam i get a perfect rendered result, and the color interpolates over the edge. Where i have split the UV's and hardened the edge there is a glaring seam. What you are proposing here is also how my logic has operated thus far. However, my logic seems to have suddenly broken  

It should also be noted that I have set Marmoset TB to MikkSpace, so that is also not the issue.

Bek

Right — now that I look closer at your images I can tell the problem you're having here is known as "zooming in too much". Normal maps are limited in what they can do; at extremely close limits you can see that a sharp edge is in fact a sharp edge. If it's too much of a problem, you add more geo (bevel the edge). This is totally normal.

The only other thing I should mention is that the single UV shell example it should be apparently why the shading is bad and why it's fine with an OS map — though you should post your bevelled example because if you're getting seams on soft edges that's a problem (but, perhaps you meant on the uv seam / hard edges of the bevels). Keep in mind that you're also viewing the model under the worst possible circumstances (untextured, high gloss) which may also be why the seam seems so noticeable.

Ludvigs

Ah, so I have simply hit upon the limitation of what can be accomplished with a hard egde seam then? I tried beveling the edges, but still got a seam, though that needs more testing from my side. Moving the seam will probably alleviate the issue. Good to know it's a thing, and not just me being dense  

I'm not too worried about the shading issue, some support-edges should take care of that. I may just go with the conventional wisdom of hardening edges at 90 degree angles then, for consistency of seams if nothing else. Oh, and I'm not experiencing any seams on soft edges, no.

As you say, it's the worst possible circumstance. I was under the impression that it should be possible to get a "perfect" bake even under these cirumstances, but it appears it was a red herring. Learning to hide the flaws will be the next project then :P 

Thanks for the help!

P.S: One last thought: If i was to paint a gradient over the hard seams, would that actually smooth the seam?

Ludvigs

I have hit upon another hole normal map theories. I rendered a mesh with no hard edges, just UV seams where "needed". As expected, the shading artifacts go crazy in the tangentSpace render. However: I still get a seam where there is a soft edge and a seam in the UV. Here's a Zoomed Too Far(tm) image of said seam. The seam is where the UV is split, but as i understand it, not having a hard edge there should smooth the normals over at the cost of some crazy shading. This I can't produce. 

It can also be seen in objectSpace, though without the shading artifacts:

I realize I'm probably being obtuse here, but I would just like to be entirely sure that there is no way to have a seam free mesh. My OCD requires no less  Thus i can swallow that frog and move on with my work.

Edit: This page in the UE4 documentation actually states this to be the case, though they seem to neatly ignore the shading artifacts you get from heavy gradients.

Bek

Ludvigs said:

P.S: One last thought: If i was to paint a gradient over the hard seams, would that actually smooth the seam?

Normal maps are not something that should be considered human readable nor editable (barring a middleman like nDo), and there are other very good reasons not to hand-edit normal maps (primarily it's a bad pipeline as any rebake means a repaint, and others baking your assets in a team setting won't understand their odd results), but in this specific case painting is bad because the gradients are meant to be there and even if it were possible to easily hand paint smooth edges on objects it wouldn't help here because:

Ludvigs said:

However: I still get a seam where there is a soft edge and a seam in the UV. Here's a Zoomed Too Far(tm) image of said seam.

the seam you're seeing has nothing to do with the normal map. You're literally seeing the hard edge of the cube's geometry. To better visualise this look at the cube side on and look at the corner. To make the cube appear rounded, the edges have a gradient that extends to the edge of the cube — there's not transparency on the object, that hard edge is still there — it's just hard to see because it's tiny in comparison to the big smooth gradient. Better yet, toggle the normal map on/off quickly while looking at the cube's corners.

In real situations this is rarely a problem though as it's either the case that 1) the asset is never seen so up-close for the edge of the geometry to be noticed or 2) Extra geometry reduces the High to Lowpoly mismatch. You'll likely notice the mismatch when baking a cylinder. Depending on how many sides the lowpoly has, the cylinder cap could look terrible in trying to match the "perfectly" round highpoly. Same thing, but the cause isn't as obvious on a cube. Anyway, you're willing to test things which is awesome, I think you might find it useful to put all this theory into practice with a real asset where you consider how the lowpoly geometry, shading, and uv's all come together to make the best possible final product. Cube's might not be the best test case.

Ludvigs

Bek said:

Ludvigs said:

P.S: One last thought: If i was to paint a gradient over the hard seams, would that actually smooth the seam?

Normal maps are not something that should be considered human readable nor editable (barring a middleman like nDo), and there are other very good reasons not to hand-edit normal maps (primarily it's a bad pipeline as any rebake means a repaint, and others baking your assets in a team setting won't understand their odd results),

Indeed, i would be very reluctant to do this barring a last ditch effort to get rid of a nasty seam. More of a thing i was curious about really. 

Bek said:

Ludvigs said:

However: I still get a seam where there is a soft edge and a seam in the UV. Here's a Zoomed Too Far(tm) image of said seam.

the seam you're seeing has nothing to do with the normal map. You're literally seeing the hard edge of the cube's geometry. To better visualise this look at the cube side on and look at the corner. To make the cube appear rounded, the edges have a gradient that extends to the edge of the cube — there's not transparency on the object, that hard edge is still there — it's just hard to see because it's tiny in comparison to the big smooth gradient. Better yet, toggle the normal map on/off quickly while looking at the cube's corners.

Hmm, the only thing which bothers me with this is that I dont get a shading seam where there is no seam in the UV's, they're smooth no matter the angle, unless i look at the silhouetted outline, of course. I was hoping to get this smoothness across all my seams. I would also like someone to buy me a new house for christmas! ...Thats beginning to seem more likely than the elusive "perfect bake"  (Oddly enough I think a perfect bake would make me happier x)

Bek said:

In real situations this is rarely a problem though as it's either the case that 1) the asset is never seen so up-close for the edge of the geometry to be noticed or 2) Extra geometry reduces the High to Lowpoly mismatch. You'll likely notice the mismatch when baking a cylinder. Depending on how many sides the lowpoly has, the cylinder cap could look terrible in trying to match the "perfectly" round highpoly. Same thing, but the cause isn't as obvious on a cube. Anyway, you're willing to test things which is awesome, I think you might find it useful to put all this theory into practice with a real asset where you consider how the lowpoly geometry, shading, and uv's all come together to make the best possible final product. Cube's might not be the best test case.

It was actually a real asset that i was building that set me on this path to begin with. I was getting all these ugly seams and shading artifacts on a deceptively simple mesh. Here seen in the front-plates of a W40K Dreadnought:

I've learned a lot from my dive into the rabbit-hole though, so I'll do a few more tests, and then proceed to run bevels my low-poly. Guess I'll have to stop being a cheapskate with my low-res geo

Polycount really is a treasure-trove of information. Some of the guys in here could make good money of doing webinars I think. I know I'd be signing up ;D

Bek

I'm not sure of the technical reason but the hard edges are slightly more apparent on UV seams (irrespective of the edge being hard or soft), that's just part of the deal I guess. But this is rarely ever a problem as I've mentioned; part of making a good lowpoly is choosing where your UV seams / hard edges go. You'll soon get the hang of it. Also for your real-world test something else that might be going on is aliasing — ideally all the edges of your UV's will be straightened (even sometimes at the cost of extra distortion) to avoid aliasing.

Other than that it might be worthwhile to bevel certain sides in order to reduce the number of UV islands/seams, which also makes texturing easier. There's a lot of factors at play when it comes to getting the best final result so don't stress too much; it might just take a few completed projects to see how it all works together in the end.

Ludvigs

Seeing as it's technically 2 vertices along a UV seam, that might have something to do with it? 
Didn't think of the aliasing issue at all actually. Straightening the edges at the cost of distortion is an interesting concept. And choosing sides to bevel to reduce islands... So much to learn  

I've only been baking characters so far, and that's apparently a lot easier than hardsurface. They tend to be pretty good with a straight up zbrush bake. 

Thanks for all your help and encouragement, I have a good angle on where to proceed now.