lowpoly shading with normal maps for flat stuff?

[Deleted User]

I thought I knew how to do this, but it appears I don't. The picture explains my question better.

How do I make it not do stuff like in the upper left picture? That chair is what I was trying to do initially, and I made the other thing to try to make a cleaner example, but it ended up with less of what I was referring to. It has some too though. What am I doing wrong?

fantasymaster

Everywhere you have an angle of more than 45 deg and a hard edge, you need to separate the UV-shells. Otherwise you always will have this lowpoly artifacts on your normal map. Only 3ds Max has this problem, it’s just fucked up!

You can bevel everything, or set some smoothing groups to achieve a better shading on your lowpoly first. Than make a little gap between UV-shells, everywhere you have a hard edge. Try to bake again. Make sure that your cage covers the lowpoly.

Hope this helps.
Cheers

Mark Dygert

Are you sure those artifacts are baked in?
The low poly could have those viewport lighting issues by default and slapping the normal map over the top doesn't get rid of them.

Also know that the viewport is horrible at displaying normal maps doesn't really matter what shader, it fudges up regardless. This very long thread gets to the heart of the problem and I think works out some kind of a solution. I think it displays object space normal maps fine and the solution converts back and forth? I haven't read it in a while, my work flow is not dependent on the viewport for normal map previewing.

How does it look if you render it? Does it look that way in game?
Remember your UV seams will be hard edges and Max allows smoothing groups to run right over UV seams. I remember reading that there's a way to force Unreal to resmooth meshes across UV seams but I haven't found a way, or someone could have been talking out their ass...

If its baked into the map I'd probably say forget it and go use xNormal.

[Deleted User]

Okay, I tried those things and here's what I came up with:
Here it is like I was doing it before. (It's not as bad as in the first picture because I found some extra verts and fixed them):

Here's the same thing, but with different smoothing groups. The flat parts look okay, but the edges are messed up:

Here it is with the UV shells split at angles > 45. It looks about the same as the first way:


All the bakes were done in xNormal, and it renders the same way. On a flat plane, they have the same issues too:

EarthQuake

This is the age old problem of using a baker and a renderer that do not have synched up tangents. You see virtually every baker/renderer likes its normals just a littttle differently, and this causes smoothing errors as the data is not matched up. What you're seeing isnt the computer baking errors into your texture, its trying to compensate for smoothing of your lowpoly mesh, thus the gradiations etc. Showing it on a flat plane is pretty worthless for learning anything about it, as you wouldn't expect the mesh's normals/tangents to be the same as a flat plane.

We've got a pretty extensive thread on the issue here: http://boards.polycount.net/showthread.php?t=68173

The reason your edges are messed up in the example with hard edges is that your cake is broken along the edges. In maya and max this is pretty easy to solve(both use an averaged cage by default). In xnormal you have to go into the 3d editor and manually set up a cage, and make sure you weld all of the verts.

The second to last image i'm confused with. Are you baking with hard edges and then setting the edges back to soft again? That isn't really how this whole thing works, you cant edit the mesh normals without regenerating the bakes.

[Deleted User]

Oh ok. I think I get it. I'll read the thread. Thanks.

In the second to last image, the edges were soft (one smoothing group), but the UVs were split at the edges. I was just trying stuff. I don't know what I'm doing.