Normals map woes =(((

EarthQuake

Ok, so i have this model set up in max, and it renders out just about perfeclty with max's renderer. Now the problem is getting the same result in anything else seems to be imposible. Its wrong in our engine and its wrong in xnormal etc.... I think it may be problems with the tangents of the mesh but i dont really know enough technically to really have a clue.

Heres a render from max(yay)


And heres a screenshot from the viewport using ben's 3 light shader.


Basicly im really wondering if anyone has any idea what sort of crazy voodoo magic max is pulling when it renders.

poopinmymouth

Have you tried inverting the green channel? the red channel? both? Are any of your uv chunks flipped oppositely of it's surrounding areas on the model? (not talking about mirroring).

Really it looks like you need to invert your green channel.

EarthQuake

Thats not the problem, the green channel thing would have much much worse errors than this. These are much more subtle than that, if it was just that it would be universally fucked with errors everywhere and very easy to spot. The shader has an option to flip the green channel anyway, that was one of the first things i checked and this is definately not the problem

Eric Chadwick

All tris set to Smoothing Group 1?

Also might try the STL Check modifier, check for coincident faces, unwelded etc.

Show the low-res wire?

StrangeFate

Set the smoothing angle of the hipoly pretty low (10-35degrees) and dont change the smoothing angle of the lowpoly after baking it ...mmm all i can think off right now.

doc rob

yay, looks like the same crap I was dealing with on a gun model about a month ago. I wasn't able to figure it out (and none of the suggestions I got here worked).

SHEPEIRO

it looks like probs i get all the time with using mechanical modelling and normals. they really dont like greater than 90 degree angles. try altering the shading groups in the low poly. and re rendering you wont get as many nice smooth transitions and edges :-( but the smoothing should be better in these places.

if you dont want to do this and have spare polys in your budget, try adding chamfers or inserting edge loops where the problem areas are.

ie look at the low poly and if the shading is ugly and extreme, sort these bits out.

Eric Chadwick

In my experience, neighboring faces with different smoothing groups cause the shared vertices to be duplicated in-game.

But, using a single smoothing group and adding a chamfer there instead causes the same duplication (same cost) but gives a lot more control over how the edge is smoothed.

I use smoothing groups only if I'm in a hurry and the quality doesn't really matter. But it's always better with an extra chamfer, and it's the same cost...

SHEPEIRO

ah thats interesting, never been told that by any of the coders ive worked with.

Eric Chadwick

They're holding out on you!

Yeah a vertex on an edge between smoothing groups causes that vert to get at least two normals, so you get a hard shading seam there. As I understand it, each vertex can usually only store a single normal. So it's split into two vertices when the mdoel is converted into the game's runtime geometry format.

SHEPEIRO

cool thing to know. cheers mate. i use hardeges in my low poly stuff all the time, and id never heard any beef about using them.

also if thats the case, what happens with hard edges that fade to soft edges, in max i have to assign three shading groups. does this still just split the mesh along the edge and re-assign 1 smoothgroup?

maybe the coders were worried about giving away their secrets, if only i knew what 2 + 2 is, then the path to coding enlightenment would be mine wmaharhahahah.

sorry for the HI-jack, hope it helps anyway.

Eric Chadwick

Only the vertices that have two (or more) normals are duplicated. Or at least only those should be... up to the export code.

So if you look at Poop's tutorial, only the row of verts along the bottom of the nose get duped. The others, incl. the outsides of the nostrils where the crease ends, get just one normal each.

Edit: notice the 3rd example on the left, Pooper shows that those verts get two normals each. Try it out in Max... if you do some SGs, then add an Edit Normals modifier, you'll see where your normals get duplicated. Pretty cool.

SHEPEIRO

got ya.

man i feel like my primary school education was severly deficient

Eric Chadwick

Oh no, don't feel that way. That's what forums like this are all about. I get schooled all the time around here. If no one here knew more than I, I'd get real bored around here.

SHEPEIRO

i dont feel bad ))))) ive had coffeeeee

EarthQuake

Chamfering and using smoothing groups are just hack ways around the problem, not a solution. As you can tell by the first render max is doing something that displays it properly. Its not the highres thats the low poly model just rendered with scanline and standard normal bump material.

I'm still thinking its a problem with tangents, dont think they get averaged properly. Ill post more info on it here as we try and work it out.

Eric Chadwick

Oops I assumed the first render was the source model.

I don't know if this helps, but this old comment from Chris Subagio might provide a clue why the viewport shader is different than the scanline one:

[ QUOTE ]
The vertex position, normal, color and UVs arrays in Max don't correspond 1 to 1. Each is stored as a flat array of vertices indexed by a face array, i.e. an array number_of_faces long of 3 indices. To get coherent flat data out of it, you need to get at the vertex data through the face data.

So in the case of positions, you need to loop through the faces. Ask each face for for the IDs for verts 1, 2 and 3 in the vertex positions array, then use that to look up the actual position data. You then ask the same face for it's 3 normals indices, looking those up in the normal array. Rinse and repeat for any other vertex info you like.

[/ QUOTE ]

CrazyButcher

the only way to fix it, is knowing exactly what tangents... max uses when it generates the normalmap, as that is rendered with the scanline pipeline it makes sense that it also works when applied in the offline renderer... I can confirm eric's quote, max thinks in "polys", while realtime renderers only have vertex streams basically.

maybe someone can ask autodesk for giving out more details (or maybe those already exist ??), on what data they use in the baker. Would allow people to code proper export data.

poopinmymouth

We had this same problem when going from max to our engine. I don't think I can say much about how we fixed it, other than we figured out exactly what max does with it's normals, and then emulated that in our shader. Once that was done, *poof* instant yum.

SHEPEIRO

so is there a way to solve this in max or do i have to start using xnormal propperly, havnt so far cos it doesnt really suit my work flow

EarthQuake

Xnormal seems to display the exact same errors as ben's shader in max.

Eric Chadwick

[ QUOTE ]
Xnormal seems to display the exact same errors as ben's shader in max.

[/ QUOTE ] Was the normal map created in Max or in Xnormal?

EarthQuake

Max, but i created one in xnormal as well with the same problems.

SHEPEIRO

it seams stoopid that maxes normals arnt much use in anything but its renderer :Z

MoP

[ QUOTE ]
it seams stoopid that maxes normals arnt much use in anything but its renderer :Z

[/ QUOTE ]

Yes, but that's not Max's fault, really