So what changed between maya 2012 and maya 2013/14 with regards to normals.

dur23

Nothing is baking properly anymore? Is there some User file being stored somewhere that's causes ones normals to take a fucking leap? Seriously.

losing my shit.

AlecMoody

maya changed the default weighting of normals and added a drop down menu to control normals weighting. There is a handplane video where I show this in action:
[ame="http://www.youtube.com/watch?v=BpPQXvaSft8"]Using handplane with Maya and XNormal - YouTube[/ame]

you can skip to 2:30

dur23

Hey alec. So here's the deal. I have a mesh that i baked the tangent normals in maya 2012. The normal map and the original obj work great in marmo. I then went and posed the obj and re-exported. Made sure to turn unweighted normals on, but i'm getting strange lil internal shading errors. like:

dur23

I'd love to email you the files? I've spent the better part of 5 hours spinning tires.

AlecMoody

those errors are caused by triangles flipping direction. Try reseting the triangulation direction (can be sticky on older meshes, freeze transform, and delete history). Its also possible that when you posed the model you changed the result that the automatic triangulation generates.

To make things more complicated. Those screenshots are from in engine or in maya? What file format are you using to pass the model into marmoset?

MM

why not just triangulate the mesh ?
if it is your final model, there is no reason to keep it in quads.
triangulate your final mesh, use that to bake normals and export it to what ever format your need.
normal map should work perfectly for the most part.

passerby

just triangulate before bake, or epxort as fbx with triangulation on for both xnrormal and the game engine

dur23

Alec: I'm exporting marmoset. The thing is I've tried what I think is every possible combination. I have a previous export (2012 I think) that works perfect. it's one of those thing where I missing one lil thing. Or else I get the strange feeling that obj/fix are storing the settings regardless what version I open.

dur23

Mm and passerby. The thing is I had already baked it ages ago. But for some reason I can't the mesh to export and work with that previous bake.

MM

did you bake on a quad or triangulated mesh ? because it looks like the original bake was done on a quad mesh.

baking on a quad mesh would only work inside maya given that the vertex normal doesnt change at all.

deforming a quad mesh(like rigging/posing it) causes it to change its internal triangle directions.

so triangulating it before baking and using that same triangulated mesh as final mesh guarantees that issues like this doesnt pop up later. you can also lock normals, etc. but i prefer triangulation since locked normals sometimes breaks during imports in different formats.

so if this was in fact baked on a quad mesh that you no longer have access to, then the normal map is pretty much useless in terms of accuracy.

dur23

It most def quads. This used to work with marmoset pretty good though? I also for some reason can't even get a normal mapped cube to work now in marmo. I feel like some crazy underlying shit is going on.

AlecMoody

Does marmoset import or calculate normals? The issue with triangulation here is that maya generates different normals for force triangulated quads. You can try locking normals, and the manually triangulating just the quads that are causing problems.

As MM mentioned. Triangulate before you bake.