Normal Map Issue HELP PZZZZ! (maya 2008/transfer maps)

Andy Kerr - Art

ok I have been having this issue for time now with my normal mapping.

A seam will appear, despite having my uvs merged and softened smoothing groups.

This has been bugging the hell out of me for some time now.

I beleive is mainly to do with the edges on my base mesh.

Iam using transfer maps

- High Poly (left) - Low Poly Bake (center) - Low Poly Base Mesh (right)

notice the seam on the diagonal splits.





I figured out two solutions on my own but they do not have optimal results.

1. Harden the smoothing group on the problem edge ring. Then pull the uvs away from the problem area, to get rid of the seam . This solution wastes pixel space.
2. Add a bevel on the problem edge ring and smooth the smoothing group. This creates additional/unessessary polys.

please please can some one enlighten me with the ultimate solution

thanks!!!!! x1000000000

Eric Chadwick

Try creating hard edges along the edges of your UV shells, and re-bake?

kdm3d

Its been a minute since I delved into Maya, but if I remember right there is an "average normals" command. Or something similar.

On your low poly, select JUST the front faces (not the chamfers). average the normals. Then do that for your top and side as well. Basically what you want is the normals on each flat surface to be perpendicular to the face. Then the normals are forced to only deform on the chamfers.

Then rebake after that. This trick works in Max for LP stuff without normal maps, but I'm sure it would work for you here as well. See if that helps:)

Andy Kerr - Art

Cheers eric. That was solution 1. I probally diddnt explain myself that great.

Just stumbled across a post about the same problem. It was resolved by using object space normals instead of tangent.

I think even you suggested that solution.

I need to read up about the differences between object space and tangent space. Still it begs the question, 'Why use tangent space, when you can get more optimal results by using object space normals on a 'poly crunched' mesh?'

Andy Kerr - Art

cheers kdm3d!

I will look into that too!

Andy Kerr - Art

awwwww shit.

just discovered udk dosnt support object space normals

ug

Eric Chadwick

Andy Kerr - Art wrote: »

I need to read up about the differences between object space and tangent space. Still it begs the question, 'Why use tangent space, when you can get more optimal results by using object space normals on a 'poly crunched' mesh?'

Pros and cons...
http://wiki.polycount.net/Normal_Map#TSVOS

PredatorGSR

I ran into and solved this problem about a month ago, need to re-open my model and see how i fixed it, I'll post later today.

Andy Kerr - Art

@kdm3d - averaging the normals just hardened the edge. After tinkering around within the 'Normals' tab, I quickly discovered that putting in the correct value in the 'Set Vertex Normal' tool, sets the Normals perpedicular to the face without hardening them. I can imagine that was command/tool that you couldnt quite remeber.

ta mate!

@eric - thanks man! Something to digest before I hit the sack tonight.

PredatorGSR

Ok, here is the solution and why it is happening.

Case 1: Hard Normals & Small Map
In this case, the uvs of this piece are taking up a tiny piece of a 2k map. There are some minor seams around the hard edges due to the lack of resolution, but they aren't noticeable from distance.

Case 2: Soft Normals & Small Map
Basically this exhibits the issue you are talking about, and the reason is that the resolution on this piece is so small that there are minor seams just because there aren't enough pixels to accurately represent the shading. The exact same thing happens with the hard edged example, however becuase there is so much shading information from the softened normals, there are a lot more seams and places to go wrong.

Case 3: Soft Normals & Large Map
In this example I blew up the uvs shells so that they took up most of the 2k map. You can see that the normal map is flawless (at least in HQ Viewport), because there is enough resolution to represent every pixel that it needs to.


From these examples you can see it is just an issue of resolution. So there are a couple ways you can approach fixing it.
- You can use hard normals, and it will look good enough with the small map to be passable for ingame use. The minor seams on the edges aren't great but it is acceptable.
- You can increase the resolution of the map, or increase the size of the uv shells within the map so that there enough pixels. This will result in it looking very good no matter if you use soft or hard normals.

Note: In my example, you'd never want to use soft normals like I did, you'd want to bevel the edges like the OP showed, if you were going to use soft normals. Hard edges will give you much more flexibility though and allow you to use a smaller map, while being slightly more expensive in terms of rendering due to the hard edges. The smaller map will trump that though.

kdm3d

Yeah.. thats right, the way Maya works would harden the edge.. Ugh, I hate maya;)

good you found the correct result though. Did setting perp normals work for you in that situation?

Andy Kerr - Art

@Predator - hey thanks for that illustrative chunk of info with pretty pictures. It really helps me understand it a bit better

@kdm3d - yup yup. it seems to work! I think i need to consider the bevel and inside faces too. thanks man!

kdm3d

well... dont make the bevel normal perpendicular, that just puts you back to where you started. but the inside edges yes. Also keep in mind that when you adjust the inside edge normals, you want to keep the normals around the curve so it still blends evenly between them. If you can select the normals indivdualy, and manualy move them into place (or close) you can keep the blend in place. Basically, you want make it so every flat surface has perp normals, and force the normal blending to happen on the smaller bevels.

sama.van

When you want to bake Hipoly model on a low poly model you need for getting normal map, you cannot just take the low poly version for that.


From the low poly version you need to modify the model to make it more friendly with normal business when rendering.

Here is a simple example it will resolve you bug :

Duplicate your low poly model and add a double line of edges like the picture.
Do not move the vertex of this new edge line or effect will be same.
Just cut and normal will be more kind when rendering the normal map.
Do not add bevel, this is just a new line of edge on the flat surface.

MM

make sure to triangulate your low poly mesh before baking. you dont need to use that many bevels to get a good bake.

PredatorGSR

Guys, it is just a lack of resolution that is causing his issue.

EarthQuake

MM: that is highly Dependant on the app, engine, etc. For instance, if you have an engine that is synched up to maya's tangents, and you freeze your normals before running it through your exporter(as triangulating changes the mesh normals) you will get perfect results without the need to triangulate. If you have a more unpredictable workflow, say max -> virtually anything, then yeah it can be good to manually triangulate.

PredatorGSR: Resolution is a problem in a perfect world, where maya is synched up 100% with your engine, but any other situation means you're going to get similar smoothing errors regardless of resolution.

Andy Kerr - Art

@ earthquake - I have been trying to export my mesh over into UDK. The updated normals look fine in maya but when I examine them in UDK, the normals have gone back to their original messed up state.

You mentioned 'freezing the normals before running it through your exporter'

How do you freeze the normals in Maya?

Iam in the habit of freezing the franformations and deleting the history before I export it with ActorX. Would that have any effect on the normals transformations?

thank you:)

EarthQuake

Sorry, i keep saying "freeze" when i mean "lock". Normals -> Lock normals, this should keep the normals of the quaded mesh instead of changing the normals when you triangulate, but it all depends where the triangulation happens(if it does when its loaded into UE, then you're screwed).

Now, thats not to say it will actually fix your issue, as to get perfect results in bakes your engine's tangent space has to sync up to your baker's, which is pretty unlikely that UE3 is synched to maya's tangents, so you still may need to use hard edges to get good results. But that is not a problem, as long as your hard edges are along your uv seams it doesnt mean any worse performance as those verts are already duplicated.

And as long as your cage is averaged, you wont get any seams etc that you wouldn't otherwise get using a completely averaged low mesh.

PredatorGSR

EarthQuake wrote: »

PredatorGSR: Resolution is a problem in a perfect world, where maya is synched up 100% with your engine, but any other situation means you're going to get similar smoothing errors regardless of resolution.

Well, my point is that resolution is what is causing the specific issue that he posted about in this specific instance. It isn't demonstrating smoothing errors, just lack of resolution errors.

I don't disagree with anything you've said, I've read the max thread as well as all the normal map threads too, you're preaching to the choir :P. Now that that seam is solved, syncing up tangent calculations is a whole nother can of worms.

EarthQuake

Yeah the resolution thing is a tricky beast indeed. After i started using maya, it was something that would crop up on complex meshes, you'de get perfect results when you had enough resolution, but these "resolution based" errors where you didnt.

At the end of the day, even tho maya can do near-perfect bakes, you've gotta use hard edges just to get passed the resolution based errors.