Baking normal map problems

deemccabe · Oct 2016

Hi there, im having a problem with my normal bake from my high to low poly model. Every time I bake my normal map and apply it to my low poly model I get this really blurry and flat result. Im using 3ds max and im using the projection cage to achieve my results. The images attached are the results im experiencing and if you need any more info or pictures please let me know. It really has me stumped. Any help would be much appreciated. Thanks

David

Michael_Ingrassia · Oct 2016

check you face normals. they may be reversed. looks like poo. The part you are expecting to show detail is really small on the map. Can you split up your UVs maybe to get the plate to be much larger?

kasigawa · Oct 2016

Just a quick suggestion, you could try baking with Substance Designer or with xNormal. I always have the best results with Substance.

Quack! · Oct 2016

This is not an app problem. So keep baking in 3dsmax if you want to.
Like Michael said your normal is inverted and your texture map is way to low resolution.
So flip the green channel of the normal map and give a higher resolution texture for the normal map to fix the issues.

deemccabe · Oct 2016

sorry for the late reply folks and thanks very much for the helpful information. ive got a few more questions regarding the problem. my aim with this project is to high to low bake the tank im modelling for use in games. so the issue with the resolution is that if i have to say for example render that one little plate at 1024x1024 just to get a descent normal in that small area then it cant be very practicle to use in games because you would need alot of maps. i dont want to use anymore than 2 diffuse, spec, normal ect in the whole project, so im wondering how is it done for games?? any help would be much appreciated and if anymore info is needed please dont hesitate to ask. Thanks

huffer · Oct 2016

Looks like you didn't set up the material properly - use a Standard 3dsmax material, plug a Normal map node in your Bump slot, increase the value to 100%, then plug your baked map in your Normal Map node. Then remember to enable Show materials with textures and with hardware shading (or realistic, not sure how it's called now). If the map still looks wrong, load it with gamma 1.0 (if it was at 2.2) and maybe flip green.

musashidan · Oct 2016

That's always going to be the issue with large assets....large unwraps and limited pixels for smaller parts. Maximising your UV space is critical. Stack as many symmetrical/identical islands s you can to save even more space.

As you've mentioned you're going to use 2 sheets so that's a good start. Also, if you channel pack your textures as much as can you could even use 3 sheets, as you'll save draw calls by packing.

There are other methods such as FWVN and tiling textures/decals. (This method is quite common for large assets like spaceships) Or texture atlasing to reuse sections of a texture on parts of the tank like the panels.

deemccabe · Oct 2016

ok so ive flipped the normal's fine and rendeed it at 2048x2048 just for testing. this is the result. its definitely an improvement.

Image: https://us.v-cdn.net/5021068/uploads/editor/ln/cv5udree0ief.png

i still dont feel like the bump is doing it justice though. how do i make it more prominent because it is still very flat. also like i said in my last post im going to use this technique for quite a few of the smaller pieces on the model and im confused as to how to manage the space and size of of the maps if it took a huge texture map just to achieve this. also if you could elaberate on the term you used "splitting the uvs" it would be much appreciated. Thanks

AlecMoody · Oct 2016

I haven't attempted to view normals inside max since they broke support for all the viewport shaders (I think that was 2012). I would suggest loading up a game engine to view your normals. Marmoset, unreal, unity.. Ideally whatever the final engine the model is going to go into.

deemccabe · Nov 2016

ok so ive viewed it in unreal engine 4 and i moved a spot light around it to get a better feel of what it looks like.i then in Photoshop set a few overlays to intensify the map and i seem to be getting a better result . has anyone any ideas of how to make the map better? ive looked at some of the google images for peoples high to low bakes and they look amazing. Thanks

musashidan · Nov 2016

For what it is it has baked fine and looks exactly as it should. It's a very small element and takes up relatively few pixels. I'm not sure what else you expect?

Normal maps are a complete illusion so, when viewed straight on - the way the bake projection sees it - they are convincing, but the more perpendicular your viewing angle the more that illusion breaks. Also, an AO map works in conjunction with the NM in a PBR workflow.

Why not get some texturing done and then see how you fee? I honestly wouldn't get too hung up on it.

deemccabe · Nov 2016

ok, thanks for the help. yea im gong to apply the AO map now and start to texture it and then i can see how it looks. i just wanted to make sure i wasnt missing anything important in the process but i think im got the basics of it down. thanks again for the help and ill post a pic of the final result here so you can have a look. Cheers!

deemccabe · Nov 2016

ok ive noticed another problem which is visible in the last image i uploaded. if you look at how the light is falling onto the plate and bolts you'll see that the light is coming from the bottom up rather than top down. sorry if that doesnt make much sense. i only noticed it when i was playing with the lighting in ue4 for testing. any ideas or explanations??

deemccabe · Nov 2016

sorry i got it fixed. i had something flipped the wrong way. Thanks

musashidan · Nov 2016

Yes, UE4 uses directx, Max uses Opengl so, a Y swizzle flip is in order.

deemccabe · Nov 2016

ok so ive hit another issue with the baking. the edges are creating this black line around the hard edges. i understand why this happens because of the normal ray projecting perpendicular to the 90 degree face. i tried it with different smoothing groups, chamfered edges to add a 45 degree angle ray and i dont want to break up the UV edges because it will be a nightmare to texture. ive watched a lot of videos about this and cant work it out. the images attached is in UE4 and im just wondering has anyone got any suggestions of how to fix it. Many Thanks.

Image: https://us.v-cdn.net/5021068/uploads/editor/tc/n81yh41vesox.png

musashidan · Nov 2016

Did you export/import the tangents/bi-normals in the .fbx from Max. Because the tangent basis of UE4/Max is different you need UE4 to respect the Max t/b-n.

With so many modern, mikkt-based baking options available these days, if you are going to be using UE4 a lot I would recommend baking elsewhere. Max baking really is a clunky and outdated, comparatively.

deemccabe · Nov 2016

sorry for the late reply, thanks very much for the response. ok so if i export with tangents and bi normals checked, do you think that will solve my problem becuase ue4 will be able to get the tangent basis correct? please correct me if im wrong. Thanks

musashidan · Nov 2016

I just realised that you mention above that you didn't match UV/SG splits. This is necessary if your baker/engine do not have the same tangent basis(unsynced) which Max/UE do not.

Those black lines on corners are classic missed rays due to a non-averaged/unsynced bake with no UV/SG splits.
It's good that you're trying to understand the whole process and working through it but eventually you might want to avoid some of these issues by trying out Xnormal or Handplane. Both are free, modern, and use MikkT.

Or if intent on continuing with Max you should check out this free script. Makes baking in Max much more streamlined.

https://www.youtube.com/watch?v=A3Psdkanh5s

deemccabe · Nov 2016

thanks very much for all the help, ill check that vid out in a min. just out of query i have split the smoothing groups at any 80/90 degree angle but i havent split the uvs because will it not be a pain in the ass to texture if the uvs are all split??

musashidan · Nov 2016

Those black lines are because there are no uv splits. UV/SG splits are one of the fundamental things to understand in the NM baking process as it is a strict rule when baking non-averaged. This is why people use an averaged cage bake or an averaged 'synced' bake. I won't explain the reasons here as you can read all but it on the stickies above.

deemccabe · Nov 2016

ok so i understand that when you have those 90 degree angles that it is best to split the uvs to avoid the black lines ive been getting. my question then is how to do go about texturing when the uvs have padding between each interaction giving obvious visual breaks in the texture??

musashidan · Nov 2016

My answer to that would be........in 3D.

Or if you're limited to Max you could texture in Photoshop as normal and paint out the seams in Max's(hopelessly neglected) Viewport Canvas. Or you could use the Render to Surface tools to create a seamless, blended base texture, and go from there.

I believe 3DMotive.com have an old tank texturing tutorial done in PS. Might be worth a look if that's your workflow.

deemccabe · Nov 2016

thanks very much for all your help, i really appreciate it. thanks for taking the time to help me through the process.

deemccabe · Nov 2016

ok so im back again lol. i still keep getting this weird line at the hard edge. ill explain the process im using and then show a screen at the bottom of the problem im having. any help would be greatly appreciated. Thanks

1) i have my low poly, high poly and cage

2) the low poly model has split uvs at any sharp angle

3) applied a different smoothing group to all the different uv splits

4) used x normal to bake with tangents and bi normals checked

5) exported low poly from 3ds max .fbx with tangents and bi normals ticked

6)imported to ue4 with normal generation method set to mikk Tspace

this is the result.. it looks fine at a bit of distance but this happens when your up close

Image: https://us.v-cdn.net/5021068/uploads/editor/zx/t35fj1johlco.jpg

huffer · Nov 2016

Looks like normal map limitation..there is no fix (Maya baker handles it better though).

Obscura · Nov 2016

You could try using "full precision UVs". This is a checkbox in the static mesh editor. It would probably help some, but yeah, you should not expect it to be 100% perfect....

Scruples · Nov 2016

With a Hard edge+UV seam at 90° this is going to happen. It's caused from padding and texture filtering, it would be better to add some more geometry and a soft edge, any interpolation errors wouldn't be noticeable then. A model with geometry this low and normal map this high isn't well optimized anyway.

pior · Nov 2016

deemccabe said:

1) i have my low poly, high poly and cage

2) the low poly model has split uvs at any sharp angle

3) applied a different smoothing group to all the different uv splits

4) used x normal to bake with tangents and bi normals checked

5) exported low poly from 3ds max .fbx with tangents and bi normals ticked

6)imported to ue4 with normal generation method set to mikk Tspace

These steps seem to suggest that the lowpoly model used for baking is different from the one used in UE. While this is not necessarily a problem, it could be.

But first and foremost I would suggest you to upload your low and high so that people can test things out for you.

deemccabe · Nov 2016

thank you everybody for the helpful advise. ive managed to fix it by adding a chamfer to the hard edges in 3ds max and assigning 1 smoothing group to the piece and that gave me the smooth edges i was looking for in ue4. this an image of the result. please tell if you think it looks right or not. again many thanks for all the help.

Image: https://us.v-cdn.net/5021068/uploads/editor/ch/iqnkc8cc2fvx.png

deemccabe · Nov 2016

hey guys, just a quick question. i have modelled a high poly tank and a lower poly version. im just curious as to weather or not i even need to high to low poly the larger pieces. the reason i had started doing it was to get the smooth edges on the larger pieces rather than the hard edges as you can see if you look at my previous pics on this tread. i will high to low normal map for all the nuts and bolts and things but im interested to hear everyone's opinion to if its wort it or not for the larger piece edges?? ive attached a pic of both the high and the low poly versions and if you could give me your opionion on wether not it even needs high to low normal maps for the hard edges it would be greatly apprecited. Thanks

low poly

Image: https://us.v-cdn.net/5021068/uploads/editor/ha/id3g3wo7vwju.png

high poly

Baking normal map problems

Replies