Alright noted. I'll preview in UE4 rather than Max since the end result is supposed to be better there. Alright I'll see what I can do about the tight edges.
Will post very shortly. Working things out. Created some extra meshes and I'm just working on those. I've got UE4. (If you didn't know I formatted my WIN10 OS).
User 'Natsta' helped me out by creating personalised tutorials for me clearing up on how I should go around exploding my mesh and why I should be doing it. I'm quite clear on how I should be exploding my meshes now. So happy because if I look back 3 months ago I was freaking out over this stuff lol.
You were right. Viewing an asset gives you different results in the UE4 viewport compared to the viewport of max. The normal map looks nearly perfect in UE4. The straight edges look quite round.
Hey. So I've created this crate model. Looks terrible I know. But that isn't the point. The point is, have you noticed how you can stll see hard edges when the NM is applied? No idea why these pics are 1.9mb in total. Anyway, the normal map still isn't as intense. There's huge white seams across the edges. I saw a post about this some time ago and 2 things come to mind. Tangent Base Normal Maps and Object Space Normal Maps. Now to my knowledge, the former is what I'm using. The normals are rendered out in accordance to every normal on every face of the mesh. The latter is more to do with the world around the object... or something like that.
Sometimes it helps with the hard edges if you bake the normals at a really high resolution and scale it down afterwards. You should also make sure your projection cage isn't getting split at hard edges.
Not to throw a spanner in your learning progress as this is all valuable and vital info to digest, but you can save yourself a lot of hassle with the older 'hacky' method of baking by moving to a synced(averaged vert) baking workflow. Because your baker (XN) and destination engine (UE) both use the same tangent basis (mikkt) you can 'sync' the workflow. The baker and the engine compute the tangents and binormals automatically. With this method you don't have to split smoothing groups/uv islands(consequently less verts in-engine), you have more freedom when unwrapping, and you don't even need to use a cage.
The only trade-off/caveat I see with this workflow is that sometimes you can get compression/mipping artifacts in-engine, but that is something you want to test out yourself and may be unnoticeable in most cases.
Also, when baking in Substance Painter (which is now free for students) you don't need to explode to parts as this is done automatically based on a naming convention.
Also, it's hard to follow this thread as you keep changing models before you've finished one.
Sometimes it helps with the hard edges if you bake the normals at a really high resolution and scale it down afterwards. You should also make sure your projection cage isn't getting split at hard edges.
Hmm. I'll try that. Thanks. What do you mean by my projection cage isn't getting splits at hard edges? Here is my cage for the model, is this okay?
@musashidan I'll look into the older method you're talking about. Thanks. Also, I'm fully aware of SP's convenience in terms of baking. It's killing me to use it but this is how I look at it. I could either just make everything easier for me and start using SP. But, I want to learn the basics of traditional baking methods. It just gives me more potential as a 3D Artist. I could always combine and alter aspects of both workflows in accordance to my needs. Like SP and Adobe Photoshop for example. Instead of thinking 'What should I use?' I came to the conclusion I should just use both. Best of both worlds. So I will be using SP for everything in the near future, I just want to get down traditional baking.
One thing I don't understand in your workflow is, you're telling me that my model should have 1 smoothing group? So I would select all the faces, click 'clear smoothing groups' and click '1'? So no hard edges?
@musashidan I'll look into the older method you're talking about. Thanks.
One thing I don't understand in your workflow is, you're telling me that my model should have 1 smoothing group? So I would select all the faces, click 'clear smoothing groups' and click '1'? So no hard edges?
You're using the older method The 'synced' workflow I'm talking about is the more modern method. And yes you're right, I commend your dedication to learning this stuff as thoroughly as you can. Understanding the WHY behind why we do certain things when baking NMs is so important and overlooked in favour of mindless button-pushing by a lot of beginners. I'm not recommending dropping everything you're currently doing, I was just making you aware of the 'synced' workflow. This industry is constantly moving and none of us can afford to stay static in our approach as new workflow tools and tactics are continually being introduced. Even a lot of senior artists are so set in their workflows that 'hard edge splits and cages' is almost a mantra.
Yes, a single smoothing group -averaged normals. The baker/destination engine are synced as they both use the same tangent basis and calculate the tangents/binormals on import so the surface vector RGB data is encoded in the texture and uv/SG splits do not need to be 'hacked' into that data.
@musashidan Oh okay. So, for the synced workflow, would I have all the faces of my model as one smoothing group? What would I do with the UV's? Split at hard edges? Sorry I really don't understand the method you're talking about. Could you elaborate the steps I would need to take? Yeah with me, if I don't learn the 'Why' of something, I don't really remember it.
Yes, a single smoothing group, no splits necessary. If you read through the 3 threads I linked above I go over the steps. It's actually very straightforward.
Just swap out the substance Painter parts for Xnormal. It uses mikkt by default(and also has compute binormal in pixel shader on by default) But you can double check this by clicking the little plug icon left of the XN UI, find/click the mikkt plugin, click configure.
Here are the steps:
- Low and high silhouette matching closely - single smoothing group(no UV/SG splits necessary) -
export .fbx - low-poly has smoothing groups checked and has already
been triangulated. Tangents&Bi-Normals unchecked(SP will look after
this - see next step) - SP import - compute tangent space per fragment checked - Bake with between .01 and .04 ray distance -
perfect results every time, and because it's using mikkt with T&BiN
being calculated by baker you are guaranteed that it will be perfect in
any other software that is synced - mikkt.
Understood. I looked through all 3 threads previously but I thought the instructions were maybe specific to OP's situation. Say, if the the smoothing groups can only be 1. That means the whole mesh will be smoothed therefore removing the look of sharp edges. So what if I want certain edges on my model to be sharp and some to be smooth? It's the part that confuses me the most. If I want one area sharp but some other areas smooth, how does that work out in all of this?
I also understand that faces that are 90 degrees to each other don't need to be split in the uv's? So I can have them stitched? I'll try this method out on this crate mesh I've got and I'll post results shortly. Thanks.
The hard/soft edges are only stored in the RGB data of the texture, not on the physical mesh as it is all averaged verts (single SG) so once you apply your NM the edges will appear as they are on the high poly. And yes, you are free to do what you want with your UVs, you can even have a single shell if you want.
Good idea. Do some tests and I'll answer any questions if I can.
Okay so I followed your steps and I got this result in UE4: One thing, I believe XNormal only accepts .obj files. Not .fbx. The LP and Cage have a single smoothing group. I left the UV's untouched. Don't mind the colour of the mesh. I don't know how to change the base colour of the meshes material yet, (Darky grey).
Make sure you are Importing normals when you are using synced workflow, select the drop down in import options on the mesh and select import normals. Reimport after, or hard import.
This method is easier, but the old method is still widely used within the industry, you need to know both.
You shouldn't need to flip any sizzles using this method. Did you make sure to NOT export tangents/binormals and allow XN to compute them?
See, thats what I probably did wrong. I decided to test out if XN likes .fbx or not and I realised it does. So I tried the method and now this is what I get: .fbx options: XN Highpoly: XN Lowpoly+Cage: XN Baking Options:
The way I've layed out the scene should be easy enough for you to figure out. 3 layers: LP, HP & Cage. The cage layer consists of the exploded LP, HP and Cage. You already know this I know. Just want to clear it up to avoid any confusion. Thanks for looking into this.
Just baked out a map in both XN and SP. Both results were identical using the workflow above. The only difference is that Xnormal uses OpenGL(+Y) and Substance Painter(and UE4) use DirectX(-Y). This is essentially just a matter of flipping the swizzle from + to - for UE4.
Also, I did a re-UV on your mesh. Just stitched up the islands. Single smoothing group on low-poly. And a tip for unwrapping identical Objects is to unwrap one and then duplicate and place. This also works for baking(instead of UV offsetting) If you want variation on your texture then unwrap and bake 2 elements and then vary the texture by moving/rotating them around when you clone them. This obviously isn't what you want to do if you want a completely unique texture. So it's a trade-off between UV space and 100% unique texture.
Substance Painter Viewport(which will be identical in UE in terms of the normal map because they are synced)
Oh my god. My model looks so beautiful. :') Even though the model it self is a result of me spending 10 minutes on it to quickly make it... the edges of it.... looks gorgeous. I spent a whole 5 minutes looking at it.
Anyway. So how would I get this result? I need to flip the sizzles on my own engine right? Could you instruct me on how I can get the results you got? Also, some of the planks of wood were the same scale so yes I could have duplicated them... but most of them were at a different scale, so I don't know if I would have gotten weird results if I tried to duplicate the uv's.
EDIT: Oh wait I get what you mean with the duplicating part. I make one plank, UV it and THEN duplicate them and rescale them to my needs. That way the UV's rescale too n accordance to the actual plank mesh. Ahh okay thanks for the tip.
One thing I don't understand however is, why there is only one set of UV's for one plank. I feel like I'm stupidly not realising something here. I've noticed how there is only one set of uv's for one plank but in the end all of my planks have that normal map texture on them.
About my previous UV's. Aside from how I could have conveniently UV'd one plank and duplicated it, could you tell me how I could have improved them more? What's exactly wrong with my UV's and any advice for my future UV'ing endeavours?
To get this result just follow these steps that I posted above:
- Low and high silhouette matching closely - single smoothing group(no UV/SG splits necessary) -
export .fbx - low-poly has smoothing groups checked and has already
been triangulated. Tangents&Bi-Normals unchecked(UE&XN will look after
this - see next step) - XN import - Xnormal uses mikkt by
default(and also has compute binormal in pixel shader on by default) But
you can double check this by clicking the little plug icon left of the
XN UI, find/click the mikkt plugin, click configure. Also in the Normal map options change +Y to -Y. Remember, UE uses DirectX so the Y swizzle needs to be negative.
- Bake with with default ray distance(no cage required)
- In UE leave the defaults on 'import normals'. UE will auto-detect your NM and set sRGB and compression accordingly.
-
perfect results every time, and because it's using mikkt with T&BiN
being calculated by baker you are guaranteed that it will be perfect in
any other software that is synced - mikkt.
There is only one set of UVs for the plank because they are stacked(overlapped on top of eachother) When you duplicate a UVed mesh the UVs sit in exactly the same spot in UV space. So what you are seeing is all those planks sitting on top of eachother. This is the trade-off I was talking about above: More UV space Vs more unique texture.
Your UVs are OK. There is some large texel density discrepency but you already know about this. This can be easily solved by clicking the 'rescale elements' button in the UV editor. You have too many unesscessary seams, but you obviously did this for the uv/sg split workflow. If using the 'synced' workflow you can minimise seams as much as is suitable. This is also a bonus when texturing. You don't have enough padding. In some places there is only a pixel. This will cause island bleed. Padding is needed to avoid this for both baking and mipping.
Okay. I understand the UV'ing part now. All the planks are there it's just that they're overlapping. One thing I wanted to clear up with this workflow is... smoothing groups. So edges that will be hard or soft are determined using the highpoly's smoothing groups? So that means that the highpolys smoothing groups will have to be appropriately set? If an edge is smooth in the highpoly, that means that edge will be smooth in the lowpoly in terms of normals.
I'll give your instructions a go and see what results I get.
The high poly smoothing groups are irrelevant as the surface smoothness is being driven by sub-division. The only thing you ever have to worry about with your low-poly is that it looks good on the screen. The surface normals of the high poly are being baked directly to the low-poly so if it looks clean and you set up the bake correctly then the results will be 1:1.
Could you send me a pic of the normal map that you rendered out? Because this is what I'm getting and I have no clue what to expect: Also, compute binormal in pixel shader was not on by default so I turned it on. Yeah I just tested out the map in UE4 and I'm doing something wrong. I'm getting weird results. Here are my settings:
The high poly smoothing groups are irrelevant as the surface smoothness is being driven by sub-division. The only thing you ever have to worry about with your low-poly is that it looks good on the screen. The surface normals of the high poly are being baked directly to the low-poly so if it looks clean and you set up the bake correctly then the results will be 1:1.
Ahh okay. I got it. So I get what I see in max's in viewport.
You have averaged normals turned on in XN. You need to leave it at use exported normals. That's one of the main ideas behind this workflow, because baker/engine use same tangent space (mikkt) you have to allow them to calculate the bi-normals.
Also, you still need to add padding to your UVs.
I think you need to step back a bit. You are overloading your brain with too much info. If you read back over this thread carefully and take it one step at a time. And read the 3 threads I linked above again. This workflow is very simple but you need to follow all the steps carefully.
Well the reason I switched it to averaged normals is because if I don't I get this: If we can't figure out what I'm doing wrong I think I'll create another mesh piece to study this workflow. A clean slate and I'll follow the instructions step by step.
Yep it's a plan. I'll let you know how I do. Really apreciate you taking the time to help me out like this. People like you and @3dReaper kind of keep inspiring me to learn so that maybe one day I get to your level and help out other people in need to as well as moving on to actually making proper projects of mine. I really love your YouTube tutorials btw, keep them coming.
No problem. Happy to help. Glad my vids are useful.
And here's what the Nm looks like. The extreme gradients are due to the averaged verts(single smoothing group) but don't worry about that as mikkt takes care of it, and it's what an NM using this workflow should look like. The only caveat that I see with this workflow is that the gradients can, in some cases, cause artifacting when compressed/mipped, but is mostly unnoticeable when the asset is fully textured and in-engine. Best of luck.
okay so how does this affect my workflow if say I want to use something like NDO2? And say in the future I do get noticable artifacts... How would I resolve them? Another thing. How does this correlate in terms of other maps? AO? All the same settings right? I don't change the -Y sizzle?
Alright so I created a simple cube. Duped it and made a turbosmoothed highpoly version. Made sure they are aligned, not sure if that matters though. The LP is on 1 smoothing group. Made sure all the XN parameters were correct according to your corrections. Now to my surprise, the normal map result I got looks like the result you got: The gradients look correct. But the result I get in UE4 is this: It's like the normal map isn't being read at all. I've got it applied on the mateiral here: The material is applied to the mesh. I know since I changed the base colour of it. What could be the issue?
Can you post an image of your highpoly in Max? And your import dialogue in UE. And your UVs? Double click your NM in UE and make sure that sRGB is unchecked. Do this for all linear maps(greyscale) Also make sure that your NM is set to use the correct compression method.
NDO doesn't bake maps. DDO doesn't use mikkt.
The artifacts shouldn't be much of an issue. I havn't had any issues with them and I've been baking this way for 1.5 years.
Your normal map is the only map that cares about surface normals in this way. It is an encoded data map not like other textures.
I think you may be worrying too much about 10 things at once rather than concentrating on 1 step at a time.
Alright. Sorry gentlemen for my lack of posts these past few days. I'm back now and I'll be posting again. Hopefully we can figure out this normal map mystery. Will be posting shortly.
Okay I'm back. I repeated the whole thing all over again just to make sure I can get a good scan and I'm getting exactly the same result. A lot of pictures.
My highpoly: LP in correlation with HP (maybe it will help?): Excuse my amazing mouse handwriting.
is 400 even a lot? Also, you're the great gentlemen who introduced me to this workflow! I knew nothing about it. I did some searches about it and its used but its not as common as the other workflow so idk documentation is meh.
Okay so what would I do? Increase the pixels between my uv's? Increase the padding? If this will fix it... then wow thats one easy workflow. It just hit me now, how easy that actually is. I'll work on it in the morn and let ya know.
btw, as a far as texture bleeding and padding goes, I know minimum amounts.
Replies
AO:
Also, preview your mesh in unreal engine instead of what looks like Max? That should solve some of the issues.
User 'Natsta' helped me out by creating personalised tutorials for me clearing up on how I should go around exploding my mesh and why I should be doing it. I'm quite clear on how I should be exploding my meshes now. So happy because if I look back 3 months ago I was freaking out over this stuff lol.
You were right. Viewing an asset gives you different results in the UE4 viewport compared to the viewport of max. The normal map looks nearly perfect in UE4. The straight edges look quite round.
The point is, have you noticed how you can stll see hard edges when the NM is applied?
No idea why these pics are 1.9mb in total.
Anyway, the normal map still isn't as intense. There's huge white seams across the edges.
I saw a post about this some time ago and 2 things come to mind. Tangent Base Normal Maps and Object Space Normal Maps. Now to my knowledge, the former is what I'm using. The normals are rendered out in accordance to every normal on every face of the mesh. The latter is more to do with the world around the object... or something like that.
Help?
The only trade-off/caveat I see with this workflow is that sometimes you can get compression/mipping artifacts in-engine, but that is something you want to test out yourself and may be unnoticeable in most cases.
Also, when baking in Substance Painter (which is now free for students) you don't need to explode to parts as this is done automatically based on a naming convention.
Also, it's hard to follow this thread as you keep changing models before you've finished one.
Have a look at this thread where the I baked out the OP's problem mesh using the workflow I'm describing: (second last post)
http://polycount.com/discussion/comment/2443944#Comment_2443944
And also this thread where I did same for OP using same workflow:
http://polycount.com/discussion/comment/2441974#Comment_2441974
And this thread:
http://polycount.com/discussion/comment/2428355#Comment_2428355
@musashidan
I'll look into the older method you're talking about. Thanks.
Also, I'm fully aware of SP's convenience in terms of baking. It's killing me to use it but this is how I look at it. I could either just make everything easier for me and start using SP. But, I want to learn the basics of traditional baking methods. It just gives me more potential as a 3D Artist. I could always combine and alter aspects of both workflows in accordance to my needs. Like SP and Adobe Photoshop for example. Instead of thinking 'What should I use?' I came to the conclusion I should just use both. Best of both worlds. So I will be using SP for everything in the near future, I just want to get down traditional baking.
One thing I don't understand in your workflow is, you're telling me that my model should have 1 smoothing group? So I would select all the faces, click 'clear smoothing groups' and click '1'? So no hard edges?
Yes, a single smoothing group -averaged normals. The baker/destination engine are synced as they both use the same tangent basis and calculate the tangents/binormals on import so the surface vector RGB data is encoded in the texture and uv/SG splits do not need to be 'hacked' into that data.
Oh okay. So, for the synced workflow, would I have all the faces of my model as one smoothing group? What would I do with the UV's? Split at hard edges? Sorry I really don't understand the method you're talking about. Could you elaborate the steps I would need to take?
Yeah with me, if I don't learn the 'Why' of something, I don't really remember it.
Just swap out the substance Painter parts for Xnormal. It uses mikkt by default(and also has compute binormal in pixel shader on by default) But you can double check this by clicking the little plug icon left of the XN UI, find/click the mikkt plugin, click configure.
Here are the steps:
- Low and high silhouette matching closely
- single smoothing group(no UV/SG splits necessary)
- export .fbx - low-poly has smoothing groups checked and has already been triangulated. Tangents&Bi-Normals unchecked(SP will look after this - see next step)
- SP import - compute tangent space per fragment checked
- Bake with between .01 and .04 ray distance
- perfect results every time, and because it's using mikkt with T&BiN being calculated by baker you are guaranteed that it will be perfect in any other software that is synced - mikkt.
Say, if the the smoothing groups can only be 1. That means the whole mesh will be smoothed therefore removing the look of sharp edges. So what if I want certain edges on my model to be sharp and some to be smooth? It's the part that confuses me the most. If I want one area sharp but some other areas smooth, how does that work out in all of this?
I also understand that faces that are 90 degrees to each other don't need to be split in the uv's? So I can have them stitched?
I'll try this method out on this crate mesh I've got and I'll post results shortly. Thanks.
Good idea. Do some tests and I'll answer any questions if I can.
One thing, I believe XNormal only accepts .obj files. Not .fbx.
The LP and Cage have a single smoothing group.
I left the UV's untouched.
Don't mind the colour of the mesh. I don't know how to change the base colour of the meshes material yet, (Darky grey).
This method is easier, but the old method is still widely used within the industry, you need to know both.
See, thats what I probably did wrong. I decided to test out if XN likes .fbx or not and I realised it does. So I tried the method and now this is what I get:
.fbx options:
XN Highpoly:
XN Lowpoly+Cage:
XN Baking Options:
https://www.dropbox.com/s/zbjfeeq3q3cobgm/cratescene.max?dl=0
The way I've layed out the scene should be easy enough for you to figure out. 3 layers: LP, HP & Cage.
The cage layer consists of the exploded LP, HP and Cage. You already know this I know. Just want to clear it up to avoid any confusion.
Thanks for looking into this.
Also, I did a re-UV on your mesh. Just stitched up the islands. Single smoothing group on low-poly. And a tip for unwrapping identical Objects is to unwrap one and then duplicate and place. This also works for baking(instead of UV offsetting) If you want variation on your texture then unwrap and bake 2 elements and then vary the texture by moving/rotating them around when you clone them. This obviously isn't what you want to do if you want a completely unique texture. So it's a trade-off between UV space and 100% unique texture.
Substance Painter Viewport(which will be identical in UE in terms of the normal map because they are synced)
Even though the model it self is a result of me spending 10 minutes on it to quickly make it... the edges of it.... looks gorgeous.
I spent a whole 5 minutes looking at it.
Anyway. So how would I get this result? I need to flip the sizzles on my own engine right? Could you instruct me on how I can get the results you got?
Also, some of the planks of wood were the same scale so yes I could have duplicated them... but most of them were at a different scale, so I don't know if I would have gotten weird results if I tried to duplicate the uv's.
One thing I don't understand however is, why there is only one set of UV's for one plank. I feel like I'm stupidly not realising something here. I've noticed how there is only one set of uv's for one plank but in the end all of my planks have that normal map texture on them.
About my previous UV's. Aside from how I could have conveniently UV'd one plank and duplicated it, could you tell me how I could have improved them more? What's exactly wrong with my UV's and any advice for my future UV'ing endeavours?
- Low and high silhouette matching closely
- single smoothing group(no UV/SG splits necessary)
- export .fbx - low-poly has smoothing groups checked and has already been triangulated. Tangents&Bi-Normals unchecked(UE&XN will look after this - see next step)
- XN import - Xnormal uses mikkt by default(and also has compute binormal in pixel shader on by default) But you can double check this by clicking the little plug icon left of the XN UI, find/click the mikkt plugin, click configure. Also in the Normal map options change +Y to -Y. Remember, UE uses DirectX so the Y swizzle needs to be negative.
- Bake with with default ray distance(no cage required)
- In UE leave the defaults on 'import normals'. UE will auto-detect your NM and set sRGB and compression accordingly.
- perfect results every time, and because it's using mikkt with T&BiN being calculated by baker you are guaranteed that it will be perfect in any other software that is synced - mikkt.
There is only one set of UVs for the plank because they are stacked(overlapped on top of eachother) When you duplicate a UVed mesh the UVs sit in exactly the same spot in UV space. So what you are seeing is all those planks sitting on top of eachother. This is the trade-off I was talking about above: More UV space Vs more unique texture.
Your UVs are OK. There is some large texel density discrepency but you already know about this. This can be easily solved by clicking the 'rescale elements' button in the UV editor. You have too many unesscessary seams, but you obviously did this for the uv/sg split workflow. If using the 'synced' workflow you can minimise seams as much as is suitable. This is also a bonus when texturing. You don't have enough padding. In some places there is only a pixel. This will cause island bleed. Padding is needed to avoid this for both baking and mipping.
One thing I wanted to clear up with this workflow is... smoothing groups. So edges that will be hard or soft are determined using the highpoly's smoothing groups? So that means that the highpolys smoothing groups will have to be appropriately set? If an edge is smooth in the highpoly, that means that edge will be smooth in the lowpoly in terms of normals.
I'll give your instructions a go and see what results I get.
Also, compute binormal in pixel shader was not on by default so I turned it on.
Yeah I just tested out the map in UE4 and I'm doing something wrong. I'm getting weird results.
Here are my settings:
Ahh okay. I got it. So I get what I see in max's in viewport.
Also, you still need to add padding to your UVs.
I think you need to step back a bit. You are overloading your brain with too much info.
If we can't figure out what I'm doing wrong I think I'll create another mesh piece to study this workflow. A clean slate and I'll follow the instructions step by step.
And here's what the Nm looks like. The extreme gradients are due to the averaged verts(single smoothing group) but don't worry about that as mikkt takes care of it, and it's what an NM using this workflow should look like. The only caveat that I see with this workflow is that the gradients can, in some cases, cause artifacting when compressed/mipped, but is mostly unnoticeable when the asset is fully textured and in-engine. Best of luck.
And say in the future I do get noticable artifacts... How would I resolve them?
Another thing. How does this correlate in terms of other maps? AO? All the same settings right? I don't change the -Y sizzle?
Now to my surprise, the normal map result I got looks like the result you got:
The gradients look correct.
But the result I get in UE4 is this:
It's like the normal map isn't being read at all. I've got it applied on the mateiral here:
The material is applied to the mesh. I know since I changed the base colour of it.
What could be the issue?
NDO doesn't bake maps. DDO doesn't use mikkt.
The artifacts shouldn't be much of an issue. I havn't had any issues with them and I've been baking this way for 1.5 years.
Your normal map is the only map that cares about surface normals in this way. It is an encoded data map not like other textures.
I think you may be worrying too much about 10 things at once rather than concentrating on 1 step at a time.
I repeated the whole thing all over again just to make sure I can get a good scan and I'm getting exactly the same result.
A lot of pictures.
My highpoly:
LP in correlation with HP (maybe it will help?):
Excuse my amazing mouse handwriting.
UV's:
NM:
UE4 Import options:
UE4 NM Texture Options:
Result:
I made sure the XN stuff was all correct. Any ideas?
You have no padding on your UVs so the bleed-over is baking the abrupt surface normal change that you can clearly see in the NM.
Okay so what would I do? Increase the pixels between my uv's? Increase the padding? If this will fix it... then wow thats one easy workflow. It just hit me now, how easy that actually is. I'll work on it in the morn and let ya know.
btw, as a far as texture bleeding and padding goes, I know minimum amounts.