You can just stitch the seams. UV splits don't matter with this workflow.
The reason there isn't much info about this workflow is that it's relatively new (last few years) and it's only now that applications as finally syncing tangent basis through mikkt. Whereas the older method has been around as long as I can remember and is by far the most popular method simply because it's an entrenched workflow and was the only option (along with object space skewmesh conversion) for a looong on time.
Wait wait, so my seams do have to be stiched? Confused. If they don't matter, then why do they have to be stiched? Trying it now. Will post results. And yeah I figured this workflow was quite modern.
Your seams don't matter in the same sense as they do in the uv split workflow. They can be stitched or not.
All the info is in this thread. I've already explained the UVs factor earlier in the thread. Maybe you should stick with the cube test and just read back over my posts.This really is as simple a workflow as it sounds once you understand why you are doing certain steps in the process.
"You can just stitch the seams. UV splits don't matter with this workflow." Okay now I understand this. I was blatantly thinking of something else when you said this. Overthinking it as usual. Visible seams can be stiched if they need to be. Having them split or stiched isn't required at all.
I looked back through the thread, I followed your steps one by one carefully, twice. I still can't spot the problem. I can see that it is a dead simple workflow. It seems like a UE4 thing. Since if the normal map that is coming out of XNormal is how it's supposed to look (compared to the normal map you produced) then I obviously can't be doing anything wrong there.
Here is what I did with my UV's if that changes anything at all: I'm afraid to say I'm out of ideas. I feel like I'm missing something blatantly stupid in UE4. I don't have much knowledge on UE4 in the first place so I can't put my finger on it.
Did you flip the green channel on your NM? Xnormal is OpenGL by default. UE4 is directX.
I think I'll do a step by step video for you and put it on my YT channel as a request, if you'd like? Just give me a day or 2. Very busy with work atm.
No I don't think I did. I flipped the swizzle but ik thats not it. I'll give it a go.
As for the video. That would be amazing. As long as its cool with you and not taking too much time off of yours hands. It would be the star video to watch for people using this workflow. I would say it would be Polycount Wiki worthy.
Hey. I have no clue on how to flip my green channel for my nm. Would I do it in UE4? Did a few searches and I'm getting nothing result worthy. Sorry if its a pretty obvious thing.
Okay I tried to find this option in UE4 but I can't find it. I looked every where twice. I'm gonna look again. I double click the normal map which brings up the texture editor. Is it somewhere on there?
Yes the docs! That should have been my first place to go for this. Thanks. I'll try this and see if this can hopefully give me a worthy result. I think this may do it.
Okay so this has changed something. It's still not great but its kind of started to work a little now. I've noticed that 3DReaper said: Red=Right (RR) Green=Bottom (GB)
This isn't the case with my NM: Green is coming from the bottom but red from the left instead of the right. But I don't how that affects this. Obviously now Flip Green Channel is checked. Some edges seem to be rounded but some still sharp in correlation to the map. No base colour or anything is applied to the mesh. Just a normal map.
I should have that video tutorial I mentioned created at the weekend. Just been very busy. Can you post a high poly image of the cube? How sharp are the corners? You shouldn't have to mess with the Red channel at all.
Yeah don't worry. I figured you were busy. What I'm going to do is try to get the result in Substance Painter. Atleast if I can get it in there then we've really isolated the problem to potentially UE4.
None the less, here is my highpoly WITH and WITHOUT turbosmooth. I think the topology of it is fine. All loops are evenly spaced even though that doesn't matter:
Okay so in painter I'm getting different results. Weird. Maybe I'll need to repeat it for painter but I'm gonna wait until @musashidan's tutorial or a reply. See if I can solve this in UE4 first.
As someone who struggled a lot with getting bakes I started following @musashidan's steps and have been getting very consistent ones since. Here's the steps I took for a simple cube:
Base mesh and it's HP. UV layout is the same as yours.
Bakes in Substance and Xnormal. Notice how the 3rd image is from xnormal and the green channel is different from the substance one. Middle one has it flipped so that both are the same.
Here's the settings from Substance. Pretty much as it comes out of the box. Same applies to xnormals. Only change in both is max rear distance to get the corners of the cube to bake nicely
And how it looks in UE4 with the base setup. You'll see I loaded 2 normal maps (the one from Xnormals and Substance). Both give very similar results, but I tend to get weird stuff in general with xnormals (its the one that looks like it has a grid in ue4) so what you see is with the substance bake applied.
As a note, I've barely used UE4 so far so there might be a better way to have it display. I quickly read the docs linked above about how to flip the green channel in UE4 and when I do I get a result similar to yours. So I reckon that's where your issue lies. So flip it in UE4 or do it in photoshop before hand.
Hey @Tomseas. I appreciate your explanation. But the thing is, I've already tried flipping my green channel in UE4. It helps somewhat (some edges become smooth and some stay sharp) but it still doesn't give the desired result. You're correct about how your HP and base are the same as mine. So I'm not doing anything wrong there. I'm going to try out increasing the max rear distance value like you've done and see if that changes anything.
This does sound like a seriously easy workflow but I honestly don't know where I'm going wrong. EDIT: Okay so I've tried in SP but I'm getting another weird result. The normal map isn't very effective. Some edges are getting slightly rounder but same result.
My parameters are set exactly like yours. Compute tangent space per fragment is checked. My normal map format is OpenGL but changing it to DirectX weilds the same result. : / This is very weird. I have noticed however that your normal map looks quite different to mine. Theres that blurring on the edges which means thats how it is supposed to be since its working as intended. My NM looks pretty sharp. Here is the solo normal map view:
@CodeferBlue could you drop all your files in dropbox? From the looks of it, your HP's bevels are either too sharp so they don't show or you exported it without the turbosmooth applied.
OKAY. So. I applied my turbosmooth. And it changed something. My normal map looks MUCH better. So I have to actually BAKE my turbosmooth modifier into proper geometry and not just preview geometry? If so, thats something incredibly essential that I didn't know.
I still however get sharp corners OR is that just normal? Pun not intended. Look: Normal Map: Baked in substance Any tips? EDIT: I bumped up the map resolution from 512x512 to 2048x so it looks much better. But any other tips would be nice too. Or maybe this is just how its supposed to look and I'm just crazy. :P But it looks great. Very close to the actual highpoly. But yeah any clarification on if I need to have by turbosmooth modifier baked down into the actual mesh would be good.
Been real busy lately. Sorry. So when you export your HP from max if you do it as an FBX it will not bake the turbosmooth in even if the box is ticked in the options. Unless my max is bugged you can test it out using a box with a turbo applied on it export and import it and it will have disappeared. If you export HP's as OBJs however they will stick. So either do OBJs so you keep the stack intact, or just clone your HP, bake the modifiers and export as FBX.
As you found out, the higher resolution you go for your textures, the better it will look.
As for the sharp corners, those are normal because what you see is your low poly which has those corners. If you look from certain angles the low poly will be visible no matter what, which is why it is often stressed that one of the important things in a low poly is keeping the silhouette intact. On my substance picture you can also see the sharp corners of the low poly.
@Tomseas Okay I understand now. No wonder I was confused. I was previously working with OBJ's. So the export file type matters incredibly and can really affect your workflow. Thanks for helping me out there. Yeah about the sharp corners, I got it. At the end of the day, a normal map is only just a 2d texture map. It can only contribute so much to the desired look. It looks great however.
This small problem went on for too long. I'm going to try to apply this convenient method and make some quick props to see how it goes. I've already created one. I've put no thought into this at all, just for testing purposes. I can clearly see the smooth edges. The resolution for its map is 1024x if I remember correctly. Its realistic since not all projects will have 2k textures for assets.
About one 15 minute youtube video tutorial on normal maps and baking would have cleared up ton of these questions because you would actually see the settings the tutorial maker used and see what he did with the high poly, etc. For modeling, Arrimus3D and Chamferzone(to learn what chamfering can do vs support edges) youtubers are about all you need. I'd suggest following along with Arrimus with a few models yourself. Some of what your probably missing is just repetition, which hammers home to concepts you already know, but also allows you to not have to think about what your've already learned so much, and focus on further techniques that build upon the existing stuff you already know. I'd also suggest finding a good explanation of what normal maps and baking actually do, because the terms being used are kinda arbitrary based on concepts that are kinda arbitrary. Seeing how rays are cast and why they miss for example might really help your understanding. Especially at this point now that you know many things, watching baking tutorials will probably give you many AH-HA! moments.
One thing to remember though, i would recommend always clarifying your knowledge of normals maps and baking on polycount, because after just going through the process of learning this stuff myself over the last couple years, polycount itself tends to have the best info, especially thanks to Earthquake's sticky threads. All the info provided in those sticky threads by Earthquake are stuff you will have to learn and problems you will run into at some point in a 3D career. Just remember they're there is all so when you encounter problems i'd go straight there and check those. I didn't know about them when i was learning. But do watch some videos like the ones below, just know that tutorial makers on youtube tend to pass information as absolute in trying to be helpful and clear, but its not always accurate.
Replies
The reason there isn't much info about this workflow is that it's relatively new (last few years) and it's only now that applications as finally syncing tangent basis through mikkt. Whereas the older method has been around as long as I can remember and is by far the most popular method simply because it's an entrenched workflow and was the only option (along with object space skewmesh conversion) for a looong on time.
And yeah I figured this workflow was quite modern.
Evidently it didn't work. I'm obviously not understanding this.
Your seams don't matter in the same sense as they do in the uv split workflow. They can be stitched or not.
All the info is in this thread. I've already explained the UVs factor earlier in the thread. Maybe you should stick with the cube test and just read back over my posts.This really is as simple a workflow as it sounds once you understand why you are doing certain steps in the process.
Okay now I understand this. I was blatantly thinking of something else when you said this. Overthinking it as usual. Visible seams can be stiched if they need to be. Having them split or stiched isn't required at all.
I looked back through the thread, I followed your steps one by one carefully, twice. I still can't spot the problem. I can see that it is a dead simple workflow.
It seems like a UE4 thing. Since if the normal map that is coming out of XNormal is how it's supposed to look (compared to the normal map you produced) then I obviously can't be doing anything wrong there.
Here is what I did with my UV's if that changes anything at all:
I'm afraid to say I'm out of ideas. I feel like I'm missing something blatantly stupid in UE4. I don't have much knowledge on UE4 in the first place so I can't put my finger on it.
I think I'll do a step by step video for you and put it on my YT channel as a request, if you'd like? Just give me a day or 2. Very busy with work atm.
As for the video. That would be amazing. As long as its cool with you and not taking too much time off of yours hands. It would be the star video to watch for people using this workflow. I would say it would be Polycount Wiki worthy.
Ensure light is coming from the Right in Red and light is coming from the Bottom in Green.
Red=Right (RR)
Green=Bottom (GB)
You can also do this in the texture in UE4, there is an option to do so in the texture setting. It says Flip Green Channel, check the box.
Check the picture in that link, its listed there
I've noticed that 3DReaper said:
Red=Right (RR)
Green=Bottom (GB)
This isn't the case with my NM:
Green is coming from the bottom but red from the left instead of the right. But I don't how that affects this.
Obviously now Flip Green Channel is checked.
Some edges seem to be rounded but some still sharp in correlation to the map.
No base colour or anything is applied to the mesh. Just a normal map.
None the less, here is my highpoly WITH and WITHOUT turbosmooth. I think the topology of it is fine. All loops are evenly spaced even though that doesn't matter:
Here's the steps I took for a simple cube:
Base mesh and it's HP. UV layout is the same as yours.
Bakes in Substance and Xnormal. Notice how the 3rd image is from xnormal and the green channel is different from the substance one. Middle one has it flipped so that both are the same.
Here's the settings from Substance. Pretty much as it comes out of the box. Same applies to xnormals. Only change in both is max rear distance to get the corners of the cube to bake nicely
And how it looks in UE4 with the base setup. You'll see I loaded 2 normal maps (the one from Xnormals and Substance). Both give very similar results, but I tend to get weird stuff in general with xnormals (its the one that looks like it has a grid in ue4) so what you see is with the substance bake applied.
As a note, I've barely used UE4 so far so there might be a better way to have it display. I quickly read the docs linked above about how to flip the green channel in UE4 and when I do I get a result similar to yours. So I reckon that's where your issue lies. So flip it in UE4 or do it in photoshop before hand.
This does sound like a seriously easy workflow but I honestly don't know where I'm going wrong.
EDIT: Okay so I've tried in SP but I'm getting another weird result. The normal map isn't very effective. Some edges are getting slightly rounder but same result.
My parameters are set exactly like yours. Compute tangent space per fragment is checked. My normal map format is OpenGL but changing it to DirectX weilds the same result. : /
This is very weird. I have noticed however that your normal map looks quite different to mine. Theres that blurring on the edges which means thats how it is supposed to be since its working as intended. My NM looks pretty sharp.
Here is the solo normal map view:
From the looks of it, your HP's bevels are either too sharp so they don't show or you exported it without the turbosmooth applied.
https://www.dropbox.com/sh/gcqz7iz8kjkcaob/AAAYHDG4QXrzuJgCE-2GOlmLa?dl=0
My HP's bevels can't be too sharp. If you look above I posted a pic of it. Looks very rround.
I still however get sharp corners OR is that just normal? Pun not intended. Look:
Normal Map:
Baked in substance
Any tips?
EDIT: I bumped up the map resolution from 512x512 to 2048x so it looks much better. But any other tips would be nice too. Or maybe this is just how its supposed to look and I'm just crazy. :P
But it looks great. Very close to the actual highpoly. But yeah any clarification on if I need to have by turbosmooth modifier baked down into the actual mesh would be good.
So when you export your HP from max if you do it as an FBX it will not bake the turbosmooth in even if the box is ticked in the options. Unless my max is bugged you can test it out using a box with a turbo applied on it export and import it and it will have disappeared. If you export HP's as OBJs however they will stick.
So either do OBJs so you keep the stack intact, or just clone your HP, bake the modifiers and export as FBX.
As you found out, the higher resolution you go for your textures, the better it will look.
As for the sharp corners, those are normal because what you see is your low poly which has those corners. If you look from certain angles the low poly will be visible no matter what, which is why it is often stressed that one of the important things in a low poly is keeping the silhouette intact. On my substance picture you can also see the sharp corners of the low poly.
This small problem went on for too long. I'm going to try to apply this convenient method and make some quick props to see how it goes. I've already created one. I've put no thought into this at all, just for testing purposes.
I can clearly see the smooth edges. The resolution for its map is 1024x if I remember correctly. Its realistic since not all projects will have 2k textures for assets.
One thing to remember though, i would recommend always clarifying your knowledge of normals maps and baking on polycount, because after just going through the process of learning this stuff myself over the last couple years, polycount itself tends to have the best info, especially thanks to Earthquake's sticky threads. All the info provided in those sticky threads by Earthquake are stuff you will have to learn and problems you will run into at some point in a 3D career. Just remember they're there is all so when you encounter problems i'd go straight there and check those. I didn't know about them when i was learning. But do watch some videos like the ones below, just know that tutorial makers on youtube tend to pass information as absolute in trying to be helpful and clear, but its not always accurate.