3ds max, Substance Designer, Marmoset and UDK workflow, someone please shoot me.
So, i actually ragequit the hangout earlier because i was getting so frustrated with this workflow between the above programs, and before i go blow up a minecraftworld with TNT i need to get some help with this.
As everyone knows, when you bake a normalmap in 3Ds max you need to flip the green channel when you want to use it in UDK or Marmoset. That is fair enough.
But, appareantly substance designer doesnt feel that way, so it requires an unflipped green channel from Max, making the workflow look something like this.
3ds -> flip green -> UDK
3ds -> Substance -> UDK
3ds -> flip green - Marmoset (tb1)
So, this doesnt make any sense to me, actually none at all. So for the current enviroment challenge i've resorted dropping marmoset completely for showcasing the WIP work, and just wait till i've got a more ready asset in UDK.
Is it possible that i've found a setting somewhere in substance that automatically flips the green for me?, if so, where is that setting, because i'd really like to just work with 1 set of normalmaps, instead of a different set for both UDK and Marmoset.
Cheers
As everyone knows, when you bake a normalmap in 3Ds max you need to flip the green channel when you want to use it in UDK or Marmoset. That is fair enough.
But, appareantly substance designer doesnt feel that way, so it requires an unflipped green channel from Max, making the workflow look something like this.
3ds -> flip green -> UDK
3ds -> Substance -> UDK
3ds -> flip green - Marmoset (tb1)
So, this doesnt make any sense to me, actually none at all. So for the current enviroment challenge i've resorted dropping marmoset completely for showcasing the WIP work, and just wait till i've got a more ready asset in UDK.
Is it possible that i've found a setting somewhere in substance that automatically flips the green for me?, if so, where is that setting, because i'd really like to just work with 1 set of normalmaps, instead of a different set for both UDK and Marmoset.
Cheers
Replies
1. Set tangent space to max on a per-object basis, this will make is so you don't need to invert anything
2. Set Flip Y in the normal map slot in the material editor
Not sure about the substance thing.
You can bake normalmaps in Substance?, i thought it was only world space from your regular normalmaps, as well as various other maptypes.
Now we're talking about substance designer.
i'm getting this error:
whenever i try to publish a substance with more than 2 4k maps in it. Is it just substance that isnt designed to work with 4k maps, or is it me who is doing something wrong?
For your error, maybe try switching engine to DirectX10? If you're on SSE it might be because of that.
Hmmm, this seems like pretty generalized advice. If the engine he's sending art to is synced to Max, that would be a bad idea.
For UDK, afaik, the best/only way to get synced normals with skeletal meshes is to bake object space, and then use handplane to convert. Explicit normals may work ok for static meshes though.
For TB2, its best to bake in Max or Maya, and set the per-mesh tangent space to the appropriate setting.
I personally have not used substance to bake, but I can't imagine it being as advanced as maya/max when it comes to setting up the cage, and certainly it would be preferable to bake in the same app where you set up your mesh normals as well.
I will try to run some tests with UE4 and skeletal meshes to see if we get something consistent too.
@Setes: Your error could come from different things. Make sure you don't have any missing dependency to your package.
Does it only happen when you have those 2 4k maps in there? Also are these PSD files? What happen if you choose jpeg compression instead of RAW for these 2 bitmaps?
I'll try out with jpg compression, and see if i can replicate it with a 2k texture.
As for the psd files, i import maps using tga, because; bad habits, and that substance designer cant handle 4k maps in a psd format. At least not on my computer
Edit; when set the export/publish options to 2k instead of 4k, it publishes just fine, no issues there. (found the compression settings, nvm)
Edit2: Hurraaah, it works!! Thank you so much for your help, really, i've been trying to fix this for the past day or so, thank you very much!
Interesting, it would be cool to see how the worst-case test mesh looks with this workflow.
https://dl.dropboxusercontent.com/u/499159/maxworstcase.zip
zip with obj containing high and low, exported from max, and NM baked in max, this is as good as synced gets in terms of a max bake with max tangents in TB2, so If you could send me a FBX out of max, and a NM bake from substance to compare that would be awesome.
An FBX exported out of Maya and baked in SD to test against our Maya tangent space would be great as well.
Low is offset by 3 btw.
I have no idea about UE4, just talking about UDK and the issues with normals on static vs rigged meshes. I would hope UE4 has improved in this regard but I don't have access to it.
WHen i'm about to publish i the substance parameters (the info window that comes up when you click the substance), and scale it up so it says "4096", then export it. In my head this makes total sense, though it appears that Substance designer has something else in mind, since when i import the substance into UDK the textures used are in a 2k resolution.
I honestly dont know what i'm breaking with this, or how i manage to break so many things
@Earthquake: I will give the mesh a go and see how it comes out.
I'm using the default Marmoset tangent space. It looks synced to me (and it is synced in the SD viewport too).
Here is the normal map: https://dl.dropboxusercontent.com/u/9804576/testnorm.png
Thanks for doing that, heres some quick shots.
A. Normal map baked in max, max tangent space selected
B. Normal map baked in SD, max tangent space selected
C. Normal map baked in SD, marmoset tangent space selected.
C looks better than B, but A is noticeably better than either. A is what I would consider synced. C being better than B makes sense as for the marmoset tangent space, as the tangents are loaded from the mesh for that one.
Actually, that might be the problem here, I am loading this from an OBJ (so no tangents) out of max, can you send me the exact FBX file you used?
Thanks, that file gives me basically the same results as the OBJ I had here. Using a Max bake and the Max TS gives noticeably better results. So I'll stick with my earlier statement, if you want the most accurate normal maps for Toolbag 2, a bake out of Max or Maya is really the way to go.
This may vary for other engines of course. If your engine isn't synced to a known tangent space like Max or Maya, baking with the explicit tangents/bi-normals from the fbx file and then using an engine that loads that data from the imported mesh is probably your best choice. I know later versions of UDK do this (but not for rigged meshes), and possibly UE4?
Some engines use a custom tangent basis, or standards such as Max or Maya that they calculate on load though, in which case you don't want to bake with the tangents/bi-normals imported with the mesh.
SD just bakes with whatever tangents are in the FBX file, so if marmoset doesn't use those then yeah no sync.
You might disagree but I firmly believe that replicating the tangent calculations of the exporter in the engine/final viewer, is not the best way to do things. If your baker and engine both use whatever is in the model file, then things will be synced. It's what we did at Splash Damage with a custom baker and it's what we do at DICE.
Though it appears that i'm messing something up still. The export to bitmaps from SD does indeed export it as a 4k now, but when i publish a substance it imports to UDK as 2k maps.
Those are the settings i've got. Is it possible that it might be UDK who's in error in this case?, anyway, the published substance is only some 370 kilobyts, which is somewhat small for 3 4k textures. If that is how it works.
Yes by default, using the "marmoset" tangent space, this is what we do, I could be wrong here and I'll double check but I'm like 95% sure we load the tangents directly from the file by default.
It could also be a case of this working well but not quite as well as a properly synced up system using the exact same tangent basis. The mesh I'm using is a worst case mesh and (hopefully) meshes used in production will not look like this.
No I totally agree, if that is what your engine does its best to use a baker that does the same, if you're not using a specific tangent basis (ie: the engine calculates the tangent basis on load for all meshes) then using a baker like SD, or converting in Handplane which also has an option to bake from explicit tangents is a good idea. Handplane is great in that you can quickly pop out a map from the tangents in the file, and another in a variety of other tangent spaces for use in other apps/engines.
Unless someone knows the tangent space of the engine the OP is using we're sort of going around in circles here though. From my tests I find that Max or Maya baked maps are the most accurate in TB2, that's all I can really conclusively say.
This doesn't mean you can't get good results in TB2 from maps baked in substance, on the contrary if you follow basic common sense rules in mesh construction (avoid extreme shading, use hard edges along your uv seams, etc) I bet the bakes will be totally acceptable in most cases.
There is two type of engines in Substance Designer : CPU or GPU (SSE2 vs DirectX).
CPU if a bit faster but can only handle 2K, while GPU can handle 4K.
The integrated system in the UDK only use the CPU engine because the GPU is not available (already used for the rendering of the engine). Therefore under UDK you are limited to 2K.
Unless you bake down the textures before importing them in UDK (so they become regular textures), you are limited to 2K indeed.