[SpeedTree][UnrealEngine] How do I export AO/Roughness/Metallic map?

BTDarmstad

Hi everyone! I'm very new to SpeedTree, but I thought I'd learn it for a personal project I'm working on in UE4. The title speaks for itself. When trying to export to UE4, I can't see any clear way to get the essential packed texture with AO, roughness and metallic information. I only get the color map, normal map, and a subsurface map. I can't find any information about it online either, and all tutorials I've found on it appear to be happy with color map and normal map and don't bother with anything else. Really? I also find it annoying how SpeedTree is using gloss instead of roughness, so I have to invert the map and hope it looks correct in the export.

How am I supposed to export the additional texture maps when exporting?

Alex_J

First, ensure you have the textures assigned in the material bar:

toolmaterialsassetsbar [SpeedTree Documentation]

Second, check how they are being packed:

expgame [SpeedTree Documentation]

I don't have it installed right now so can only point to docs. IIRC, you need to manually generate the AO by clicking the AO button up on the main toolbar.

It won't be generating a metalness map. I think by default the AO and glossiness get packed together, but I am usually only using color, opacity, and SSS.

BTDarmstad

Thanks, Alex_J, but I've looked everywhere in those menus, and all I can find that makes sense is which color channel I want to assign a texture to.

I cannot for the life of me find anything that resembles a menu or field where I specify which texture assigned to said channel goes to which material. They appear to just get discarded on export, and this is the UE4 preset. Still only color, normal, and subsurface. I must say this is way less intuitive than Substance Painter, where you can clearly see which channel goes where, but as far as I can see, there's no way to do it in SpeedTree. That really can't be the case.

The color map is also exported differently from what I'm used to. Instead of the usual RGB and opacity in alpha channel, the colors are translucent. I'm not even sure if it's supposed to be like that.

Alex_J

It's too bad I don't have a subscription to it at the moment. I might be renewing it soon in which case I can take a look. But I do remember messing with the way textures were packed before, so I think it is possible at least.

I took a look at the speedtreee textures in my ue4 project and I'm not sure what you mean about the color textures. They are packed to RGB+A with opacity in the alpha. If you drop that in photoshop or similar should be able to confirm that.

You might also try on the speedtree forums. In the past they've been helpful for me, though seems like a lot of unanswered questions there lately.

Benjammin

Speetree is packing its textures in a way that can be read automatically by speedtree materials inside Unreal.

Roughness/glossiness is in the alpha channel of the normal map. AO is baked into vertex color IIRC.

BTDarmstad

https://polycount.com/discussion/comment/2765805#Comment_2765805

Oh, you're right! I've never seen it done like that. Just a little disappointed I can't use the AO map to create more variation in the material with it, like adding moss or dirt in little cracks, etc. Maybe things like that have to be done in SpeedTree itself. Seems the rest of the maps are indeed in there. Thanks!

FALA

Hello, if anybody in the future wants some customization in shi*tty and limited export setup from SpeedTree:
You need to go 

{YOURS_SPEEDTREE_DIR}/texture_packing

Then you can select Unreal's UE4.fx and UE4.xml and duplicate that (rename also to fit yours packaging method name). Open it in some code editor (e.g. VS Code or Notepad++). 

In XML File: 

<div><?xml version="1.0" encoding="UTF-8"?></div><div><SpeedTreeTexturePacker Shader="PACKAGE_NAME.fx" ForceTextures="false" ></div><div>&nbsp; &nbsp; <Texture0 Suffix="_Color" MipGenerationFlag="AlphaIsOpacity" /></div><div>&nbsp; &nbsp; <Texture1 Suffix="_Normal" /></div><div>&nbsp; &nbsp; <Texture2 Suffix="_Subsurface" /></div><div>&nbsp; &nbsp; <Texture3 Suffix="_AO_Roughness_Height" /></div><div>&nbsp; &nbsp; <Texture4 Skip="true" /></div><div></SpeedTreeTexturePacker></div>

Here I just changed package method name and added one more texture to setup and updated Shader , which is name of your new preset. 

From:

<Texture3 Skip="true" />

To:

<Texture3 Suffix="_AO_Roughness_Height" />

This is just declaration and naming convention of your output file. I picked here something similar to Quixel's packing method. Actual packing is handled by second file - .fx

You need to add this somewhere in your code (.fx file):

sOutput.m_vTexture3 = float4(sInput.m_fAmbientOcclusion, 1 - sInput.m_fGloss, sInput.m_fHeight, 1);

Ofc. outside of if instructions 
Breakdown:

sOutput.m_vTexture3 - it's your output file. You just declared it in XML step, just like normal map is declared as sOutput.m_vTexture1.
float4 - you're creating RGBA image - 4 channels

sInput.m_fAmbientOcclusion - your AO value - we can put this in R channel
1 - sInput.m_fGloss - simple OneMinus operation on Gloss map, to create Roughness value. As is second argument it goes to G channel.
sInput.m_fHeight - I just picked Height for B channel.
1 - it's just alpha channel

You can select whatever float type you want as any of them (as I think, I can be wrong here).

Last step - select your preset while exporting

In my case, preset name is Gotha.
The result is Quixel like packed texture 

Sorry for my bad grammar, but I hope someone will use this right way in the future anyway. Here goes the docs: https://docs9.speedtree.com/modeler/doku.php?id=texturepacking - field names, flags etc.

Eric Chadwick

Thank you for posting the write up! Always great when people share what they’ve learned.