Witcher 3 Architectural Material in Unreal 4

icegodofhungary

I spent two days trying to re-create this method in Unreal.

https://polycount.com/discussion/174377/witcher-3-blood-and-wine-architectural-material/p1

First I used an elaborate IF node tree and then I discovered the 2D Array Lookup node. I thought it would take the texture's UVs, split it into a given number, and then basically turn each square section into a 0,1 UV space. Then that would let me have an array of textures I could select. So I tried it with a simple 1024x4096 texture.

Also attached is the copy/paste from the UE4 material editor. I thought I cracked it but when I tried a real texture:

I'm now  assuming the 2D array node just looks at one specific point at the UV coordinates. I tried looking it up but I keep getting stuff about blueprints that don't have anything to do with this specific node. Any other nodes I should look at?

I know I could do this without a texture array, I just hoped to be able to use them.

icegodofhungary

I figured it out for the most part. Could still use some work on the blending procedure. The masks don't quite work the way I want them to but it's good enough for now. I want a height blend effect on things lower than the brick surface but also I want plaster to cover the bricks at a certain point. And where there's a brick partially covered, I want a realistic transition rather than a gradient.

Anyways, I made it into a MF so I can re-use it. It takes a single vertex color, 0 to 1, splits the range up by the number of materials you have (expressed as MAs) and blends with an opacity based on your position within the range. If your VC color is 0.15 and you have 4 MAs, then it selects the first (bottom layer) MA and blends it at 0.6 opacity with the next layer. It also has masks so you can get a more natural blends. And you still have two (or three) whole VC channels to use for other stuff!

KrashDer

good work ! 

lele83

icegodofhungary said:

I figured it out for the most part. Could still use some work on the blending procedure. The masks don't quite work the way I want them to but it's good enough for now. I want a height blend effect on things lower than the brick surface but also I want plaster to cover the bricks at a certain point. And where there's a brick partially covered, I want a realistic transition rather than a gradient.

Anyways, I made it into a MF so I can re-use it. It takes a single vertex color, 0 to 1, splits the range up by the number of materials you have (expressed as MAs) and blends with an opacity based on your position within the range. If your VC color is 0.15 and you have 4 MAs, then it selects the first (bottom layer) MA and blends it at 0.6 opacity with the next layer. It also has masks so you can get a more natural blends. And you still have two (or three) whole VC channels to use for other stuff!

@icegodofhungary
I'm trying to do the same, I've found this (https://www.youtube.com/watch?v=fuvDy4lvGOM) but it's an old custom engine modification, in 4.26 it's not working (I've already tried ) and on 80lvl a guy basically tried to approach it without creating a tex2Darray but using a mask.
On UE4 forums people is still asking about it and apparently, it has got still some limitations.

I'd wanted to ask you if you did any progress? 

Thanks for sharing, it's nice not to feel alone

icegodofhungary

I came across that video as well. I haven't changed anything since the material function I posted. I'm probably going to wait to improve/tweak it until I'm done with my current project. It works well enough for what I need it to right now even if it's a little unwieldy. I have some ideas to make it better but I just need to finish this monthly challenge thing soon.

The array stuff would make file handling easier but I can live without it tbh. The real prize for me is attributing several blends to one VC channel and then using the blending to create variety so I can get away with smaller tiling textures. Being able to do those two things helps so much with efficiency. If I can sort out the blending to be a little better then I will be happy even if I never get the array part down.

poopipe

texture arrays can be enabled on newer versions of the engine - you have to put some words in a config file or something iirc 

A_B

This is really cool - I might be able to use something like this for a terrain texture issue I'm having - did you get the masks between materials to work ok? Could you share a little more info about your setup and how it works? I'd like to try some fiddling myself.

icegodofhungary

@A_B Sorry for taking so long to respond. If haven't messed with this in some time because I was doing other things.

As far as purely getting better vertex painting results (minus everything else I did here) there's this video which walks you through everything pretty well:

https://www.youtube.com/watch?v=OPHnxGb6KHA

That would be pretty useful for terrain and for architectural materials on its on. And that's what I was trying to do in addition to picking the texture by value input. I haven't actually fully done that yet.

As far as my setup, you have to enable Material Functions in unreal. I'm not sure how familiar you are with Material Attributes and Material Functions, but knowing what those are is necessary.

https://docs.unrealengine.com/en-US/RenderingAndGraphics/Materials/Functions/index.html

https://docs.unrealengine.com/en-US/RenderingAndGraphics/Materials/MaterialLayers/index.html

I use material attributes just so the graphs are as clear as possible. I pack it into a Material function for modularity. That way if I wanted to do another material for the floor I don't have to rebuild the graph again. You can download the text file I previously attached, copy/paste it into your unreal material editor. But as a material function, which you have to enable in Project Settings, iirc. Just open project settings and search for "function" you'll see the option. Create a new material function (Add/Import > Materials & Textures > Material Function) and then paste the contents of the text file into that window. Then you save the material function as something like MF_vertexblend or whatever, then you can drag that material function into your wall/floor/ceiling/terrain/etc master material. To do that, create a typical material, add in your texture files, then hit the "use material attributes" setting on your material. Plug your textures into a "make material attributes" node. Once you have that you can plug your make material attributes into the given material function inputs. On the node, next to each input is the type of input it requires (MA for material attribute, S for scalar, T for texture, etc). I think I commented each input too, so if you hold your mouse cursor over each input, a tool tip will tell you what each one does.

Here is a poor screenshot of the unreal graph of the material function, I can't get a good zoom level to see everything.

Here is how it looks in a typical material

The ones with (MA) beside it take Material Attributes. The S take scalars. VC takes a single vertex color channel. The masks take a single color channel of a mask.

The general idea here is that you would take in 4 materials. The red vertex color channel picks which texture you're painting. It does this by splitting the total range of possible values (0 - 1) into 4 parts. So you get points where a material is at 100% opacity, and any materials below that are blended smoothly. You have 4 materials. You could add more by adding more to the chain inside the Material Function. If Unreal had proper implementation of arrays, then you could just have parameter that would set the amount of materials. Though you don't want to go above 16 because of the math. The difference in red vertex color values would be too small between textures and it would be harder to use without just typing in values. The 0 - 1 range that unreal gives corresponds to 8 bit black and white values. 128 values per channel. 128/4 = 32. So each material is afforded *roughly* 32 values where it is either 100% opacity or blended with other materials. I say roughly because you count from 0 so 0 - 32 is actually 33 values. I think I adjusted for that in the math but the last material (top or bottom can't remember) gets an extra value. If you were to do 16 materials then that means the step in values is only 8. Trying to squint and pick out exactly 8 points in the value slider is a pain. You'd have to type it in.

So once you pick your material, or some blend of two materials, depending on what shade you're painting in with your vertex paint tool, this also blends them against a mask. I used 3 separate masks for reasons I don't now understand. I think my thought was to do cavities, edges, and then a height. But that doesn't make sense right now. Anyways.

The goal would be to combine the vertex blending techniques from the video into this. It probably wouldn't be too hard. I just didn't develop the blending part very much. I would have to play with it to tell you what and where to plug better blending technique in. I think the part I couldn't get my head around conceptually is that the red drives which materials you use, but it also does a little bit of blending itself because "picking" a material in this case just means having its opacity be between 0% and 100% based on the math I described above. So you have to blend there but it would also be good to have another blend using the green vertex channel. The red would be blending between layers, think plaster on top and brick on bottom. The green would then alter the shape of where the plaster is present and where it isn't. So instead of getting a straight plaster against brick, you get brick, damaged plaster, chipped plaster, and then clean plaster. More of a detailed look.

If the arrays in unreal worked like they do in the example of Witcher 3, then it's a little different because you generate a index value for selecting the material/texture before any blending takes place. red only picks the material, it doesn't involve any opacity or blending. Then your green channel completely drives blending. Again, this is hard to do unreal because the only method I found for making an array was virtually stack the materials as layers and then change the opacity to select between them. Though thinking about it right now I'm thinking I over complicated it. You can still do that. Just that I'm doing blending in the wrong place.

Have your 4 material attributes, have them going into the same if-statement nodes. Anything between 0- 31 is material 1, 32 - 63, material 2, etc. Except this time make it so that anything within those ranges returns the material at 100% opacity, no blending at all. Then you would plug those results into a different set of if-statements for the green channel to do the actual blending, implementing all the stuff from the above video. Therefore when you do your actual vertex painting you're actually painting both green and red at once. Red to pick the material, and green to tell it which value to blend. I think that would work better.

Feel free to experiment with the node I provided. If I fix it up I'll post an update.

Frigus

Great topic awesome work.

frmdbl

Looks cool but you have to prepack the textures right?

melviso

poopipe said:

texture arrays can be enabled on newer versions of the engine - you have to put some words in a config file or something iirc 

Do you mean texture arrays in one texture like The Witcher workflow?

 lele83 said:

on 80lvl a guy basically tried to approach it without creating a tex2Darray but using a mask.

Is this the one you are talking about:
https://80.lv/articles/ue4-gradient-array-mask-material/

Would be nice to have like 8 or even 16 materials in one. Is there a reason why ue4 or ue5 hasn't implemented this fully in the last 5 years?

icegodofhungary

frmdbl said:

Looks cool but you have to prepack the textures right?

If you mean channel packing, you can. If you mean pack the textures into an array, no, I couldn't get that to work. I simply used channel packed textures.

The tutorial @melviso posted from 80.lv is pretty good. There's a GDC talk on blending materials in painter which explores this as it applies to substance and unreal. I didn't think about that at the time I made this, I was trying to do it all in-engine. But doing it in Painter would certainly be easier and have more painting control.

melviso

I still think there should be a way to do this in unreal engine especially with an array. The previous possibilities have been with modifications of the engine. I am curious why the Epic devs haven't looked into implementing it.

Doing it in ue4 I think makes things faster and smoother. 

lele83

melviso said:

poopipe said:

texture arrays can be enabled on newer versions of the engine - you have to put some words in a config file or something iirc 

Do you mean texture arrays in one texture like The Witcher workflow?

 lele83 said:

icegodofhungary said:

 and on 80lvl a guy basically tried to approach it without creating a tex2Darray but using a mask.

On UE4 forums people is still asking about it and apparently, it has got still some limitations.

Is this the one you are talking about:
https://80.lv/articles/ue4-gradient-array-mask-material/

Would be nice to have like 8 or even 16 materials in one. Is there a reason why ue4 or ue5 hasn't implemented this fully in the last 5 years?

I’ve managed to create a texture array and make it work, unfortunately Unreal4 doesn’t stream such type of textures, so basically no mipmaps.
What Eric Zimmer did is different from what CD project has done in The Witcher 3.

A good example is what Arvin Villapando did, which is pretty much mine same approach.
https://www.artstation.com/artwork/PoJqmB

I believe Epic didn’t do any improvement to implement texture array because they are focusing more on Virtual texture, and I think you can’t have a Virtual texture array, but this is just a guess, I’m not sure if it’s possible, probably yes XD.

melviso

@lele83 Thats good to know.

The one on github:
https://github.com/fracturedbyte/UE4-T2DA
I understand this was a modified version of ue4. I am guessing it supported mipmaps. Without mipmaps, it is going to have a bad look in the distance and might affect performance since its using the full texture size so I think I get why @icegodofhungary implemented it this way..

Virtual texture array would be nice. 

melviso

@lele83 How about having two array textures and have a distance blend between them? use a smaller and blurred version at a distance and the higher and full size upclose?

melviso

Hmm.. tried using mipmaps with 2d texture array:


Ignore the texture, just a placeholder. Tweaking the mipmap value with a scalar parameter in a material instance. Seems to work. I am using ue5 btw.

Changed the 2d array texture settings- Mip Gen settings to - From No mipmap to Texture Group and it is streamed.

TECHNOBOG

@melviso Thank you for posting info in this thread. I'm always pleasantly surprised by people willing to contribute their findings like that.
In the end, are you saying that you got everything working?
Albeit with the need to control mips by camera distance in the shader.

I'm on UE5EA as well and I'm using a lot of tiling textures and I'm wondering whether I can consolidate a bunch of materials into one using Texture Arrays to improve performance.

lele83

@melviso  I remember You can call mipmaps inside your shader (you can do that in UE4 too)
the only downside is not really performance-wise, but I guess if you are using this approach for a shader that is going to cover your entire environment, maybe it would be affordable.
@TECHNOBOG  Drive your MIPS by camera distance is an interesting approach but it may get even more expensive.

Honestly, from UE5 I believe RVT is the best way.

This is a Tutor, and I work with him here for the MA in games, have a look, really good stuff and what he shows is game dev focus. 
https://www.youtube.com/watch?v=XW6-SVGS22M

TECHNOBOG

@lele83 I'm not sure I understand what you suggest

How would you use RVT to do texture lookups based on an index? In my understanding, RVT is basically a fancy and performant render target.

The issue is in having many (potentially dozens) of different elements, all seperately unwrapped onto a tiling texture (meaning the UVs are overlapping and are outside of the 0-1 space) makes it necessary to use multiple materials. With the method outlined in the post about the architectural workflow from Witcher as well the fracturedbyte UE4 modification videos linked above, you could use a single material (and therefore induce a single drawcall instead of multiple)