UE5 - Virtual Texture Streaming, does it improve performance ?

GlowingPotato

Hi, 
I'm currently developing a game in UE5, as the number of assets grows, new and unique textures also grows. 
I'm wondering if Virtual texture streaming will improve (decrease) VRAM use.

Reading the documentation, and googling my question, I found little to no information at all about the benefits (real world examples) of using VTS.

In my game, I have some 4k textures and lots of 2k and smaller textures.

Does anyone have experience in projects where all textures (or some) were converted to VTS and if it bring any improvements in VRAM use ? The documentation mention that it will have more instructions per texture sample, that can affect GPU time and shader compilation times...

Does anyone have information about it  ?

radiancef0rge

it brings extreme texture memory savings, only the necessary tiles of a texture can be loaded - and loaded a very particular mip level needed.  

it has nominal effect on gpu performance, its main downside is cpu overhead - which is relatively minimal and extremely scalable. 

virtual textures are specifically designed to enable large numbers of unique textures, and can reduce shader complexity if used correctly (by baking "shader features" back into the unique texture set)

GlowingPotato

@radiancef0rge Awesome to hear that! Thanks for sharing this information

GlowingPotato

@radiancef0rge I did a build using VTS and the gains in VRAM saving is indeed huge. it went from 8GB to 3.5GB ! during gameplay!
The increased time in GPU and CPU is almost non existent. 

Thanks a lot for pointing me to the right direction. 

poopipe

Soo.. the VRAM thing is slightly misleading although it is still a big win and a great idea to use RVT. 
The static memory footprint of the assets that render into the RVT doesn't change - not a huge problem, but worth knowing that stuff doesn't completely evaporate.
You need to load all the high mips / models etc. at the time the RVT is generated (on level load  / when forced)  - this means the machine doing it actually needs to be able to load all the things it needs. 


There is one other huge benefit to be had outside of the VRAM

Anything rendered into the RVT isn't visible at runtime so you can (for example) cover your landscape in little mesh decals (not projected decals) tell them to render into RVT (and be hidden as a result) and not pay the occlusion query or mesh rendering cost per frame.
This works extremely well - particularly on shit hardware like xbone and even switch  (you still have to store the meshes in memory )


and a note
The more stuff you shove into the RVT (and the more complicated the stuff), the more time needs to be spent on updates. This can manifest as spikes in your frame times and is generally only really a problem if you are moving/turning/teleporting the camera quickly but it's worth knowing that it's a thing in case you start to see stutters.  
the simple first solution is to reduce the number of actors getting rendered into the RVT - eg. by merging meshes 

GlowingPotato

poopipe said:

Soo.. the VRAM thing is slightly misleading although it is still a big win and a great idea to use RVT. 
The static memory footprint of the assets that render into the RVT doesn't change - not a huge problem, but worth knowing that stuff doesn't completely evaporate.
You need to load all the high mips / models etc. at the time the RVT is generated (on level load  / when forced)  - this means the machine doing it actually needs to be able to load all the things it needs. 


There is one other huge benefit to be had outside of the VRAM

Anything rendered into the RVT isn't visible at runtime so you can (for example) cover your landscape in little mesh decals (not projected decals) tell them to render into RVT (and be hidden as a result) and not pay the occlusion query or mesh rendering cost per frame.
This works extremely well - particularly on shit hardware like xbone and even switch  (you still have to store the meshes in memory )


and a note
The more stuff you shove into the RVT (and the more complicated the stuff), the more time needs to be spent on updates. This can manifest as spikes in your frame times and is generally only really a problem if you are moving/turning/teleporting the camera quickly but it's worth knowing that it's a thing in case you start to see stutters.  
the simple first solution is to reduce the number of actors getting rendered into the RVT - eg. by merging meshes 

All of this also applies to Virtual texture or just Runtime virtual texture ? Its two different things right ?

poopipe

The update stuff is purely for RVT.

SVT / VT are static so there are no updates at runtime 

GlowingPotato

poopipe said:

The update stuff is purely for RVT.

SVT / VT are static so there are no updates at runtime 

Got it! Thanks for the in depth explanation. I would love to know more if you have anything else to share.

poopipe

I guess there's a couple of other fairly important things to know

1: using SVTs (you can bake them from the RVT volume) is a very effective way of reducing the amount of resources required at runtime- they are static so will not include any runtime edits to the RVT and are used in place of the lower mips of the RVT (try 5 levels as a starting point)
2: rendering things that animate into the RVT is expensive - those fancy demos where RVT is used to do trails in snow are probably not realistic use cases for large scale environments (not saying you couldn't, but expect some pretty significant overheads)

Beyond that, they're textures.  You can use the channels for anything you want and you can render basically anything into them - the compression schemes limit you somewhat but 5.3 'should'  allow you to manage a lot more RVT assets than previous releases (haven't tested it yet)

radiancef0rge

was the original question about runtime virtual textures? 

poopipe

radiancef0rge said:

was the original question about runtime virtual textures? 

probably not but i went off on the tangent about RVTs cos a lot of people don't appreciate that there's a difference between the static and runtime systems in Unreal and get caught out. 

GlowingPotato

radiancef0rge said:

was the original question about runtime virtual textures? 

its about the non runtime variant, just Virtual texturing streaming.