Hi, I am trying to render in high quality but it gives me this error message``Out of memory when trying to allocate trace backend frame storage¨ and I cant do it
The error you're seeing here is specific to the RTX Accelerated ray tracing back end, which means your 3070 is likely being detected and used.
Toolbag 4 requires additional memory when compared to Toolbag 3, especially when using ray tracing. So if you were able to fit your scene into video memory with Toolbag 3, that doesn't necessarily mean you will be able to with Toolbag 4. All assets in your scene must fit into video memory with some additional memory free for the ray tracing system to function. Some things you can try:
Close all other applications when working with complex scenes in Toolbag, especially other content creation applications, but even web browsers can take up a significant amount of VRAM if left open with many tabs for an extended period
Reduce the resolution and bit-depth of textures if possible. For instance, an 8K 16-bit image requires 8x as much memory as a 4K 8-bit image.
If your scene has dense meshes (ie: millions of triangles) consider decimating them
Remove any unused meshes or textures, all objects in the scene are stored in video memory (even hidden objects or materials that are not applied to meshes)
If you've set your viewport resolution to High DPI, try setting it to standard instead. Using a smaller area for the ray traced viewport can help to manage memory as well, for instance using the setup workspace and only having one of the 4 viewports set to full quality
Set the ray tracing engine to the Generic mode, you can do this in Edit > Preferences > GPU (this will require a reset). This will result in more efficient VRAM use, but slower ray tracing performance on RTX cards. We have some work-in-progress features that should reduce memory use in the RTX mode in the future
Generally speaking, the VRAM counter can be a little unreliable. Unfortunately, the operating system/video drivers don't report VRAM information very well. What this number shows is how much of the VRAM budget is currently in use, this budget is assigned by the operating system and can vary depending on what other applications you have open. You can check VRAM usage in Windows Task Manager as well, that number isn't necessarily accurate either but may give you a better idea of overall VRAM usage.
Replies
Toolbag 4 requires additional memory when compared to Toolbag 3, especially when using ray tracing. So if you were able to fit your scene into video memory with Toolbag 3, that doesn't necessarily mean you will be able to with Toolbag 4. All assets in your scene must fit into video memory with some additional memory free for the ray tracing system to function. Some things you can try:
Generally speaking, the VRAM counter can be a little unreliable. Unfortunately, the operating system/video drivers don't report VRAM information very well. What this number shows is how much of the VRAM budget is currently in use, this budget is assigned by the operating system and can vary depending on what other applications you have open. You can check VRAM usage in Windows Task Manager as well, that number isn't necessarily accurate either but may give you a better idea of overall VRAM usage.