Improving final render quality?

Elarionus

I'm just curious, as most of the stuff I've rendered out of IRay in Substance Painter seems to look better than my Marmoset renders. I've found things like Enable GI, and the different GI settings for specular and other things, as well as positioning lights manually and using the HDRI map built into Marmoset. I've gone through 5 or 6 tutorials that just detail things like that, mostly "how to put a light in the scene." A lot of regurgitation of the documentation (that's fun to say out loud). I was curious, in what methods will Marmoset really let me push my renders farther than IRay out of Substance Painter? I know it should be a lot better, and I want to make use of the Marmoset Viewer for sure on Artstation. But I want my stuff to look magical first! Thanks in advance!

EarthQuake

The primary differences between Toolbag and Iray are that Toolbag is a real-time renderer, more like a game engine where most of the rendering systems are optimized for speed rather than absolute quality.

Iray on the other hand, is a traditional ray tracing rendering (like Vray, Arnold, etc). So when it comes to ray traced reflections, shadows, etc, you can expect a traditional ray trace rendered to produce better quality than Toolbag.

Toolbag's main strengths are its ease of use and iteration speed. Generally speaking you'll get better results with Toolbag if you make good aesthetic choices in terms of lighting with the Sky and direct lights. Creating nice renders and presentations is more of an artistic endeavour than it is a matter of picking the best render quality settings.

Saying all that, there are a few things that you can do to improve render quality in Toolbag:
1. Firstly, Speedy Mode (the rocket ship icon in the viewport) is your friend. When this is enabled it turns off many expensive render effects to give you a very high frame rate. What this means is you can crank up all the settings to max, turn this on while you're making adjustments to lights etc, and then turn it back off when you want a full quality preview.
2. Some effects, like Local Reflections and Dithered transparency will look noisy in the viewport, but can be supersampled to reduce noise at render time. Go to Capture Settings (ctrl+P) and adjust the Sampling setting to control this. More samples = better anti-aliasing but slower render times.
3. With GI, to max the quality here you'll generally want to set Occlusion Detail to 4x, and Voxel Resolution to High. Again the Speedy Mode will help here, as Voxel Resolution: High may result in poor performance depending on scene complexity and how fast your GPU us. There are some practical limitations with GI, it uses a voxel system which means it creates a voxelized (simplified) version of the scene to ray trace against. This can result in somewhat blocky GI and reflections in certain cases.
4. Shadow quality can be improved by raising the Shadow Resolution. If you have a larger scene/environment, Directional light quality can be improved by enabling Use Cascades as well. Shadow maps are mapped to the entire scene when using Directional lights, so shadow quality can often be improved by using Spot lights instead, which cast shadows in a narrower range and thus have better localized resolution. Omni lights will have worse shadow resolution vs spot lights due to the expanded area in which shadows are casted.

Lastly, the Resolution setting in the Render Options can help improve the quality in the Viewport. Setting it to double will essentially super-sample the result. It's import to note that this will generally render about 4x slower, and that this setting does not effect final render quality, so this is only for viewport preview.

Elarionus

EarthQuake said:

The primary differences between Toolbag and Iray are that Toolbag is a real-time renderer, more like a game engine where most of the rendering systems are optimized for speed rather than absolute quality.

Iray on the other hand, is a traditional ray tracing rendering (like Vray, Arnold, etc). So when it comes to ray traced reflections, shadows, etc, you can expect a traditional ray trace rendered to produce better quality than Toolbag.

Toolbag's main strengths are its ease of use and iteration speed. Generally speaking you'll get better results with Toolbag if you make good aesthetic choices in terms of lighting with the Sky and direct lights. Creating nice renders and presentations is more of an artistic endeavour than it is a matter of picking the best render quality settings.

Saying all that, there are a few things that you can do to improve render quality in Toolbag:
1. Firstly, Speedy Mode (the rocket ship icon in the viewport) is your friend. When this is enabled it turns off many expensive render effects to give you a very high frame rate. What this means is you can crank up all the settings to max, turn this on while you're making adjustments to lights etc, and then turn it back off when you want a full quality preview.
2. Some effects, like Local Reflections and Dithered transparency will look noisy in the viewport, but can be supersampled to reduce noise at render time. Go to Capture Settings (ctrl+P) and adjust the Sampling setting to control this. More samples = better anti-aliasing but slower render times.
3. With GI, to max the quality here you'll generally want to set Occlusion Detail to 4x, and Voxel Resolution to High. Again the Speedy Mode will help here, as Voxel Resolution: High may result in poor performance depending on scene complexity and how fast your GPU us. There are some practical limitations with GI, it uses a voxel system which means it creates a voxelized (simplified) version of the scene to ray trace against. This can result in somewhat blocky GI and reflections in certain cases.
4. Shadow quality can be improved by raising the Shadow Resolution. If you have a larger scene/environment, Directional light quality can be improved by enabling Use Cascades as well. Shadow maps are mapped to the entire scene when using Directional lights, so shadow quality can often be improved by using Spot lights instead, which cast shadows in a narrower range and thus have better localized resolution. Omni lights will have worse shadow resolution vs spot lights due to the expanded area in which shadows are casted.

Lastly, the Resolution setting in the Render Options can help improve the quality in the Viewport. Setting it to double will essentially super-sample the result. It's import to note that this will generally render about 4x slower, and that this setting does not effect final render quality, so this is only for viewport preview.

Alright, I'll take your advice and apply it to my current projects, thank you! As for why I'm using Toolbag, it was the only one in my price range as a student. I did look at Octane, but it is quite expensive (though I do love the results), and of course Keyshot and VRay. I just always see some very impressive renders out of Marmoset, like the recent Shogunate Artstation challenge. Blows me away, and I always wonder why my renders don't turn out quite the same. ;P

Violet

That's nice advice I'll try out.
I'm a novice at MT, but I'm quickly learning and making use of it's quirks, especially the UV offset animation.

One of the things I've noticed is the realtime view in the editor and the view in the viewer can be massively different - my scene is simple, but it's using three skylights pinned on the environment map. On export, these provide no lighting or shadows to the viewer scene. Does the viewer compress or doing something other to the environment map once you export.?

EarthQuake

Violet said:

That's nice advice I'll try out.
I'm a novice at MT, but I'm quickly learning and making use of it's quirks, especially the UV offset animation.

One of the things I've noticed is the realtime view in the editor and the view in the viewer can be massively different - my scene is simple, but it's using three skylights pinned on the environment map. On export, these provide no lighting or shadows to the viewer scene. Does the viewer compress or doing something other to the environment map once you export.?

There is a limit to the amount of shadow casting lights in Viewer. Only 3 shadow casting lights are supported, so if you have more than 3, any lights after the 3rd will not cast shadows. If you have 3 or less there may be some other problem, and more information would be good to have.

Violet

The first viewer scene in this artstation page. It just has the cubemap lighting the scene with three spots on the map.
The buttons are emissives. Would be good to get this looking a little bit like the viewport, and I've probably missed something, but I'm still working on it. Thanks.

https://www.artstation.com/artwork/Bm3zOz