toolbag 4 - ray tracing render shows UV seams

Alex_J

when using the ray trace renderer in toolbag 4 I have obvious grey lines at UV seams that are not visible when using the raster renderer.

I tried ticking on and off all options within the ray trace renderer with no result. I also changed the renderer from GPU to the nvidia and back again with no result. (i have an nvidia GPU)

Any ideas? I can try editing the UV shells but I don't think that helps understand the problem at all.

(note, this is a watertight mesh.)

NhodgesVFX

I have this issue when I use any type of sss with raytracing, several people on the Facebook group do as well. It happens at uv seems or across mesh boundaries or at self intersections for me when using sss and raytracing.

Alex_J

Thanks @NhodgesVFX , I just tested and that is the case with me as well. Hopefully that narrows things down, does seem more like unintended bug versus user error.

Sreliata

Did you find any workaround for this, I wonder? I have the exact same issue and it's a pretty frustrating one, since I can't seem to fix it. Not even with baking all the textures (i.e: head, body etc) onto one ._.

Alex_J

What support told me was that Toolbag ray tracer sees each material as separate object. Converting your model to a single material should have solved your issue then (it did for me.)  

@Sreliata

Sreliata

So what they are saying is I'd have to completely redo the UV layout as well as the textures and cram it all into one single texture file.. for that to work? Really? 😰

Alex_J

yeah.

Sreliata

Oh my god.. I see. That's very unfortunate. I guess I'll have to make use of Photoshop to shop away the seams after rendering is done. Everything else is way too much of a re-doing of everything that has been done so far already. ;_;

Thank you so much for your response, Alex.

EarthQuake

Yes, as Alex mentions, this issue is due to the ray tracer treating each unique material section as a unique mesh. This is something we would like to resolve in the future, however, it is a difficult technical problem so we do not have an ETA on this at the moment.

The current workaround would be to create a duplicate version of your mesh with all UV sections that use SSS combined to a single UV set/material. So if you have a 4K UV set for the face and a 4K for the body, try combining those to a single 8x4K.

Another possible workaround would be to hide the seams behind sections of clothing, though of course, that is more involved.

Alex_J

for human character I just auto lay out the uv's and rebake from the original at double resolution. Takes ten minutes and result is the same except less materials to fuss with setting up. Like mentioned, I was using 4 x 4k textures, baked them to a single 8k texture. No resolution loss. 

@Sreliata

Sreliata

I'm honestly not too well versed with the baking into a new texture. I tried it earlier and the results were incredily washed out. What I did do then was to use photoshop to get all the different textures (Head, Body, Mid area) onto ONE image basically- and ten I pulled the UV sets accordingly, in blender. It's.. a little bit of a hassle but the only way I could make it work without much loss to it. Does any of you two have a good tutorial on how to bake textures from two separate mesh areas onto one big one, without any quality loss? I'd love to learn more about that, but didn't find anything that covers that in particular.

In any case: THANK YOU SO MUCH!! I didn't even expect to get a reply this quickly. 
Here is the result of the way I did it, to get rid of the green lines:

Alex_J

@Sreliata
You are doing it the very hard way. There is an easy way. 

You need the source model and the target model. Same as any other bake. 

The source is your model with multiple materials (aka texture sets.) The target is your model that has the UV's compacted and a single material ID assigned. 

Make your baker in Toolbag and  be sure to tick the option for multiple texture sets. Then you just bake as normal, except add teh options to transfer the albedo and roughness and whatever else maps you got. 

It may sound a little complicated at first but it's simple once you work through it one time. This is a powerful way to transfer maps between models when you got to change UV's (which you'll likely do a lot when making games.) It's really worth learning how to do. 

EarthQuake

Honestly, I would say setting up bakes to do this is the hard way. Scaling the UVs, re-exporting, then combining the maps Photoshop would take very little time with fewer overall steps.

But yeah you can bake the maps in Toolbag, what you would want to do is:

• Create a copy of your mesh with the new UV layout
• Export both the new and old meshes
• Add a bake project in Toolbag
• Load the old version in the high slot
• Load the new version in the low slot
• Set up your materials for the old versions with all the maps you need
• Configure your bake project to output those map types too
• Bake at whichever resolution you like

If you end up completely repacking the UVs and rotating some UV islands, baking would be your only option since the normal map needs to be translated (which Toolbag will do when baking). But going from 2 or 4 1:1 UV layouts to a 1:2 or bigger 1:1 wouldn't need repacking, you would just move/scale the UVs so they fit in the 0-1 area, which should only take a few seconds to do.

Sreliata

EarthQuake said:

Honestly, I would say setting up bakes to do this is the hard way. Scaling the UVs, re-exporting, then combining the maps Photoshop would take very little time with fewer overall steps.

But yeah you can bake the maps in Toolbag, what you would want to do is:

• Create a copy of your mesh with the new UV layout
• Export both the new and old meshes
• Add a bake project in Toolbag
• Load the old version in the high slot
• Load the new version in the low slot
• Set up your materials for the old versions with all the maps you need
• Configure your bake project to output those map types too
• Bake at whichever resolution you like

If you end up completely repacking the UVs and rotating some UV islands, baking would be your only option since the normal map needs to be translated (which Toolbag will do when baking). But going from 2 or 4 1:1 UV layouts to a 1:2 or bigger 1:1 wouldn't need repacking, you would just move/scale the UVs so they fit in the 0-1 area, which should only take a few seconds to do.

Since I have you here already, I have one last question in terms of 'UV's.

Is it possible to apply an AO (Ambient Occlusion /Occlusion) map to another UV layer? Say, I have the diffuse on UV layer 1, and the AO Map on UV Layer 2. There is no option to choose what UV Set a map should lay on. Is there a workaround for that? ._.

Edit: nevermind. I think it's because Marmoset only reas .obj or .fbx files and.. somehow blender doesn't seem to export UV layers with it. 

EarthQuake

Sreliata said:

EarthQuake said:

Honestly, I would say setting up bakes to do this is the hard way. Scaling the UVs, re-exporting, then combining the maps Photoshop would take very little time with fewer overall steps.

But yeah you can bake the maps in Toolbag, what you would want to do is:

• Create a copy of your mesh with the new UV layout
• Export both the new and old meshes
• Add a bake project in Toolbag
• Load the old version in the high slot
• Load the new version in the low slot
• Set up your materials for the old versions with all the maps you need
• Configure your bake project to output those map types too
• Bake at whichever resolution you like

If you end up completely repacking the UVs and rotating some UV islands, baking would be your only option since the normal map needs to be translated (which Toolbag will do when baking). But going from 2 or 4 1:1 UV layouts to a 1:2 or bigger 1:1 wouldn't need repacking, you would just move/scale the UVs so they fit in the 0-1 area, which should only take a few seconds to do.

Since I have you here already, I have one last question in terms of 'UV's.

Is it possible to apply an AO (Ambient Occlusion /Occlusion) map to another UV layer? Say, I have the diffuse on UV layer 1, and the AO Map on UV Layer 2. There is no option to choose what UV Set a map should lay on. Is there a workaround for that? ._.

Yes, in the Occlusion panel in the Material editor, there is a UV Set dropdown. Change this from UV 0 to 1 to use a secondary UV set for AO.

Sreliata

EarthQuake said:

Yes, in the Occlusion panel in the Material editor, there is a UV Set dropdown. Change this from UV 0 to 1 to use a secondary UV set for AO.

Yeah I just realized. Did you ever experience that this might not change anything at all? My character has a second UV layer on which the Ambient Occlusion lays. If I choose the 'second' set of UV's in the  UV Set option in Marmoset, nothing changes and the _O map layout remains completely scrambled. It's.. weird, somehow. I'd gladly send you the scene to look at if you'd want? 

EarthQuake

Sreliata said:

EarthQuake said:

Yes, in the Occlusion panel in the Material editor, there is a UV Set dropdown. Change this from UV 0 to 1 to use a secondary UV set for AO.

Yeah I just realized. Did you ever experience that this might not change anything at all? My character has a second UV layer on which the Ambient Occlusion lays. If I choose the 'second' set of UV's in the  UV Set option in Marmoset, nothing changes and the _O map layout remains completely scrambled. It's.. weird, somehow. I'd gladly send you the scene to look at if you'd want? 

Sorry for the delayed response. The AO slot only darkens the diffuse ambient lighting in Toolbag for physically accurate reasons. Depending on your lighting setup (ie: if you have a lot of direct lights) it may not appear to have much of an effect. If lighting the scene with only the sky light, it's generally more obvious.

Other renderers might multiply the AO over the direct lighting or even reflection passes, but this generally creates an unrealistic, dirty looking result. Imagine an AO map baked underneath your stove, and then shining a flashlight down there. If the AO multiplied on all the lighting passes, the flashlight would never be able to illuminate that area.

As a workaround, if you need to multiply AO data on direct diffuse/reflection passes, you can load the map via the cavity slot, though this is generally not recommended.

AO maps are typically not needed when using ray tracing, as ray tracing provides occlusion - not just occlusion, but bounced lighting, which is more accurate/realistic than a simple AO pass.