Smoothing Groups and Baked Normals

62_5MHz

Hi everyone. Could you shed some light on my issue with this model?  

This is the first model I started with the hi-poly version then did a retopo for the low-poly version.
I am having a hard time with the normals. The smoothing groups are set to UV shells using Textools in 3dsMax.
When on SP viewport everything seems to be fine, but when rendering with iray the surface seems to 'sink' on some areas.


SP viewport: everything looks fine


Iray render: 'sunken' areas


UVs

Kanni3d

Looks fine really - which sunken areas are you talking about exactly?  Nevermind, I see the middle part between the two buttons is inverted. Post your high poly screenshots, along with your baked normal map?

FourtyNights

This is a known tangent space issue in Iray, and one of the SP's developers confirms it:

62_5MHz

Kanni3d said:

Looks fine really - which sunken areas are you talking about exactly?  Nevermind, I see the middle part between the two buttons is inverted. Post your high poly screenshots, along with your baked normal map?

Thanks for the reply Kanni3d.
As for the 'sunken' parts, now I pointed them out in the first post.

Here it goes:

high poly on SP viewport and iray's


Baked normal map


3ds Max render of the low poly

Kanni3d

Yeah, as suspected everything baked and looks fine - but these gradients are probably the culprit. It looks fine in painter, since it probably handles the underlying smoothing groups correctly to compensate for the gradienting, but in Iray, the gradient causes the sunken/indent effect.



To fix that gradienting, check out what your lowpoly is doing vs your highpoly. From what I see without a wireframe, I think you've got this entire part smoothed over, while your highpoly has this detail flattened. So therefore, your normal map is trying to compensate via gradienting it out. Maybe break this part off of your uv shell, and give it a seperate smoothing group, and rebake.



You can always troubleshoot this by exporting it and checking it in yet another renderer, like Marmoset, or an engine like Unreal. 

FourtyNights

Did you guys even read what I posted before? FCORREAR's normals are fine, there's no problem there. As you can see, they work perfectly on the SP's viewport, because it's synced with mikkt space. Iray isn't, and that's the issue. Those baked normals will work on UE4, Unity, MT3, etc. so forget about the Iray, until Allegorithmic themselves fix it first.

Don't try to fix something which isn't broken, lol.

Kanni3d

FourtyNights said:

Did you guys even read what I posted before? FCORREAR's normals are fine, there's no problem there. As you can see, they work perfectly on the SP's viewport, because it's synced with mikkt space. Iray isn't, and that's the issue. Those baked normals will work on UE4, Unity, MT3, etc. so forget about the Iray, until Allegorithmic themselves fix it first.

Don't try to fix something which isn't broken, lol.

Yeah. But if he really wants to mitigate issues, or really insists on an offline renderer like iray for reasons, he can simply give  the  steps above a try to see if it'll work. The only problem area in the iray render seems to be the gradient baked area.


Doesn't hurt to give things a shot, instead of simply saying 'nothing u can do it's bugged'.

62_5MHz

Kanni3d said:

FourtyNights said:

Did you guys even read what I posted before? FCORREAR's normals are fine, there's no problem there. As you can see, they work perfectly on the SP's viewport, because it's synced with mikkt space. Iray isn't, and that's the issue. Those baked normals will work on UE4, Unity, MT3, etc. so forget about the Iray, until Allegorithmic themselves fix it first.

Don't try to fix something which isn't broken, lol.

Yeah. But if he really wants to mitigate issues, or really insists on an offline renderer like iray for reasons, he can simply give  the  steps above a try to see if it'll work. The only problem area in the iray render seems to be the gradient baked area.


Doesn't hurt to give things a shot, instead of simply saying 'nothing u can do it's bugged'.

Thanks for pointing that out, FourtyNights. That problem seems to be the case, but since I was not sure about my model's quality (and blindly relied on SP) I tried to fix it first.
I just downloaded a Marmoset trial, checked the first model and the one with the new UV layout on the affected areas.

The first model on Marmoset:


The model with updated UVs:
- The left pic is from Marmoset. Everything seems fine.
- The right one is from SP's Iray. The new UV layout helped with most of the 'sunken' areas, but Iray also highlights their seams.



New UV layout:


Thank you everyone for helping me out.
Sadly, my SP license is for 2018 versions, so no 2019 possible fixes for me at the moment $$. LoL

Would you recommend a renderer I could use besides Marmoset?

toxicsludge77

Would you recommend a renderer I could use besides Marmoset?

UE4?

FourtyNights

@FCORREAR

Well, making changes to your UVs may ease and relief the "sunken details" a bit, but that's just mostly like patching up with a duct tape and saying "that'll hold, I guess", if hoping to render in Iray, 'cos its tangent basis is unknown, and there are no bakers synced to it. You can trust SP's viewport, Marmoset or UE4 though, those are telling the true quality about your normal map of your model.

Why not using Marmoset though, or do you wish to use an offline renderer instead? You could try Blender's Cycles. It also supports mikkt space for normal maps as well.

I'm not familiar with other offline renderers, since I'm all in for real-time rendering only, but I think Arnold, or V-Ray (Maya, 3ds Max) may be other ones you can try (they may support mikkt space, not sure though)... if you're not familiar with Blender.

UE4 also has either real-time raytracing or path tracing as options on top of its traditional real-time rendering you could also try out.

Just for the sake of the clarity, this Iray issue is happening for everyone:

https://polycount.com/discussion/212358/issue-with-i-ray-in-substance-painter#latest