3ds Max Normal Map Errors?

Chrismartinartist

Can anyone explain why I keep getting this weird error on my normal map bake? It is a complex object, and they are diagonal in orientation but everything is within the cage. I can't find what is causing this and it's driving me nuts!

Eric Chadwick

Your texture baker is grabbing neighboring surfaces by accident. You need to adjust the ray distance or use a cage mesh.

But more importantly, you should not uv like this, it's extremely inefficient. Look into trim sheets to understand the workflow.

http://wiki.polycount.com/wiki/MultiTexture#Trim_Sheets

http://wiki.polycount.com/wiki/Modular_environments

Chrismartinartist

It is using a cage, there's nothing else within the cage. This is as an 8k TGA.

I don't see how a Trim Sheet is relevant to the UV layout, there are that many posts and they all need to be unwrapped, I don't want them all stacked on top of each other because it will be too obvious that there'll be texture repeats.

Chrismartinartist

Weirdly, I rebuilt the entire low poly, testing every so often still rendering as a TGA but doing 1k test renders, and I didn't get this. Last night I rendered the full thing as a PNG and got a strangely faded version but with no errors. So I re-rendered as a TGA and now the errors are back.

Eric Chadwick

Well, speaking from >20 years of experience making art for games, this is unequivocally the wrong way to UV an asset like this. 

UV shells don't need to be stacked exactly atop on another when using a trim sheet. If it's tiling along 1 axis, you can shift the shells left and right along the trim to hide the repetition. See my links for examples.

Another consideration... this doesn't look like a hero asset. It's a fence or frame, essentially a prop. So it should use less memory, and reuse materials amongst similar props.

If it was a character, or a FPS weapon, or something else constantly up front in the player's, then yes you would be expected to unwrap it (mostly) uniquely.

Also 8k is extreme, not used except in extreme cases. 2k is the largest you would be expected to go for a large prop, but more likely 512 or 1k.

Kanni3d

You can use a trim sheet and have variation (ie. trim sheet with 4-8 different wood grains etc)

Ghogiel

Since it's just bevels and not detail being baked down to the normal here, you could slap a chamfer and face weight them and not even use a large res, heavily gradient, ineffecient normal map and get the same sort of result.

I actually don't understand the UV layout. The shells I am looking at don't make total sense for a 4 sided box, I only see 3 sides of them in the UV?

And to reiterate the points above, texturing that many posts indiviually is probably giving everyone nightmares if it's for a game env. If it's not, you can disregard that to some extent. But you could probably bake out 10 of them uniquely and clone those around to construct the assets, rotate and flip them etc, and literally never see the same texture a single time.

Secondly, if you think this asset is that important to warrant an 8k and all uniquely unwrapped and textured you could sacrefice some extra verts and have a shader solution and use vert color to blend some textures and which could go a very very long way to hiding any tiling 

But the actual issue, assuming its not a bug, I honestly haven't baked normals in max for years since other bakers are just that much better and do mikkt etc,  I am not sure. It just looks like an intersection error and it's catching the back face of something nearby.

Chrismartinartist

Eric Chadwick said:

Well, speaking from >20 years of experience making art for games, this is unequivocally the wrong way to UV an asset like this. 

Also 8k is extreme, not used except in extreme cases. 2k is the largest you would be expected to go for a large prop, but more likely 512 or 1k.

Ok, so how should it be UV'd?

I don't see how else it can be UV'd especially as you're also saying use trim sheets, which to avoid repetition would involve spacing the UV's out even more along a repeating texture?

I'm using high res maps for 2 reasons:

I had a portfolio review and the guy said everything in my portfolio looked like it was designed for mobile. It wasn't specifically, but the course I did focused quite heavily on going low poly. For this project I said I'd do no smaller than 4k.

The second reason was something I heard in relation to trim sheets about if something is 1m in real world size you would use a 1k map. This piece of this project is huge, I was going to use a 16k map but 3ds max 2019 and RTT don't seem to go higher than 8k.

If you look here:

https://www.artstation.com/community/projects/J9WwZa/edit

You can see this piece is like an iron gurder structure around the concrete base of this rocket launch pad. It's a huge environment and I've been trying to plan this project such as that I can put it on Sketchfab, CGTrader, Turbosquid, Gumroad, and take renders from it in Unity.

The reason the UV layout doesn't look like 4 sides of a post is because one of the sides is facing into the concrete and not visible, so everything that can't be seen I UV'd separately and shrunk.

Chrismartinartist

Ghogiel said:

Since it's just bevels and not detail being baked down to the normal here, you could slap a chamfer and face weight them and not even use a large res, heavily gradient, ineffecient normal map and get the same sort of result.

I actually don't understand the UV layout. The shells I am looking at don't make total sense for a 4 sided box, I only see 3 sides of them in the UV?

And to reiterate the points above, texturing that many posts indiviually is probably giving everyone nightmares if it's for a game env. If it's not, you can disregard that to some extent. But you could probably bake out 10 of them uniquely and clone those around to construct the assets, rotate and flip them etc, and literally never see the same texture a single time.

Secondly, if you think this asset is that important to warrant an 8k and all uniquely unwrapped and textured you could sacrefice some extra verts and have a shader solution and use vert color to blend some textures and which could go a very very long way to hiding any tiling 

But the actual issue, assuming its not a bug, I honestly haven't baked normals in max for years since other bakers are just that much better and do mikkt etc,  I am not sure. It just looks like an intersection error and it's catching the back face of something nearby.

Chamfer and face weight? I don't understand that workflow, that would still 4x the geometry wouldn't it? Which is the reason we use normal maps?

The 4th side is hidden by other objects in the scene so I UV'd it seperately and gave it less space. I know if it's not seen then why model it but I was also under the impression that making things 'watertight' is better than leaving uncapped holes?

Eric Chadwick

You link seems to be broken.

The first tutorial here should answer many of your questions
http://wiki.polycount.com/wiki/Modular_environments

For really large models, the key is to use trim sheets rather than making the texture maps gigantic. You can also add detail textures, but the downside to that is it's handled differently in different engines/viewers.

Eric Chadwick

Oh also, chamfers and face-weighted normals:
http://wiki.polycount.com/wiki/Face_weighted_normals
http://wiki.polycount.com/wiki/VertexNormal#3ds_Max_Tools

Eric Chadwick

Just for fun, check out the kind of quality that can be achieved with a single 512x256 trim texture.
https://polycount.com/discussion/89682/an-exercise-in-modular-textures-scifi-lab-udk/p1

This is rather extreme, as you wouldn't apply such heavy stress on the shaders and UVs in a real production situation, but it does prove that trim sheet workflow is indeed a powerful trick to have up your sleeve.

Chrismartinartist

Try this:

https://www.artstation.com/artwork/J9WwZa

Chrismartinartist

Eric Chadwick said:

You link seems to be broken.

The first tutorial here should answer many of your questions
http://wiki.polycount.com/wiki/Modular_environments

For really large models, the key is to use trim sheets rather than making the texture maps gigantic. You can also add detail textures, but the downside to that is it's handled differently in different engines/viewers.

This confuses me even more, how can I apply any of that to what I'm doing? He's essentially kit bashing a model from parts. I already have a model I'm just trying to figure out why the normal map is coming up with a strange shading pattern.

Chrismartinartist

Eric Chadwick said:

Oh also, chamfers and face-weighted normals:
http://wiki.polycount.com/wiki/Face_weighted_normals
http://wiki.polycount.com/wiki/VertexNormal#3ds_Max_Tools

So the issue is I haven't weighted the normals of the high poly before baking? I didn't with any other piece in the scene though?

Chrismartinartist

Eric Chadwick said:

Just for fun, check out the kind of quality that can be achieved with a single 512x256 trim texture.
https://polycount.com/discussion/89682/an-exercise-in-modular-textures-scifi-lab-udk/p1

This is rather extreme, as you wouldn't apply such heavy stress on the shaders and UVs in a real production situation, but it does prove that trim sheet workflow is indeed a powerful trick to have up your sleeve.

It is impressive. I don't understand most of what he did, or in what program.

I have used Trim sheets in other projects, but to be honest I didn't enjoy using them, I found I had obvious repeats, and it was fiddley lining up UV's with specific parts of the texture.

Chrismartinartist

This is the result I'm trying to get, with the nice soft round edges. But without the weird black oval outline on the inside of the cross posts.

Not all of the pieces are the same size.

Eric Chadwick

Nope, you bevel & adjust normals on the in-game mesh, instead of using a baked normal map.

Eric Chadwick

Chrismartinartist said:

It is impressive. I don't understand most of what he did, or in what program.

I have used Trim sheets in other projects, but to be honest I didn't enjoy using them, I found I had obvious repeats, and it was fiddley lining up UV's with specific parts of the texture.

If you read the OP's replies, he explains how he does it, even with wireframes & UVs. 

The trick with repeating geometry like your girders is to UV and texture as you go, not wait until the end. UV a single cross piece, then duplicate it to make the full structure.

To get variation, plan into it with variations in the trim sheet, as mentioned earlier. Then simply build a few sections of girders, offset the UVs for variety, clone those larger chunks around, rinse and repeat.

Ghogiel

Chrismartinartist said:

Chamfer and face weight? I don't understand that workflow, that would still 4x the geometry wouldn't it? Which is the reason we use normal maps?

Generally I wouldn't smooth shade boxes with 90 degree angles and then use a normal map to correct the shading as a general rule. Even with todays synced tangent workflows, compression, mipping etc still isn't nice for deep gradients in normal maps that span across large surfaces. So usually in similar circumstances you don't increase the vert count all that much by chamfering because of using hard edges to avoid the normal map doing such heavy lifting to correct the shading.

But if I was adding 8k maps to a low poly fence, and those choices were all based on being optimised, I'd be less worried about upping the tri count if it meant using smaller textures. Anyways, chamfering and faceweighting is just one of many options.

Chrismartinartist said:

The 4th side is hidden by other objects in the scene so I UV'd it seperately and gave it less space. I know if it's not seen then why model it but I was also under the impression that making things 'watertight' is better than leaving uncapped holes?

There often isn't a reason to leave hidden faces in there unless you can't be bothered to delete them or just haven't gotten to the stage of wanting or needing to optimise vert counts in a scene. There might be a specific reason to do so on the odd occasion, but generally if you can't see it, it's probably a waste but unless you are trying to optimise or someone is yelling at you to sort them out you can just leave them be.

Chrismartinartist

I don't know what it is with this shape, even remodelling just one piece is giving that same error, so it's nothing to do with number of UV elements. So can't even do it like a trim and duplicate pieces.

Eric Chadwick

Quick example with face-weighted normals, hope it helps!