No More Distortions With Floaters - Flattened Floaters (3DS Max)
polycounter lvl 6
Hi.
This is a method I came up with and originally shared on chamferzone, but since I can't see any evidence on someone using it before me I feel I'd share it to a wider audience.
The concept is that when you're creating a highpoly to be baked later onto your lowpoly, if you use floaters they'll often get distortion because of averaged normals. This isn't avoidable, not without extra geometry or masking multiple sets of normals. This method on the other hand is fast and prevents the distortion in a regular, averaged normal bake.
1. Take your highpoly floater and add a 'edit normals' modifier. Select all normals and press the 'Make explicit' button. All your normals should now be green when you deselect them.
2. Add an edit mesh modifier, go into polygon/element mode and use the scale tool to flatten your floater. The floater should appear the same even when completely flat because the vertex normals are preserved.
3. Bring the floater as close as you can to the lowpoly surface, try to not intersect your highpoly object. The closer you can get it to the lowpoly surface, the less distortion will be noticeable.
Vertex normals are fragile, and most modifiers will break or reset them but the bend modifier is an exception. This means this method works best for flat surfaces, and surfaces with consistent curvature in which case you can use bend.
If you need the floater to be deformed on some kind of complex surface, you can use another option which is saving and restoring vertex normals. This script was originally shared to me here on polycount, I take no credit for it, but since the link is dead I've reuploaded it. It stores the normals using UV channel 5, so there's also a script I made that clears UV channels greater than 1 if you ever need to do so.
https://mega.nz/#F!VKImRC5T!o8iDXWj5WYHGooxlecFh0w
In this case you can store normals once you have your floater done and subdivided, you can then deform it and restore the normals whenever you're done flattening or deforming to get it onto the lowpoly surface.
This is a method I came up with and originally shared on chamferzone, but since I can't see any evidence on someone using it before me I feel I'd share it to a wider audience.
The concept is that when you're creating a highpoly to be baked later onto your lowpoly, if you use floaters they'll often get distortion because of averaged normals. This isn't avoidable, not without extra geometry or masking multiple sets of normals. This method on the other hand is fast and prevents the distortion in a regular, averaged normal bake.
1. Take your highpoly floater and add a 'edit normals' modifier. Select all normals and press the 'Make explicit' button. All your normals should now be green when you deselect them.
2. Add an edit mesh modifier, go into polygon/element mode and use the scale tool to flatten your floater. The floater should appear the same even when completely flat because the vertex normals are preserved.
3. Bring the floater as close as you can to the lowpoly surface, try to not intersect your highpoly object. The closer you can get it to the lowpoly surface, the less distortion will be noticeable.
Vertex normals are fragile, and most modifiers will break or reset them but the bend modifier is an exception. This means this method works best for flat surfaces, and surfaces with consistent curvature in which case you can use bend.
If you need the floater to be deformed on some kind of complex surface, you can use another option which is saving and restoring vertex normals. This script was originally shared to me here on polycount, I take no credit for it, but since the link is dead I've reuploaded it. It stores the normals using UV channel 5, so there's also a script I made that clears UV channels greater than 1 if you ever need to do so.
https://mega.nz/#F!VKImRC5T!o8iDXWj5WYHGooxlecFh0w
In this case you can store normals once you have your floater done and subdivided, you can then deform it and restore the normals whenever you're done flattening or deforming to get it onto the lowpoly surface.
Replies
That said, as expected, this method results in much different AO bakes then using floaters with actual depth.
That said, curvature maps are either better or the same depending on which baker you use. Substance Painter uses the normal map to generate the curvature so if the normal looks okay so will the curvature.
Baking in xNormal on the other hand which uses the highpoly to create a curvature, there's a visible seam. Flat floaters don't have this issue.
It's a trade off I guess, I still need to use this method a little bit before I decide what's best for my workflow. In my opinion I don't care much for an AO map in the texturing process, but of course it would be nice to have a good AO for export into a game/rendering engine.
https://player.vimeo.com/video/190166224?
More info here: https://www.marmoset.co/posts/toolbag-baking-tutorial/
Actually makes me mad to know Painter isn't capable of this, it's not like it's a rare problem.