Home Technical Talk

[3ds Max] Skewmesh: Tessellate Smoothing Artifacts (Solved)

polycounter lvl 10
Offline / Send Message
Nominous polycounter lvl 10
I'm trying to use the skewmesh technique for baking normals, but I'm having trouble with the Tessellate modifier creating weird smoothing in 3ds Max 2017. These smoothing artifacts appear in the baked world space (and subsequent tangent space) normal map in Substance Designer. I have a finished, UVW Unwrapped, and Reset XForm'd low poly model that I cloned and applied the Tessellate modifier to. Below are images of the low poly (top) and tessellated (bottom) pommel.



The top model is how the bottom should look like. This strange smoothing appears when using different materials. The tessellated pommel's polys share a single smoothing group just like the low poly. I'm using the same Tessellate modifier settings from beefaroni's video. Changing the settings from Faces to Polygons, Edge to Face-Center, and increasing iterations doesn't help. I've tried putting an Edit Poly modifier on top and even collapsing the Tessellate modifier in order to check and tweak the smoothing groups, but it doesn't help either. I've also tried exporting the tessellated model as a FBX (triangulation on and off) and re-importing to a new scene to no avail. Interestingly, clearing the smoothing groups (not shown here) on both models results in them looking exactly the same—faceted only along the low poly's edges.

beefaroni mentions in his video around 3:38 that the Tessellate modifier can mess up smoothing groups and cause hard edges to become soft. The pommel has a single smoothing group and I still see it intact through the Polygon: Smoothing Groups window in the Command Panel. His specific problem doesn't seem to apply to me, but I'm guessing it's related to the modifier bugging out.


This is the world space normal map baked from the high poly onto the tessellated model in Substance Designer. You can see the faceted, shadowy lines that run horizontally across the pommel (center-left) and scabbard (center). Importing the tessellated model into Substance Painter without the converted tangent space normal map applied also shows the smoothing artifacts as expected.

Does anyone know what the problem is here and how to fix it? Thank you in advance.

Replies

  • Nominous
    Options
    Offline / Send Message
    Nominous polycounter lvl 10
    I've solved this by replacing the Tessellate modifier with a Subdivide modifier instead. It basically does the same thing except it doesn't cause smoothing artifacts like Tessellate does, which I still don't know the reason for. Now my world space normal map baked in Substance Designer looks great.

    However, I've run into another problem: converting my world space normal to tangent space results in a nearly empty, flat purple normal map. Strangely, this conversion works fine on a different model that I applied the skewmesh technique to. The difference between that model and this one is that this sword is detached into four objects: Blade_low, Guard_low, Pommel_low, and Scabbard_low, whereas that one is a single, contiguous object. I applied different materials to these objects in order to generate texture sets in Substance Painter and to avoid baking errors from overlapping geometry.

    I guess I'll mark this thread as solved since this new problem doesn't pertain to the original one.
Sign In or Register to comment.