Been playing around with Quixel Suite for a couple of weeks and loving it. Have ran into a variety of issues one of which being the failure to get edges on geometry even when providing normal maps. I see from the support thread that several others faced this problem, so after investigating I wanted to post my findings.
DDO Curvature Creation Broken In Some Cases?
The main concern is that DDO's generated curvature map fails to detect geometric edges, even when provide with both tangent and object space normal maps. This failure suggests to me that there might be a bug in DDO's current curvature creation, perhaps related in some way to my specific model or its uv layout?
In addition I ran into a few issues whilst trying to use xNormal to generate a good curvature map which might have some bearing on the problem as it seems the background of the curvature map beyond the mesh uv layout, needs to be gray in order to obtain the best results. I wonder if the background colour or perhaps rouge alpha/transparency values may be messing up DDO's curvature creation?
Sadly as a user I don't think you can easily view any of the intermediate maps DDO creates like its own Curvature map to double check these issues against. That might be something worth adding to a future release as I do sometimes find myself wondering what DDO is generating.
Above you can see the test model, a very simple, strong edged pyramid type shape, its uv and tangent map.
Below you can see the resulting masks for the 'Metal Edge Wear' material applied to the model, excluding the very top section (purple) and the insets around the bottom (blue). All the following images use exactly the same settings and input maps (ID, tangent, object space, AO ), I simply change whether DDO generates the curvature map (high quality) or if I use a version from xNormal.
DDO Created High Quality Curvature - Curvature set to Edge Mode.
As you can clearly see above DDO's curvature map is completely missing the edges of the pyramid, it just barely picks up a few edges on one corner. Interestingly switching the mask to cavity mode (below) shows it pick up the opposite corner edge. Clearly there is something amiss here.
DDO Created High Quality Curvature - Curvature set to Cavity Mode.
Best Practice for Curvature Maps in DDO
The next three images use a curvature map from xNormal with a black, white and finally a gray background.
xNormal Curvature Map - Black Background
xNormal Curvature Map - White Background
xNormal Curvature Map - Gray Background
The reason for showing these is that I believe the default setting in xNormal is to have a black background for areas that don't have the mesh present. This is clearly problematic and creates poor (no existent) edge results in DDO. The thing is if you export to png from xNormal you may never notice that it is a black background because it is marked as transparent and is thus not shown in previews or thumbnails, but it is used by DDO.
Switching to a white background clearly shows the edges now, but they are very limited. As can be seen mid-point grey works the best and once you realise it, its pretty obvious the correct choice when looking at the samples Quixel have provided on their input map page. Unfortunately no-one stated that the background should be gray.
Above are two screen-shots of xNormal Curvature bake settings, the top is what I think are its default settings, the bottom are what I found best in this case, with the background being gray (128, 128, 128 ) - Opps just realised this should probably be (127, 127, 127)
I'd also recommend for anyone new to using xNormal that once you set up your high/low models and got baking settings just right, that you save the settings and store it with the source assets. There are so many litle tweaks that need to be made often between models or projects that having a per asset setting for xNormal saved would seem to be a good idea should you have to return to the asset a year or two down the line.
Finally you can find a link to download the source assets at the start of the next post.