Edges on Geometry - DDO Curvature Map Creation Bug & Best Practice
polycounter lvl 6
Hi,
Been playing around with Quixel Suite for a couple of weeks and loving it. Have ran into a variety of issues one of which being the failure to get edges on geometry even when providing normal maps. I see from the support thread that several others faced this problem, so after investigating I wanted to post my findings.
DDO Curvature Creation Broken In Some Cases?
The main concern is that DDO's generated curvature map fails to detect geometric edges, even when provide with both tangent and object space normal maps. This failure suggests to me that there might be a bug in DDO's current curvature creation, perhaps related in some way to my specific model or its uv layout?
In addition I ran into a few issues whilst trying to use xNormal to generate a good curvature map which might have some bearing on the problem as it seems the background of the curvature map beyond the mesh uv layout, needs to be gray in order to obtain the best results. I wonder if the background colour or perhaps rouge alpha/transparency values may be messing up DDO's curvature creation?
Sadly as a user I don't think you can easily view any of the intermediate maps DDO creates like its own Curvature map to double check these issues against. That might be something worth adding to a future release as I do sometimes find myself wondering what DDO is generating.
Above you can see the test model, a very simple, strong edged pyramid type shape, its uv and tangent map.
Below you can see the resulting masks for the 'Metal Edge Wear' material applied to the model, excluding the very top section (purple) and the insets around the bottom (blue). All the following images use exactly the same settings and input maps (ID, tangent, object space, AO ), I simply change whether DDO generates the curvature map (high quality) or if I use a version from xNormal.
DDO Created High Quality Curvature - Curvature set to Edge Mode.
As you can clearly see above DDO's curvature map is completely missing the edges of the pyramid, it just barely picks up a few edges on one corner. Interestingly switching the mask to cavity mode (below) shows it pick up the opposite corner edge. Clearly there is something amiss here.
DDO Created High Quality Curvature - Curvature set to Cavity Mode.
Best Practice for Curvature Maps in DDO
The next three images use a curvature map from xNormal with a black, white and finally a gray background.
xNormal Curvature Map - Black Background
xNormal Curvature Map - White Background
xNormal Curvature Map - Gray Background
The reason for showing these is that I believe the default setting in xNormal is to have a black background for areas that don't have the mesh present. This is clearly problematic and creates poor (no existent) edge results in DDO. The thing is if you export to png from xNormal you may never notice that it is a black background because it is marked as transparent and is thus not shown in previews or thumbnails, but it is used by DDO.
Switching to a white background clearly shows the edges now, but they are very limited. As can be seen mid-point grey works the best and once you realise it, its pretty obvious the correct choice when looking at the samples Quixel have provided on their input map page. Unfortunately no-one stated that the background should be gray.
xNormal Settings
Above are two screen-shots of xNormal Curvature bake settings, the top is what I think are its default settings, the bottom are what I found best in this case, with the background being gray (128, 128, 128 ) - Opps just realised this should probably be (127, 127, 127)
I'd also recommend for anyone new to using xNormal that once you set up your high/low models and got baking settings just right, that you save the settings and store it with the source assets. There are so many litle tweaks that need to be made often between models or projects that having a per asset setting for xNormal saved would seem to be a good idea should you have to return to the asset a year or two down the line.
Finally you can find a link to download the source assets at the start of the next post.
Been playing around with Quixel Suite for a couple of weeks and loving it. Have ran into a variety of issues one of which being the failure to get edges on geometry even when providing normal maps. I see from the support thread that several others faced this problem, so after investigating I wanted to post my findings.
DDO Curvature Creation Broken In Some Cases?
The main concern is that DDO's generated curvature map fails to detect geometric edges, even when provide with both tangent and object space normal maps. This failure suggests to me that there might be a bug in DDO's current curvature creation, perhaps related in some way to my specific model or its uv layout?
In addition I ran into a few issues whilst trying to use xNormal to generate a good curvature map which might have some bearing on the problem as it seems the background of the curvature map beyond the mesh uv layout, needs to be gray in order to obtain the best results. I wonder if the background colour or perhaps rouge alpha/transparency values may be messing up DDO's curvature creation?
Sadly as a user I don't think you can easily view any of the intermediate maps DDO creates like its own Curvature map to double check these issues against. That might be something worth adding to a future release as I do sometimes find myself wondering what DDO is generating.
Above you can see the test model, a very simple, strong edged pyramid type shape, its uv and tangent map.
Below you can see the resulting masks for the 'Metal Edge Wear' material applied to the model, excluding the very top section (purple) and the insets around the bottom (blue). All the following images use exactly the same settings and input maps (ID, tangent, object space, AO ), I simply change whether DDO generates the curvature map (high quality) or if I use a version from xNormal.
DDO Created High Quality Curvature - Curvature set to Edge Mode.
As you can clearly see above DDO's curvature map is completely missing the edges of the pyramid, it just barely picks up a few edges on one corner. Interestingly switching the mask to cavity mode (below) shows it pick up the opposite corner edge. Clearly there is something amiss here.
DDO Created High Quality Curvature - Curvature set to Cavity Mode.
Best Practice for Curvature Maps in DDO
The next three images use a curvature map from xNormal with a black, white and finally a gray background.
xNormal Curvature Map - Black Background
xNormal Curvature Map - White Background
xNormal Curvature Map - Gray Background
The reason for showing these is that I believe the default setting in xNormal is to have a black background for areas that don't have the mesh present. This is clearly problematic and creates poor (no existent) edge results in DDO. The thing is if you export to png from xNormal you may never notice that it is a black background because it is marked as transparent and is thus not shown in previews or thumbnails, but it is used by DDO.
Switching to a white background clearly shows the edges now, but they are very limited. As can be seen mid-point grey works the best and once you realise it, its pretty obvious the correct choice when looking at the samples Quixel have provided on their input map page. Unfortunately no-one stated that the background should be gray.
xNormal Settings
Above are two screen-shots of xNormal Curvature bake settings, the top is what I think are its default settings, the bottom are what I found best in this case, with the background being gray (128, 128, 128 ) - Opps just realised this should probably be (127, 127, 127)
I'd also recommend for anyone new to using xNormal that once you set up your high/low models and got baking settings just right, that you save the settings and store it with the source assets. There are so many litle tweaks that need to be made often between models or projects that having a per asset setting for xNormal saved would seem to be a good idea should you have to return to the asset a year or two down the line.
Finally you can find a link to download the source assets at the start of the next post.
Replies
Project assets download link - includes models (blender/obj), input maps, DDO project etc.
Below are 4 examples to illustrate the issue DDO has in creation of its own curvature map. It appears that regardless of how the tangent normal map looks it still misses 50% of geometric edges.
Edit: Just wanted to point out that the only issue here is that DDO curvature creation fails to detect actual hard geometric edges as that data is difficult to extract from a normal map. DDO has absolutely no problem creating accurate curvature data for the surface of a mesh as long as it has a decent normal map to work with.
Above: This shows the default result when using a tangent normal map created by xNormal and saved as png. As you can see in the normal map its made up of rectangles surrounding the areas of the uv layout. It is my understanding that this is normal for png files, where areas of transparency often have random or stretched pixels. The actual alpha/transparency for this png correctly masks those pixels outside of the uv layout, but it seems this information is not being utilised.
Looking at the other maps it would appear obvious that the curvature is using the outline (difference between white fill and blue tangent data) of the normal map to determine edges as opposed to the actual alpha/transparency of the png or the uv layout. This is why you see straight lines of edges far away from the actual edges.
Above: Here you see the results when the tangent normal map has a white fill applied to the areas outside of the uv layout (i.e areas that were transparent or meant to be alpha'd). The results here are better, edges are clearly found along the actual corners of the mesh geometry, but strangely only for one corner of each side of the pyramid shape. Unfortunately the edges marked in these maps are mostly outside of the uv layout, hence why they are barely visible on the 3D model.
Above: For comparison here is the result when the tangent normal map background is filled in with blue (i.e. default normal color). This time no edges are found with the exception of the text.
Above: Finally an example using a curvature map generated by xNormal, see end of previous post for the settings I used. Although the edges are a little overdone, this looks correct to me.
So in summary it would seem to me that there is currently a problem in DDO's curvature creation that does not take into account the alpha/transparency that comes with the tangent normal input map. If it did then regardless of what happens to the actual colour pixels in these transparent areas the shape of the uv layout and thus geometric edges would be correct.
In addition though even if it used the alpha/transparency it would probably still only generate similar results to that of the white background normal map, in which 50% of the edges are not detected, so thats a secondary issue that needs to be addressed.
However relying on a tangent map coming with an alpha is maybe not a good idea, neither would be relying on areas outside of the uv layout to have a different background fill. It would seem to me the most reliable method would be to use the input mesh to generate a unique alpha map of the uv layout, a mask if you like. I guess this could also be achieved via the ID map, but then you'd need to know which colour the background is, though that in itself is not a bad idea. Maybe the simplest method is just to ask for a uv mask input?
Anyway overall point being that currently DDO curvature creation would appear to frequently not be able to detect geometric edges on a mesh as normal maps don't tend to shop this.
ddo creates its curvature map using your normal map, so if thats flat, you can't get any results.
the mesh option in ddo is only so you can view your material in 3DO using that mesh, it does not use the mesh for the map creation process.
Best solutin (IMHO) create the curvature map in Xnormal the way you have, and provid it with the normal (for the details that are not geometery).
IN xnormal, I found reducing the spread angle worked very will in picking up edges better.
Yeah the predominantly flat normal map is definitely a major contribution factor to the issue, its just a shame that DDO doesn't attempt to use other data to create a better informed curvature map.
I'm also not entire convinced yet that this isn't a problem, but i'll admit that its sometimes hard to visualise what is going on with the intermediate maps and processes within Quixel (this is all guess work, accompanied with investigation and results).
So what would happen if I made my pyramid mesh 'rough' with chipped edge corners, like a very weathered stone? Obviously the normal map would pick up the chips along the edge, but anywhere that was not chipped would still have flat normals and thus would not be detected as an edge?
@Gordon
Yep, its just a shame to have to use another program, even though I've ended up having to use xNormal due to a variety of failings in Blender to create tangent normals, object space normals (cycles is broken and blender renderer has incorrect direction to color mappings) and Ambient Occulsion.
@Hugo
Glad it was useful.
Having reflected on this a bit I think my main two criticisms are;
1. Quixel does not provide enough explanation of the issues with curvature map creation or best practice for using something like xNormal, nor that using xNormal is imperative if you have hard geometric edges that aren't technically defined in the normal map.
2. I can't help feeling Quixel curvature creation could be improved to address this issue. Although thinking about it, simply treating edges between inside/outside of uv layout to create hard edges is not a good idea since certain unwrap features will use a min/max angle and thus a curved volume could be split into several disconnected islands. Thus treating islands as hard edges would give incorrect and very bad results.
However on Quixels InputMap wiki it states so i'm a little surprised Quixel itself doesn't seem to use the AO map in combination with the normal map or at least provide an option to the user for this in an attempt to generate better curvature maps. Maybe attempting to combine generic AO and normals simply does lead to good results? Though perhaps finding some means to use it in predominately flat normal areas would work?
Its a little frustrating that Quixel isn't open source as this feels like a challenge that would be fun to devise a solution for. I'm positive there must be avenues that could be explored to improve DDO's curvature creation.
Anyway hopefully this page and examples will at least inform other users as to why they might not get the expected results and the best way for them to address the issue.
dude i gues this is not what you want to hear, but you are wasting your time poking at this issue.
DDO is a photoshop tool, wich relies completely on photoshop functions to work, wich limits it to interpreting your 2D maps.
there is absolutely no reason to fix the curvature creation when you can just bake a curvature map using other open source programms, while there are tons of other issues to fix and more important improvements to make (like tools for fixing seams or actually building a stable version of the software)
But the last version (non suite one) version of DDO created curvature with a simple trick (can be put into a very shor action) to the normal map using emboss that makes a white line on convex shapes of the normal and a black one in concave and then you can blur and such to get the lines thicker.
I guess the suite still does it that way because its super fast anc because it can be done in photoshop. Baking a curvature map from the geometry cannot, thats why xnormal exists and that's why there is a curvature slot when you load your maps, i see no point in implementating a better curvature method because that method is baking it.
I'm not against using third party apps to generate maps, but currently the 'suggestion' is that DDO can create the curvature map from your input maps. However as half a dozen people in the 'support' thread have discovered it fails to account for geometric edges, thus leading to developers wasting considerable time trying to get edge/cavity masks working correctly, when DDO's own curvature creation can never be made to work for those cases.
This issue is not made clear in any of the existing documentation, and though there are suggestions to use xNormal, it doesn't tell you why its necessary. I myself just assumed xNormal would be more accurate and so skipped generating the map as I wanted to just test out DDO.
Secondly i'm not saying that DDO needs to be able to bake curvature maps from geometry, but I am saying that the current curvature maps created are generally not up to the job for many cases and that perhaps mesh data might be used to enhance it in the future.
I fully agree that in the current scheme of things, looking to improve curvature creation should be a low priority (stability, seam fixing, offseting, rotation are all more important at this stage), but getting the documentation to explain the issues with using DDO curvature creation is a high priority as developers are wasting time on something that can never give them the results they need. I'd almost go as far as suggesting that DDO needs a warning added if no curvature map is detected to explain that geometric edges will be missed.
One problem i've heard expressed about using xNormal is what if you don't have a high and low poly model version? However a quick test using a low poly in both slots and baking a curvature map it would seem to still work fine. Of course this might seem like a counter-intuitive process to some who are new to all these 3rd party processes.
Thanks for the info about previous versions. I see from the wiki that the new DDO will actually choose from the input maps based on a priority list - Cavity, Tangent space normal, AO, Height, Color, ObjSpace.
I did a quick test, removed the tangent map and for geometric edges I found AO gave a slightly improved result, but missed internal details. Though I assume issues with png format transparency might again come into play here, so its not necessarily reliable, just like normal map edges weren't - see my above screenshots.
In addition I tested using my AO map and running it through a high pass filter and some other effects and got pretty decent results. No where near as good as xNormal, but I did get some geometric edge detection.
This is what gives me some hope that curvature creation could be improved in the future, but it would depend upon having a input mesh. Though you argue that DDO is done in Photoshop thats not entirely true since it appears to run as an app that interfaces with it, allowing it to call PS scripts I assume. So I feel there is certainly scope to code up a hybrid solution where the app could generate a 2D mask from the mesh geometry to denote where actual mesh edges exist in the uv map. This could then be provided to Photoshop to allow it to mix between different 2D processes for generating curvature data, e.g. AO for areas around geometric edges, tangent for internal surface areas.
Of course this is just speculation. Really need to do some more research to find out exactly how curvature maps are produced and used to determine a good solution that doesn't involve baking.
Addendum
What might be nice is if DDO offered a means of deciding which map to use for generating a curvature map and to be able to export its generated curvature map.
if you want to create a curvature, you can use NDO to create one as finetuned as you like.
the curvature input is designed for ease of use, so you can just plug in your maps and go, without having to hassle around with any settings.
if you don't like the curvature that DDO creates by default just create one yourself using NDO or bake one or combine both.
as for the documentation; have read the documentation for any other professional software?
they all just tell you what you can do, and if there something you should be able to do but can't or there is a problem or a more effective way, of course that is never mentioned in the documentation.
noone opens up the documentation to find out what a tool can't do, thats what you need to find out yourself.
Ease of use is great. Sadly though in this case it ended up wasting a good chunk of my time that could have been better spent elsewhere. Just a little note or warning about geometric edges not being detected would have saved me (and others) a lot of time.
Just because documentation is bad elsewhere doesn't make it a standard to strive for.
If half a dozen people are reporting the same issue, that is easily explained by knowledge of the software, then it would be nice if that information would be included in the documentation.
Anyways, something is odd with dDo when using a custom curve map. Every single time I use a curve map generated form xN I get this:
Left: MatID, AO, NRM (t), NRM (o) + Curve mapo
Right: MatID, AO, NRM (t), NRM (o) only
Curve map is grayscale, generated from the highpoly.
And this is how the side of the cylinder looks on the curve map (UV shell is straightened):
Ok so what is the advantage of using this technique over the zbrush xnormal process? Once you have done all the maps you can preview different materials and different masks pretty quickly which makes the process very flexible. Not only that but you can go back and change things later if you want. You can do that with zbrush too but it is different. I found that ddo found the edges of the asset pretty well but not from a curvature map I made.
For me the best way seems to be:
Make a clean high poly.
Make a low poly. (edit: I think it works best with a one piece lowpoly).
Bake a gradient map in max (link on the wiki for the script).
Bake a colour 'block in' map in max (link in the wiki for the best method).
Bake a normal map in xnormal, tangent and object space.
Bake AO in xnormal.
Fire up the suite, plug in the maps and go for it.
I let ddo make the curvature map because the one I tried to make with xnormal on another test asset failed. I am not really super at making spec maps and I like the one ddo made a lot, not to mention the cuts and scratches, wear and tear etc.
I think the best practice would be to use the method outlined in the wiki and not try and make the suite do stuff the way you think it should work. My experience with new software is that I can crash it effortlessly by pushing the wrong buttons and not reading the documentation properly. The reason for this is we bring preconceptions into a new situation due to experience built up in another area.
Hope this helps
Cheerio