[Tutorial] Perfect Normal Map baking in Substance Designer 5 without using a Cage

Quack!

Hello everyone!

I created a video explaining the process I use to bake my normal maps. This removes cage creation from the process while retaining the benefits of an averaged cage.

Comments and critiques are welcome!

[ame]https://www.youtube.com/watch?v=sGC2X6Qazvs[/ame]

Fansub

Will be watching this ASAP,but damn your thread's title is eye catching

Bartalon

This is great. Have you tested it on much more complex models that have tight areas, holes, details across curved surfaces, etc?

EarthQuake

This is cool, great to see that you've found a smart way to automate masking between an an averaged/ and explicit normal cage bake. Thanks for putting in the time and effort to document the process!

Unfortunately, there will be real-world cases where this method doesn't work well. For it to work with complex assets, you'll need to specifically plan out your mesh to ensure you have totally flat vertex normals anywhere near floating/problematic detail. This may mean a lot more hard edges and UV seams, a trade off I wouldn't be likely to make when the alternative is cutting in a few extra verts around problem areas.

Again, nice that you've automated it though, as the manual masking/double baking process was one of the primary drawbacks to this workflow.

Quack!

Fansub wrote: »

Will be watching this ASAP,but damn your thread's title is eye catching

It is hyperbole, but I wanted to have some impact with it. :]

Bartalon wrote: »

This is great. Have you tested it on much more complex models that have tight areas, holes, details across curved surfaces, etc?

I have used this on a 10k tri object and my VERY picky normal map eyes couldn't spot errors. I'm not saying errors can't be present, but on that object it was great. I will be trying this on a 50k tri hard surface object soon and I will have a more definitive answer.

EarthQuake wrote: »

This is cool, great to see that you've found a smart way to automate masking between an an averaged/ and explicit normal cage bake... *SNIP*

Thanks for the response EQ, I knew you would have valuable input. I definitley understand there may be errors present, but I have yet to spot them. I'll be trying this out alot in the near future and will document any errors/modifications.

I don't think this statement is true for this method though(maybe I need to put more floating detail on a curve/bulbous shape and test again):

For it to work with complex assets, you'll need to specifically plan out your mesh to ensure you have totally flat vertex normals anywhere near floating/problematic detail. This may mean a lot more hard edges and UV seams, a trade off I wouldn't be likely to make when the alternative is cutting in a few extra verts around problem areas.

Substance Designers non averaged bake makes a bake with perfect details with no cutting in of verts needed, at least as far as I have tested so far.

EarthQuake

Again, the non-averaged bake relies on your lowpoly mesh normals being set up in a specific way. Let's say you have a 45 degree bevel and no seam (very common in my lowpolies) instead of a hard edge and a seam along a prominent edge. Your method won't have any affect on skewing for those faces.

If I can find some time I'll whip up a test asset that should break it.

Quack!

EarthQuake wrote: »

Again, the non-averaged bake relies on your lowpoly mesh normals being set up in a specific way. Let's say you have a 45 degree bevel and no seam (very common in my lowpolies) instead of a hard edge and a seam along a prominent edge. Your method won't have any affect on skewing for those faces.

If I can find some time I'll whip up a test asset that should break it.

AHH, I misunderstood what you meant. But I(think), I understand now. In my method, I believe I have that exact test-case? 45° angle bevel, with the adjacent areas on the same smoothing group and uv. This method just uses the non-averaged version which would have perfect detail projection.

beefaroni

Nice video! I've been doing something similar except using painter as well as support geo where needed. That's a really good idea though.

Also, with the match by mesh name, I'm able to bake a 24K model with ~15 low poly parts without exploding or cage creation. Real real nice and easy.

I am curious to see how you figure out what Earthquake mentioned about the skewed normals on the edge. My process is not nearly as automated as yours and it'd be nice to save all that time on baking if possible.

EarthQuake

Hmm, I didn't see any 45 degree bevels if your video, maybe I missed that bit.

Anyway, here you go (tested in XN w/o cage, run it through substance): https://dl.dropboxusercontent.com/u/499159/skewquack.zip



As you can see, the floaters near the beveled edges are still skewed. Floaters in the center part of the face look good, as the normals average out to pointing straight out in the center.

It's worth noting, you could probably improve this result by using your technique plus face weighted normals or edited normals to make the vertex normals on the face in question completely planar.

Quack!

Ok I see what you mean. I am going to do another test tonight and will incorporate this to see how it holds up(I don't have high hopes).

Quack!

Ok Joe did a test with what you suggested and there is some skewing happening, thanks for showing that to me.

The 3 small 'bolts' are the test with the closet being skewed, to a bit less degree then yours because of the distance to the next vert of course. The edge facing us is chamfered, the edge facing away is hard.

Bek

Interesting, it's cool that you can automate the bake + combining of maps in substance.

However I'm thinking if you're set on a cageless method you'd be better off using a skewmesh method where you feed your baker a denser mesh (i.e manually add geo where needed to avoid skewing) and output a OS normal map, which you convert to TS, ideally synched to your engine (handplane being ideal). That way you're getting the main benefits of a cage-bake without using a cage.

...However, there's nothing difficult or complex (at least in the props I've done, perhaps characters or otherwise are inherently more tricksy) with using a cage — they shouldn't be difficult to make or annoying to iterate/send to your baker. If there is I'd look to optimising that (scipts, automation) rather than a double bake method.

When I last tested the "skewmesh" OS+raydistance to TS I still concluded in favour of cage-bakes, though others will likely disagree — some of the issues I had with the skewmesh method may have been due to handplane rather than the bake itself. I might have to see if I can dig up the test scene I had. I recall generally higher quality results using xn + cage rather than xn os (skewmesh) > handplane > ts, plus with cages being a simpler workflow (especially considering iteration) cages were the clear winner for me.

Quack!

There are lots of methods to bake nowadays and many ways to get fantastic looking bakes. It all boils down to the tools at your disposal. Many parts of the pipelines can be automated to a certain degree as long as there is experience available to make that automation happen. In my case, I wanted a 'turn key' solution to a cage-less bake.

Using a cage on a 25,000 tri+ object can, and often does, get out of control. If I can remove that time from my workflow I feel it is worth a *very* slight quality loss.

Also, while skewmesh is great, it requires a number of extra steps that end up costing the time I want to gain back from not making a cage.

I still want to do a Cage VS This Method of baking for comparison to make sure the details aren't wildly wrong which as far as I can tell, aren't off.

Bek

50k (or 25) doesn't sound like a significant factor; how do you generally create and edit your cage? My typical workflow in modo is:

Create a new morphmap on my lowpoly > Use Push tool, pan around model to ensure HP is covered, export via pipelineIO script (1 or 2 clicks and my exploded lp + cage is exported to a fixed directory, which is already loaded in xN). Then in xN I'll check my settings (res, map type) and run a test bake. It's more of a hassle to subdivide and export the highpoly if anything; I wish I could feed xN the cage and let it subdivide with x iterations of psub...

The important thing there is that if I need to edit the cage and send it back to xN it's incredibly easy because the morph map stores a non-destructive cage and there's no manual saving involved. I'm curious what your workflow might be like for cages of 50k+ stuff to be problematic — I think if you can improve your cage workflow/pipeline you'll find it to be a simple and reliable solution.

(Not that I'm dissing your SD solution, it's great that you're exploring different methods — if I had SD myself I'd certainly give it a go)

Quack!

I basically make my low polys watertight nowadays. My cages, even on large objects are usually not more then a few rather large chunks. This technique isn't really about speeding me up, but speeding a large number of people up that I don't have time to personally train.

And no worries, about me taking the feedback the wrong way, I always assume professionalism!

Quack!

Cage versus Combine Bake

I didn't change the low, just added a cage, and baked a new normal map, results:


I would have to compensate for the skewing with geo using the cage.

mobpapst

Thanks for this great tutorial. iam gonna try this out on my next model.

Titus

Yey, it's nice , thanks for sharing!

Quack!

mobpapst+Titus - I hope you guys find this useful, let me know what you think!

Justo

I think it was Wes from Allegorithmic who told me he'd have the Substance devs check this thread out, though I don't know what good that'll do.

N-gon

would do this for your other maps ? like Ao , position, object space and cavity map?

aponomarev93

N-gon wrote: »

would do this for your other maps ? like Ao , position, object space and cavity map?

same method but u must use blend node for it

Quack!

N-gon wrote: »

would do this for your other maps ? like Ao , position, object space and cavity map?

Yup! There will be some modification to get it to work as aponomarev93 said, but in the end the same idea applies. I might extend this to work and then release it as a thing.

frmdbl

Pretty interesting trick, but I wonder if it works this well if you have more complex uv islands.

What I've been doing to avoid skewing is bake object space normals onto a subdivided low poly with averaged normals then convert into tangent space.

This preserves both smooth edges and projection on other surfaces.

eyo

Really well explained, thanks for sharing this Quack. Definitely going to try it out on my next bake!

beefaroni

Quack! wrote: »

Also, while skewmesh is great, it requires a number of extra steps that end up costing the time I want to gain back from not making a cage.

I just tested out both your method + skewmesh on my new rifle. While I was extremely excited about this, I think skewmesh is a better method for complex shapes.

I had some problems when it came to the exception you talked about near the end of the video. I have a few areas like that on the weapon and I'd rather just do the skew mesh.

I do think I'll test out this method for professional pipeline stuff on smaller assets that just need to be churned out quickly.

Skewmesh wasn't too many extra steps compared to this.
1. Tessellate modifier on low poly
2. Bring into subsatnce
3. Bake with match by name + no cage + average
4. Bake using object space
5. Use Substance to convert to tangent space.

Quack!

beefaroni wrote: »

I just tested out both your method + skewmesh on my new rifle. While I was extremely excited about this, I think skewmesh is a better method for complex shapes.

I had some problems when it came to the exception you talked about near the end of the video. I have a few areas like that on the weapon and I'd rather just do the skew mesh.

I do think I'll test out this method for professional pipeline stuff on smaller assets that just need to be churned out quickly.

Skewmesh wasn't too many extra steps compared to this.
1. Tessellate modifier on low poly
2. Bring into subsatnce
3. Bake with match by name + no cage + average
4. Bake using object space
5. Bring object space map into xnormal and convert.

Thanks for testing it. I think I will give skewmesh another go to refresh and compare the two.

beefaroni

Sounds good. Here's a quick screen I took. No AA or clean UVs right now, just test bakes.
https://i.gyazo.com/cfb12b83cd72f48307e50739245497d0.jpg

beefaroni

Posted some comparisons in my thread (no images of your method though, maybe I'll rebake and show if I get the time). If you look at the areas near thin edges those were the parts I had trouble with. Especially the indent in the lower battery since it's super thin. Overall it's pretty nice but I think for more complex models I'll stick with skewmesh.

http://www.polycount.com/forum/showpost.php?p=2362649&postcount=7

Quack!

Thanks for the update Chris. I'll be skewmeshing the shit out of my next asset to see what the fuss is about. : ]

Mant1k0re

Hey Nick, thanks a lot for this! Once I was done I brought the mesh to Toolbag and somehow it would show the normals as completely non-averaged - I had to tick Invert Y in Toolbag to make it work (and switch tangent space to MikkT, but this one was easy to figure out).