Baking Normals Topic: Cages. Are they old news?

Devon M

I'm relatively new to the 3D modeling world (couple years now) and I've always done just a typical high to low poly bake with my high poly mesh and my low poly mesh. I throw it in xNormal (or Substance) to bake out my Normals and tweak it a bit in Photoshop if necessary. 

However, I've been following a tutorial series over an environment piece, and the guy is using Cages to help bake out his Normals. This was the first time I ever heard of Cages. So I looked it up on Google and researched what exactly they do and why people use them. And I got really good results in my bakes when I started using Cages throughout this tutorial. 

I told a friend about it (he has roughly the same years of experience as I) and he talked to his boss at the place he works and the guy said "Cages are 'old news'" and that he doesn't use them.  

So, my question to the fine folks of PolyCount is this:

When baking out your Normals, AO, etc. etc., is using a cage still a typical, standard practice? Or is this an old way of baking out maps? Why and why not? When is it ever appropriate to use them, if at all?

Thanks everybody. 

musashidan

I haven't used a cage since I started baking in Substance about 1.5 years ago. Mainly because I use an averaged vert bake for evrything(hard-surface/organic) and use Substance Painter and UE4 which are synced by mikkt tangent space. I simply haven't come across a case where I needed a cage. But that's my preference. I'm not saying 'f**k cages'. I just don't need them.

There are many who still use cages, and swear by them. The fact is that the 'synced' method has only been around for a few years, whereas the older method has been around for an absolute age and is what people are used to, and cages have always played a big part in that workflow. I wouldn't say they are 'old news'. They will certainly remain for a while yet.

Testing is the only way to truly answer these kinds of questions. Just learn all the baking methods, test them out, and decide what works and what to throw out. The problem with baking is that so many people get confused by the process and the amount of rules and opinions, and end up not really understanding what that process actually is and why certain factors must be observed.

Devon M

@musashidan Yoo thanks for the advice. You actually dropped by my other forum topic about Dynamesh in zBrush. Thanks for that too!

Yea, I'm still sort of exploring my way in the 3D technical world of what workflow I prefer. I love dropping posts like these to see how other people work, and then practice their workflow to see why they like that.

I guess I'm just imitating the greats until I find my own way ^.^ Making a Cage isn't really all that difficult and it's pretty quick. Cage options are still a "thing" inside almost all 3D modeling/baking software too. 

Thank you for dropping by!

musashidan

No problem, mate. That's the best thing you can do, practicing, learning, testing. Every little thing you learn adds up exponentially over the years. That's what I've found anyway. And plus, this shit is so much fun to learn!

Macebo

I only bake using a cage, and setting up smothing groups/UV islands annoys me a little. The workflow you mentioned seems so straight forward and easy/fast that i might try it sometime. Does this work with other engines, like Unity?  

vertex_

Macebo said:

I only bake using a cage, and setting up smothing groups/UV islands annoys me a little. The workflow you mentioned seems so straight forward and easy/fast that i might try it sometime. Does this work with other engines, like Unity?  

Check out this script that breaks smoothing groups by UV islands, huge time saver:

http://www.martinpalko.com/create-smoothing-groups-from-uv-islands/

Devon M

@vertex_ Yooo would that work in Maya? Smoothing groups = soften/harden edges right?

@Macebo My old workflow was HP mesh and bake onto LP mesh. New workflow is HP mesh, back onto LP mesh that has a cage attached. I'm liking it so far. 

kadeschui

From my tests in modo baking with a cage generally produces the best results, though they are barely perceivable over a regular averaged vert bake, which is generally faster.

With the new baking toolkit in modo, though, a cage (push modifier) is a 30 sec process that I tend to do anyway...

edit: I've been meaning to try Substance for baking though as this seems like the way of the future, the way of the future....

Devon M

@kadeschui We actually use Substance for baking where I work. Boss swears it produces better bakes than xNormal. 

pior

"And I got really good results in my bakes when I started using Cages throughout this tutorial."

I hope I am not going to sound like a jerk here, but the first step for you to determine whether or not they are "old news" would be for you to investigate and understand why you suddenly got good results out of using them. And you also need to explain what you actually mean by "good results", because that can mean anything and therefore kinda means nothing There is not really a spectrum of good or bad bakes : the result is either mathematically accurate, or it is not.

(a hint : some baking programs, Xnormal included, split ray directions at hard edges. Meaning that if your low has hard edges, then you will miss portions of your high in these areas during casting. Loading up a cage forces the Xnormal raycasting to behave as if thrown from a continuous surface - meaning no missed spots, on top of the advantage of being able to finely control ray distance and ray direction by shaping the cage as desired.)


"The guy said "Cages are 'old news'" and that he doesn't use them."

Here too no disrespect intended, but if this artist just says this without a clear explanation he probably has no idea why they would be used or not Maybe he mostly works with organic models.


Lastly,

"Check out this script that breaks smoothing groups by UV islands, huge time saver"

A word of caution here : Turning all UV island edges into hard edges is absolutely not a requirement, and is a bit of a dangerous shorthand since by using such a workflow one might end up with hard edges in places where there are totally unnecessary and cause issues. Now of course such a brute force workflow is okay at times, but it is probably wiser to actually remember the actual reasoning  : A hard edge can only be placed on an edge that is also a UV seam, because texture filtering / mips would not be able to deal with such a sudden change of normal information.

Bottom line ... it might be best for you to show us some of your case studies, for the sake of a constructive and practical discussion

musashidan

@pior above post should be the mandatory introduction to the baking section of the wiki.

Joopson

@Devon M

This one works in Maya. I can't remember who made it, but it sits on my shelf and I use it constantly. It's in MEL:

string $objList[] = `ls -sl -o`;

string $uvBorder[];

string $edgeUVs[];

string $finalBorder[];

for ($subObj in $objList) {

select -r $subObj;

polyNormalPerVertex -ufn true;

polySoftEdge -a 180 -ch 1 $subObj;

select -r $subObj.map["*"];

polySelectBorderShell 1;

$uvBorder = `polyListComponentConversion -te -in`;

$uvBorder = `ls -fl $uvBorder`;

clear( $finalBorder );

for( $curEdge in $uvBorder ) {

$edgeUVs = `polyListComponentConversion -tuv $curEdge`;

$edgeUVs = `ls -fl $edgeUVs`;

if( size( $edgeUVs ) > 2 ) {

$finalBorder[ size( $finalBorder ) ] = $curEdge;

}

}

polySoftEdge -a 0 -ch 1 $finalBorder;

}

select -r $objList;

Devon M

@pior Thanks for all the advice! I really appreciate it. When I finalize things on my end, I'll post some screenshots. And no offense was taken .

@Joopson Yoooo thanks! I'll check it out. 

musashidan

Macebo said:

I only bake using a cage, and setting up smothing groups/UV islands annoys me a little. The workflow you mentioned seems so straight forward and easy/fast that i might try it sometime. Does this work with other engines, like Unity?  

Yes, I believe Unity now supports mikkt tangent space.

Macebo

musashidan said:

Macebo said:

I only bake using a cage, and setting up smothing groups/UV islands annoys me a little. The workflow you mentioned seems so straight forward and easy/fast that i might try it sometime. Does this work with other engines, like Unity?  

Yes, I believe Unity now supports mikkt tangent space.

 Good to know! Thanks.

pior

@Devon M
Nice Debugging normal map issues is actually pretty fun (when not in a rush) especially now that Mikkt is commonly accepted, so yeah don't hesitate to show us what you are working on !