Hacky Normal Map Correction

Gannon · Apr 2012

I was baking out some normal maps for some stones I made for a scene and discovered a nifty little trick. I'm sure it doesn't fix everything but hopefully it helps some people in a pinch.

Sometimes I just can't get a good bake on objects so I'm glad I stumbled upon this. I'd love to see how this can be used on other models and situations.

Ace-Angel · Apr 2012

I don't understand, you're killing the gradient, which with a proper smooth-group setup should give the same results?

Unless I missed something, because it looks like your Normals are one group as opposed to split by UV cuts.

Gannon · Apr 2012

Maya doesn't use Smoothing groups per say, It's just defining hard and soft edges. The closest way to mimic them would be going through and hardening the outside edges of the shell/UV island and softening the interior ones.

passerby · Apr 2012

@Gannon the same thing still applies to getting a clean normal map bake, if you would like i can post a mel script i use that automatcally hardens the edges of UV islands for you?

Gannon · Apr 2012

I know, this just a hacky trick is all.

A script would be fantastic. I've been doing it manually for a while and sometimes that's pretty tedious.

passerby · Apr 2012

just toss this on a shelf, and run it on a object, after you do your UV's

string $objList[] = `ls -sl -o`;
string $uvBorder[];
string $edgeUVs[];
string $finalBorder[];

for ($subObj in $objList) {
select -r $subObj;
polyNormalPerVertex -ufn true;
polySoftEdge -a 180 -ch 1 $subObj;
select -r $subObj.map["*"];

polySelectBorderShell 1;

$uvBorder = `polyListComponentConversion -te -in`;
$uvBorder = `ls -fl $uvBorder`;

clear( $finalBorder );

for( $curEdge in $uvBorder ) {
$edgeUVs = `polyListComponentConversion -tuv $curEdge`;
$edgeUVs = `ls -fl $edgeUVs`;

if( size( $edgeUVs ) > 2 ) {
$finalBorder[ size( $finalBorder ) ] = $curEdge;
}
}

polySoftEdge -a 0 -ch 1 $finalBorder;
}

select -r $objList;

Gannon · Apr 2012

sweet, Thanks!

unstoppablex · Apr 2012

pretty nice trick for removing unwanted normal map shading!

thanks for sharing this

Sandro · Apr 2012

Okay, nice hack but you might as well cut out whole baking process and generate your normal from diffuse. Because resulting map doesn't have any big or medium surface definition left, just surface dents and noise.

r_fletch_r · Apr 2012

why not just fix your models smoothing? like sandro says, your totally loosing the advantage of baking a high to low normal map.

Bal · Apr 2012

This process doesn't make any sense at all.
If the gradients are there after your bake, it's that they are supposed to be there to compensate for the lowpoly shading, or you did something wrong. You shouldn't have to hack around in Photoshop like this.

And your final image still looks wrong, there are some weird inner bevel type effects on all the edges.

WarrenM · Apr 2012

This has come up in other normal map threads but yeah, what they said above me. If you have to hand tweak the normal map afterwards, that means you're going to have to do that every time you update the normal map. That will add up to a lot of busy work over the course of a project. A LOT of busy work.

Better to fix your model and get a clean bake.

Gannon · Apr 2012

@ sandro - Baking a diffuse to normal isn't going to give you the right results at all. The information is usually not what you want most of the time. You usually want a height map but then you lose out on edge quality so you compensate through extra geo on edges.

Yeah, just changing the smoothing groups can help but it doesn't always. Sometimes it come down to the geo, so without retopoing the entire thing this is just a quick hacky fix.

EarthQuake · Apr 2012

oh god, terrible terrible idea, just fix your model!

[HP] · Apr 2012

EarthQuake wrote: »

oh god, terrible terrible idea, just fix your model!

This!

You can't polish a turd.

Gannon · Apr 2012

rofl, fiiiiiiiiine.

[HP] · Apr 2012

Btw Gannon, we (I guess I can speak for us all here) are not dissing on you, or the fact that you created this thread.

You went ahead and created a thread, even wrote a guide, and show people this. That's more than 95% of us ever do for this community, so we all learned here, not only you!

Don't feel reluctant next time you wanna share possible new discoveries of workflows, etc.

r_fletch_r · Apr 2012

[HP] wrote: »

Btw Gannon, we (I guess I can speak for us all here) are not dissing on you, or the fact that you created this thread.

You went ahead and created a thread, even wrote a guide, and show people this. That's more than 95% of us ever do for this community, so we all learned here, not only you!

Don't feel reluctant next time you wanna share possible new discoveries of workflows, etc.

^ This.
Hopefully we werent over zealous. There just so much of this stuff going about that it gets frustrating.

EarthQuake · Apr 2012

Right, the biggest problem with this sort of things is they have this horrible tendency to snowball. You post some seemingly harmless bit info about something you've done, and all of a sudden we'll see countless people who don't really understand what or why you've done it repeating it because they think "gradients in normal maps are bad". Even worse, these people start popping in to other threads in tech talk and telling them to follow the advice in this thread, instead of taking the time to learn and understand WHY smoothing errors occur in the first place.

Its a constant battle of miss-information.

unstoppablex · Apr 2012

I don't know why everyone is making a huge issue. It's a good trick if you do have unwanted normal map shading. Of course you should always make sure you have the best possible bake but we all know normal maps are not always perfect, and this could be useful at some point.

i think the only problem with this thread is the title and it's approach. like many of your mention, this is the intention of normals from the bake and this isn't a general problem.

r_fletch_r · Apr 2012

unstoppablex wrote: »

I don't know why everyone is making a huge issue. It's a good trick if you do have unwanted normal map shading. Of course you should always make sure you have the best possible bake but we all know normal maps are not always perfect, and this could be useful at some point.

Its not a good trick, its a hack that tries to make up for incorrect execution.
If you model the base correctly you wont get this issue. No one should ever recommend visually editing normal map gradients.

take a little look at Earthquakes work and you will see what can be achieved with correct normal map workflow.