Matching Diffuse to Normal maps

Microneezia

Hello.

This is for Environment modeling... I dont care much for character stuff, but if the technique transfers, great. I was wondering about workflow surrounding this subject. Ive seen diffuse textures line up with incredible accuracy and detail to corresponding normal maps and I am unsure of how exactly this workflow is handled in general..

My first thought is desaturate the normal map in PS, bring opacity down and just work hard with a brush on layer above it? But I wonder if this is common practice.

Maybe environment artists are using RTT a lot or something with the high poly and vector painting? hmm wait that wouldnt line up with the low poly UV's? hmm...

Thanks for any help!

Joshua Stubbles

I don't think I understand where you're going with this. If you make a normal map from a texture it's per-pixel. It should match the texture 100%. If you're baking the normal map from a hires mesh, it's baking down to the same UV's as the lowpoly already has. So again, it's a 100% match.

Are you talking about overlays? Adding additional detail to tangent-space normal maps? If so, then what you said is pretty close. Slap the normal in a layer, lower opac, paint your bump, convert it, overlay after adjusting B chan output, normalize, done.

alexk

when you bake from a high poly, assign a material and a different color to the high poly mesh and it's seperate peices. then in RTT, along with the normal map, bake out a diffuse. It'll be just plain colors that you assigned and then in photoshop you can magic wand on those colors to create your masks

Joshua Stubbles

Also alex, if you use Texporter for your UV templates, you can do the same thing by saving out solid-fill in the options. Although you don't get color in the template, the mask usability is the same.

Mark Dygert

To expand on that. You can also record a photoshop action that quickly separates the fill color from the edge color and places them on different layers. If you do this I suggest using tga's because they are uncompressed and its a pixel perfect match. I also suggest using the action to grow the fill color a few pixels around each UV piece that was you start of with a little padding.

Ghostscape

alexk wrote: »

when you bake from a high poly, assign a material and a different color to the high poly mesh and it's seperate peices. then in RTT, along with the normal map, bake out a diffuse. It'll be just plain colors that you assigned and then in photoshop you can magic wand on those colors to create your masks

This is what I do, except I use R, G, and B (and sometimes CMY if its a very complex piece, although this means I lose some fidelity) since I can use each channel as a mask and get better looking edges than the magic wand. It's especially useful for very fine details.

Once I get into photoshop I run an action that automatically sets up my PSD for me, takes my normal, fills the background color with normal blue, puts it in a group, puts my AO in a group underneath it and sets that group to multiply, and makes a "selectors" group with the r/g/b channels of my baked diffuse map into layers so I can ctrl-click the thumbs to load them up.

alexk

can you guys expand a little more on how to make a action that seperates the diffuse blockouts, puts AO and normal in its own groups, ect ect. Are you guys recording several seperate actions that do this, or just one that does it all? how does photoshop know to seperate things like diffuse block outs if each time it's different for each model?

Ryan Clark

Hey Microneezia,

CrazyBump has a feature that should help with that. Try these steps:

1. Open a normal map in CrazyBump. Make sure CB's axis setting is correct.

2. Save CrazyBump's automatic "Diffuse" map.

3. Paint over it in photoshop.


CrazyBump's diffuse will give you a much better starting point than just desaturating your normals.

Microneezia

Thanks everyone, I think I get how its being done commonly now...

@vassago - ya I was talking about high poly normal > diffuse - lining up the details in the maps... its hard to get my head around right now while looking at all this ut3 mech stuff. But I guess its just done as they say here by baking color masks with RTT... I think that is what I was reaching for.

@alex - hey man thanks I hadnt heard that before... it makes sense, the floating details will be really easy to throw a different color on, and also easy to magic wand then, as you point out. Thanks!

@vig - i dont understand this: "separates the fill color from the edge color" how is the edge color different? what edge?

@ghost - holy crap. ok sounds pro. I understand now. except how the mask is better... like if you select the color chanel its a more accute selection than the wand selecting that color? What if I use "Select color range" from the menu bar, eyedropper the color, and cut paste into a new layer. Is the color channels still more accurate?

Ryan, that sounds like a good idea, Im a gonna check out CB with that pretense for sure!

Thanks everyone, my question was vauge and i got specific answers!

Ghostscape

alexk wrote: »

can you guys expand a little more on how to make a action that seperates the diffuse blockouts, puts AO and normal in its own groups, ect ect. Are you guys recording several seperate actions that do this, or just one that does it all? how does photoshop know to seperate things like diffuse block outs if each time it's different for each model?

My action (named 'normal, AO, Selection' so I remember what order to paste the layers into), assumes I made my diffuse out of pure R,G,B and C, M, Y. it goes to the channels palette, selects the red channel, subtracts the blue and green channels (if I was using CMY as well), and makes a new layer and fills it red. Then it goes back to my selection layer, loads the blue channel, does the same, etc.

Mark Dygert

@vig - i dont understand this: "separates the fill color from the edge color" how is the edge color different? what edge?

There is a second option toward the bottom of the Render Template option for the boarder around each piece.

Here is the action I use its bound to F12, it uses these settings.
So now all you do is render your UV template in 3ds, save it as TGA, open it in PhotoShop, hit F12 and Bob's your uncle!

As for separating out different colors and materials, that could get tricky and it should be easily scriptable and I'd be surprised if someone hasn't already done it. Check scriptspot...

Mark Dygert

Opps double post...

Microneezia

Oh, vig I get it now like you mean create a guide from the UV islands? I get it. What I did previously was make the border white and fill black bacground. then copy it PS, invert, select all white (backdrop) cut. this gave me a uv guide to float on top of what I was working on. Is this what you mean? Ill just try your action and see I guess.

Thanks, im good now I think lol.