Blending Hi-poly Normal Maps with 2D Normal Maps

oXYnary

To make this simple. Think of pores on a normal mapped face. Something you dont want to try to spend the time creating on the hi-poly master.

So after creating a high-poly character and rendering the normal map unto the low poly version.(3D) I was shown a method of then creating items like pores with just black areas with a nvidia normal plug in tool (2D).

(Im using 2D vs 3D after this point to separate the difference)

The next step I was given was then to do a light transparency layer of the "2d" normal map over the 3d version for the final. The question I have, or more a problem Im seeing is this.

Since the 2d version has only one static background depth, while the 3d version has multiple depths. Well, wont this create some areas from the 2d version that are raised to high or too low on this mixed version?

In example in a concave area, the 2d normal map in the mixed version would be more apparent in the "mid" portion of the curve. At the top or bottom of the concave it would be too apparent, or too lacking if you simply did a transparency. It also would effect the 3d versions overall depth as you are again adding a one color background over something that isn't.

Am I missing something or another method of combining them that would make the 2d version "follow" the 3d versions depth? (Other than say zbrush)

pyromania

Why not convert the 2d normal map to a 3d normal map with nvidias photoshop plugin. Then layer the new 2d/3d normal map over the existing 3d normal map. Pior had a explanation of this on the old boards. I think you just had to use photoshops Overlay layer blending.

pior

I don't know if its right from the pure mathematical point of view, but at least it seems to work rather well :



Hope this helps!

oXYnary

Thats what I was describing (or attempting to). Is the overlay function what I need to retain the 3d versions depth?

PS description of the overlay:
[ QUOTE ]
Overlay

Multiplies or screens the colors, depending on the base color. Patterns or colors overlay the existing pixels while preserving the highlights and shadows of the base color. The base color is not replaced but is mixed with the blend color to reflect the lightness or darkness of the original color.

[/ QUOTE ]

So is the base color being sampled from the layer below or the layer the overlay is on itself? "base color is not replaced but is mixed with the blend color" Which still seems to imply a mixture happening versus just using the orginal layers depth?

Edit: Stealing your image pior for future reference.. MU HA HA (thanks)

MoP

Yep, if you Overlay, merge down the layers, then Normalize (with NVidia plugin), that should do the trick.

Gmanx

Could you put the 'fine' detail (like pores or whatever) on a separate bump map (if your renderer supports it)? I've heard of that being done in areas which don't require that much Z depth, and the bump map would kinda lay over your 3d normal map, following it's contours.

oXYnary

MoP, BUT as Pior implies.. Is this truly the correct method as we see the overlay does still blend all around (at least how I interpret it).

Thats what Im trying to figure out.

Gmanx, that was my original plan, but most engines only support one or the other. The one in specific Im thinking of, only does normal.

pyromania

If you wanted it to be completly mathmatically accurate, I guess you could blend the individual RGB channels manually. Gently erasing and blending one channel at a time, then normalize the whole image. Overlay seems like a much quicker way though. As long as it looks good why not use it.

JKMakowka

[ QUOTE ]
Gmanx, that was my original plan, but most engines only support one or the other. The one in specific Im thinking of, only does normal.

[/ QUOTE ]

Disclaimer: I haven't done this myself yet, but I was told it works 100%

With ORB you can just put a (handpainted) bumpmap on the low(!!no need to uv map the high!!) poly model before rendering the normal-map. The bumpmap is then automaticly included in the normalmap (your normalmap needs enougth resolution of course).
In the end you have just one normalmap but the extra detail from the bumpmap is included.

KDR_11k

Be warned: If you overlay the resulting normalmap, level adjust the blue channel to go from 0-127 instead of 0-255. Remember, you're adding vectors here and you'd want the completely flat areas (that just use the normal from the hipoly map) to have a 0 0 0 vector instead of a 0 0 1 because a 0 0 0 vector doesn't alter the existing vector upon add and normalize.

Pior: When I did the detail pass on Ryoka I used a normalmap generator material (one that represents the surface normals as colors, using three gradients with texture coordinates set to normals). That's more accurate and saves you the NVidia pass. It also allows you to resume a surface on another height since it doesn't care about height, just the normals. I've faked a lot of stuff that way.

Compare:
Perspective, using the diffuse colors, shadowing and ambient occlusion to show the trickery:


Ortho, normalcolor texture and fullbright, used as the detail normalmap (after a levels -> B 0-127 pass):


Note how the height discontinuities in the first render don't show up in the second?

Whargoul

If you want to blend two normal maps together:

1. Half the intensity of your blue channel on the top layer (make it go to 128 max instead of 256)
2. Set top layer to overlay
3. Merge layers
4. Renormalize

Works perfectly for me. If you forget to lower the intensity of the blue layer, you'll destroy any info from your bottom layer of the original (in the 2d case, it's almost always pure white, which replaces your generated map which will have important values in there). Putting it to 128 will preserve it.

[edit] shit, just noticed kdr basically pointed out the same thing

and fixed it from screen to overlay[/edit]

MoP

Ahh, cool tip Whargoul. That sounds like the perfect method.

Also, in Max7, if you apply a bump-map to the highpoly mesh (procedural textures work well), then render that down to a low-poly model's normal-map, it includes the bump information in the normal render, which is nice.

Then again, if what Mr Makowka says is true, ORB has an even better method.

oXYnary

Auh! Thought there was more to it. Thanks all. Though I wonder Mop, with Max7, have you tried a bump map on the low poly before normal mapping to see if that works also (mostly because ORB is such a bitch for me to use, and its edges aren't as "congruent")

MoP

no, i haven't tried that, but i don't think it'd work, since Max casts rays through the highpoly mesh onto the low-poly mesh... and Render to Texture just makes an image based on those cast rays. I don't think you could blend a rendered texture with one applied to the low-poly model.
Anyway, if you did that, you might as well just go the Photoshop route, since you'd have a painted greyscale map anyway. I was mainly thinking of this technique as using procedural textures to add quick repetitive bumpiness detail (like for cast plastic, or beaten copper etc.) ... much faster than painting it all by hand.

KDR_11k

Whargoul: Doesn't screen only brighten colors? Did I understand something wrong or is it overlay that you want? AFAIK that adds the difference between the color and 127 to the underlying image, which would be the vector addition we're trying to archieve. Or did I get that wrong?

pior

Woah tons of info here, that's just great!

Yeah the method I posted (some months ago) might not be 100% accurate but since there is always a lot of tweaking to be done, I think that it's the end result displayed on the model that matters

Thanks for the levels trick, sounds good.

KDR, this 'normalmap generator material' sounds like a great tool. Is that app-specific ? (I believe you use Blender?) I guess it's also doable by doing a Max7 render to texture with rays launched from a top-down flat plane ... Or by going the oldschool way, with 3 R,G,B lights around the objects to be normalrendered, with no shadowing and no light distance attenuation either.

Hmmm can't wait to try this out

StrangeFate

dunno if someone already said it. Painting in the stuff wont always work as the painted stuff is converted to normal maps as if it was on a flat surface (which it is in 2D).
If you as example wanted a seam that goes through a large part of a model it would have to change the colors depending on the direction the faces are pointing, just like the baked normal map, which it wont do.

So while on some parts of a model a detail might look beveled, on other areas it will look inverted, or a mix of both. It's usually good for local details in areas but any extensive stuff would require retouching of colors in a reas or inverting some of the colors completely.

Ryno

Or Displacement before normal map extraction.

Or just Z-Brush the whole thing, pores and all!

KDR_11k

StrangeFate: Nope. At least Doom 3 doesn't require that. On the map I posted the legs are oriented sideways, yet those details are lit properly. ORB and Doom 3 output those areas with rotated normals as well.

pior: In Blender you can define the maping coordinate source for a texture, for the normalmap material you need to use "normals" as the source and map X to U for red, Y to U for green and Z to U for blue. The textures are gradients (special texture type) that go from rgb 0 0 0 to the corresponding color. All channels are set to additive and the material to shadeless. I'm sure your app can do something similar. Lights might not work because the lighting algorythms would include the other two coordinates of the normal vector and because it might be hard to get an exact gradient from 0-255 going.

pior

I guess that's the technique you used KDR:

http://www.blender3d.com/cms/Normal_Maps.491.0.html

Maybe a bit complicated to set up but the results seem nice and crisp. Too bad that Blender can't display tangent-space maps properly tho.

I guess that a procedural texture could do that in max too... Got to try this out

Eric Chadwick

pior,
I guess you know about the lighting method.
http://www.pinwire.com/article82.html
I've used this, but it's view-dependant.

Procedurally I think you could setup a combo of Falloff maps, which I guess in theory could be rendered properly in UV space with Render To Texture... kind of a cool idea.

pior

Haaa thanks Eric I remember this page but lost the link I was especially unsure about the negative lights thing, it's now top clear

I don't know much about procedural textures and now than I think of it i believe it might be a bit hard to setup.

Hey by the way KDR I've tried to use a flat plane as a source in order to get normalmap details from a complex scene and it worked wonderfully well, exactly like what you showed in the shadowed/normal comparaison pic. Really easy a technique, very fast and easy to setup in Max7. I'm loving it

Hence the technique I explained in the minitut is only worth it for apps with no render to texture/baking options... Aw, time for a tut update

oXYnary

Post a link in here when done pior.

Whargoul

Yeah, sorry I meant overlay mode not screen.