Tutorial: Capturing Normal and Displacement Maps

Ryan Clark

I've written a tutorial on a method of capturing normal and displacement maps from real objects.

It's easy to do, and requires no special hardware. You just need a camera and a few minutes to spare.

The tutorial is located at http://www.zarria.net

Feedback and comments would be appreciated.

MoP

Haha! Great idea! Thanks for the tutorial!

poopinmymouth

Ryan is the insanely tallented programmer behind the rendering engine at warthog texas (my first job) He is a great guy, and I am pretty sure he first used this process almost 2 years ago. The tutorial has just been a little long in coming. 8-)

kdub

that is awesome. how well does it work with something like an action figure?

Eric Chadwick

This is intriguing to me, since I've had some talks with graphics programmers about how to do this conversion, and we always came away with the conclusion that it was too inaccurate. I was crossing my fingers here, hoping your tool would prove all that wrong.

I have a quickie brain model lying around, that was made procedurally (a 3ds max geosphere, with Noise, Push, and Relax modifiers). Not really a character model, but it's a good way for me to see what happens.

High-res model.

Baked from the model into a UV layout.

Tangent-space normal map, baked from geometry.

World-space normal map, baked from geometry.

Displacement map, baked from geometry.

Displacement generated from tangent map.

Displacement generated from world map.


Seems the exe prefers world maps, there's less distortion. But I'm not sure why it has trouble deriving the valleys vs. the hills. Seems like many of the creases are being created as lighter values. I guess that depends on where you start the pixel-walk. I'd love to see more work put into this, if you're willing. This could prove very useful.

Thank you for taking the time to write up the tutorial, and for sharing the exe.

I saw someone do the normal-map-extraction technique with a flatbed scanner about a year or so ago, turning a coin 90 degrees for each scan. He posted examples on the Discreet webboard. Same idea, but with your setup it's a bit easier.

It seems to me you'd get more definition in your normal map if you combined some of the angles into the blue channel, since it seems to help when blue has some variation in it.

Can you talk a bit about how you derive the displacement map? Do you start with the pixel value in one corner and walk across the bitmap, storing the angular changes as darker or lighter values? Do you normalize the heights afterwards?

Here's a similar capture method, getting displacement instead.
http://www.cee.hw.ac.uk/texturelab/projects/virtex/demo/bmpcapture.html
Another method, using stereo pairs.
http://www.virtualclones.com/

Thanks again for your post. Cool stuff!

pior

Hi Ryan!

I saw someone doing that with a flat scanner and a coin on the spiraloid forums too... Great and esy technique

Funny, I tried the exact same thing for normalmaps rendering some weeks ago. I used a little plastic skullas base and it worked surprisingly well...

I guess this is a very good way to build up a texture collection, things like fabric, scales, rocks, aso.

Great tut!

Ryan Clark

Hi Eric

Thanks for giving Displacement Map Creator a test!

Your map confused the software for two reasons:

1. Your green channel (Y-axis) is inverse of what DMC expects. Certain programs bake that way, for some reason. What software did you use to create it?

2. Your black background will confuse DMC; DMC will think the black is a VERY sharp angle on the normal map.

I altered your map to meet DMC's expectation, and here's the result:





Thanks again for testing DMC. Rest assured a new version will soon be forthcoming, with documentation and a control panel.

Ryan Clark

In response to your question about how DMC works:

It's an iterative algorithm. Starting with a flat map, each pixel calculates "flow" from its neighbors, based on each neighbor's slope and current height. All pixels are adjusted very slightly toward their neighbors' expectations, and then the process repeats.

After a few hundred iterations, this yields a pretty good displacement map, which gets normalized to an 8-bit range, and written to disc.

MoP

Ah, that makes sense. I asked my brother to write a program to do this last year, so I could create greyscale bump maps from a normal-map (I was using ORB at the time, and back in that version it couldn't generate bump maps, i think).
It just did it all in one pass though, which is why it ended up getting about half of the map right before descending into horizontal-streaky-line madness.

Ryan Clark

I tried that approach first: walking the map in a single pass and ending up with a streaky mess.

I discovered several excellent algorithms for streaky-mess-generation while writing DMC

Eric Chadwick

Awesome!

It seems like DMC is chucking some of the fine detail from the normal map, which is good in this case because the brain one is pretty noisy. I wonder how it works with better source? Do any fine-grain bumps just disappear?

It would be helpful if the control panel had an option to choose the Y direction. Our engine is currently setup to expect ATI's Y convention over Nvidia's. I generated the maps with Kaldera, which lets us choose either method.

I found the DMC map had some clipping at the extreme ends of the scale, giving me flat little plateaus and plains. I wonder if this is adjustable?

Thanks again for a great tool. This will be fun to play with some more.

Ryan Clark

Thanks for the suggestions!

Next version will have y-toggling and backdrop detection. It will also have improvement in capturing detail, possibly involving a user-adjustable slider.

Ryan Clark

Version 2 is ready. Has y-toggling, backdrop detection, and significant refinement of the algorithm.

Grab it from www.zarria.net, and please continue to give suggestions. DMC has plenty of room to grow yet.

Here's what V2 does with your brain:

Eric Chadwick

Got a big deadline, won't be able to play with it until later this week, but thanks for the update. The tool is much appreciated, very cool.

MoP

Cool stuff, nice update! I might give it a try on my next normal-mapped model...

Faucet

Extremely cool, Ryan!

I can't wait to try some of these techniques out!

Snowfly

now the max payne artists have a technique to use for the next sequel! but that is seriously cool stuff

Ryan Clark

I just finished a new example: normal and displacement captured from decorative-wall-hanging-cross. These started with better source photographs than the peanuts, and used the improved version of DMC.

screenshots:
http://66.70.170.53/Ryan/nrmphoto/crossrenders3.jpg
http://66.70.170.53/Ryan/nrmphoto/crossrenders%20copy.jpg
http://66.70.170.53/Ryan/nrmphoto/crossrenders%20copy2.jpg

captured maps:
http://66.70.170.53/Ryan/nrmphoto/crossColor.jpg
http://66.70.170.53/Ryan/nrmphoto/crossnrm.jpg
http://66.70.170.53/Ryan/nrmphoto/cross_DISPLACEMENT.jpg

doc rob

thanks a bunch, Ryan. This is extremely cool!

kleinluka

Very cool. thanks a lot! I wouldn't have thought the end results would turn out that well!

Psynex

Where in this process does tiling fall?
I'd love to take advantage of a technique like this, but lighting from four different directions would make tiling a pain.
Any steps on this?

Ryan Clark

Thanks for the suggestion! I should add an optional "tiling" step when I find the time.

Best way I've found is to capture the normal map first, then modify it for tiling with the clone brush.

You can record the tiling of your color map, and play back the same action on your normal map. A highpass filter before tiling is often helpful.

I've posted actions that flip and rotate normal maps while transforming the normals to compensate. They make it easy to tile-by-mirroring, at least.

DMC does take tiling to account. So if your normal map tiles, your displacement map will also tile.

Psynex

[ QUOTE ]

I've posted actions that flip and rotate normal maps while transforming the normals to compensate. They make it easy to tile-by-mirroring, at least.


[/ QUOTE ]

What exactly do you mean? Is that like the high-pass filter method where you chop the image into like 4 sections and move them around, patch the seems and that sort of thing.
I've seen it at cokane.com? Chris O'Kane? His website.

Ryan Clark

Conor O'Kane's tutorial looks excellent. Thanks for the link!

The difference here is that you must apply the same steps to both color map and normal map.

One difficulty is that "flip" and "rotate" commands can't be used on a Normal Map. Orientation is important with normals; if you change orientation, you need to change color accordingly. For example, a vertical flip must be followed by inversion of the green channel. The actions on my site let you flip or rotate the normal map in a single step.

so, your workflow would look something like this:

1. Open your color map, and begin recording a new action.
2. Tile your color map, follwing the example of Conor O'Kane's tutorial.
3. Stop recording your action.
4. Look through the steps of your action, and replace each "flip" or "rotate" command with the corresponding action from my set.
5. Open your normal map, and playback your modified tile-making steps.

KDR_11k

His name is Conor O'Kane, not Chris...

Ryan Clark

Right you are. Thanks!

Eric Chadwick

A few more tiling tutorials in the same vein, adding some more refinements...

Matt Vainio's Texturing Tutorial
http://www.twisted-strand.com/ut_tutorials/text_tut/index.html

Peter Hajba's Power of the High Pass Filter
http://www.3dgate.com/techniques/2001/010625/0625hajba.html

Jeremy Birn's Fixing Lighting Irregulaties in Self-Tiling Maps
http://www.3drender.com/light/EqTutorial/tiling.htm

Mark Dygert

Why is this not a sticky yet? I can't thank all of you enough for giving me great info to chew on and throughly confusing me. The more I start to wrap my head about these new ways of doing textures the more I find my brain hurting. I am starting to understand them more and more but the more I learn, the less it works when I try to apply it to a model.

All of this normal map / displacement map stuff doesn't seem natural to me as an artist. It all makes sense, but is damn hard to apply correctly when the artist in me takes over. I am sure that is because I haven't played around with it much, and or it is so new they haven't made it WYSIWYG enough for me yet. Believe it or not, this does help quite a bit, and is a brilliant idea, I just can't really use it that well for the kind of textures I am working on right now. I will squirl this one away for a rainy day

valancedom

Does anyone have some tutorials for using Normal maps in maya. As in applying them to your models and rendering them on models? Thanks!

Eric Chadwick

A bit off-topic there.

But this might help.
http://www.jeffparrott.com/normalmaptut_01.html