Home Technical Talk

Tutorial: Capturing Normal and Displacement Maps

polycounter lvl 18
Offline / Send Message
Ryan Clark polycounter lvl 18
I've written a tutorial on a method of capturing normal and displacement maps from real objects.

It's easy to do, and requires no special hardware. You just need a camera and a few minutes to spare.

The tutorial is located at http://www.zarria.net

Feedback and comments would be appreciated.

Replies

  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Haha! Great idea! Thanks for the tutorial!
  • poopinmymouth
    Offline / Send Message
    poopinmymouth polycounter lvl 19
    Ryan is the insanely tallented programmer behind the rendering engine at warthog texas (my first job) He is a great guy, and I am pretty sure he first used this process almost 2 years ago. The tutorial has just been a little long in coming. 8-)
  • kdub
    Offline / Send Message
    kdub polycounter lvl 18
    that is awesome. how well does it work with something like an action figure?
  • Eric Chadwick
    This is intriguing to me, since I've had some talks with graphics programmers about how to do this conversion, and we always came away with the conclusion that it was too inaccurate. I was crossing my fingers here, hoping your tool would prove all that wrong.

    I have a quickie brain model lying around, that was made procedurally (a 3ds max geosphere, with Noise, Push, and Relax modifiers). Not really a character model, but it's a good way for me to see what happens.

    brain_highres_render.gif High-res model.

    brain_highres_baked.gif Baked from the model into a UV layout.

    brain_tangent-space.gif Tangent-space normal map, baked from geometry.

    brain_world-space.gif World-space normal map, baked from geometry.

    brain_displacement.gif Displacement map, baked from geometry.

    brain_tangent-to-displacement.gif Displacement generated from tangent map.

    brain_world-to-displacement.gif Displacement generated from world map.


    Seems the exe prefers world maps, there's less distortion. But I'm not sure why it has trouble deriving the valleys vs. the hills. Seems like many of the creases are being created as lighter values. I guess that depends on where you start the pixel-walk. I'd love to see more work put into this, if you're willing. This could prove very useful.

    Thank you for taking the time to write up the tutorial, and for sharing the exe.

    I saw someone do the normal-map-extraction technique with a flatbed scanner about a year or so ago, turning a coin 90 degrees for each scan. He posted examples on the Discreet webboard. Same idea, but with your setup it's a bit easier.

    It seems to me you'd get more definition in your normal map if you combined some of the angles into the blue channel, since it seems to help when blue has some variation in it.

    Can you talk a bit about how you derive the displacement map? Do you start with the pixel value in one corner and walk across the bitmap, storing the angular changes as darker or lighter values? Do you normalize the heights afterwards?

    Here's a similar capture method, getting displacement instead.
    http://www.cee.hw.ac.uk/texturelab/projects/virtex/demo/bmpcapture.html
    Another method, using stereo pairs.
    http://www.virtualclones.com/

    Thanks again for your post. Cool stuff!
  • pior
    Offline / Send Message
    pior grand marshal polycounter
    Hi Ryan!

    I saw someone doing that with a flat scanner and a coin on the spiraloid forums too... Great and esy technique smile.gif

    Funny, I tried the exact same thing for normalmaps rendering some weeks ago. I used a little plastic skullas base and it worked surprisingly well...

    I guess this is a very good way to build up a texture collection, things like fabric, scales, rocks, aso.

    Great tut!
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    Hi Eric

    Thanks for giving Displacement Map Creator a test!

    Your map confused the software for two reasons:

    1. Your green channel (Y-axis) is inverse of what DMC expects. Certain programs bake that way, for some reason. What software did you use to create it?

    2. Your black background will confuse DMC; DMC will think the black is a VERY sharp angle on the normal map.

    I altered your map to meet DMC's expectation, and here's the result:

    brain_tangent_space_FIXED.jpg

    brain_tangent-space_DISPLACEMENT.jpg

    Thanks again for testing DMC. Rest assured a new version will soon be forthcoming, with documentation and a control panel.
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    In response to your question about how DMC works:

    It's an iterative algorithm. Starting with a flat map, each pixel calculates "flow" from its neighbors, based on each neighbor's slope and current height. All pixels are adjusted very slightly toward their neighbors' expectations, and then the process repeats.

    After a few hundred iterations, this yields a pretty good displacement map, which gets normalized to an 8-bit range, and written to disc.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Ah, that makes sense. I asked my brother to write a program to do this last year, so I could create greyscale bump maps from a normal-map (I was using ORB at the time, and back in that version it couldn't generate bump maps, i think).
    It just did it all in one pass though, which is why it ended up getting about half of the map right before descending into horizontal-streaky-line madness.
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    I tried that approach first: walking the map in a single pass and ending up with a streaky mess.

    I discovered several excellent algorithms for streaky-mess-generation while writing DMC =)
  • Eric Chadwick
    Awesome!

    It seems like DMC is chucking some of the fine detail from the normal map, which is good in this case because the brain one is pretty noisy. I wonder how it works with better source? wink.gif Do any fine-grain bumps just disappear?

    It would be helpful if the control panel had an option to choose the Y direction. Our engine is currently setup to expect ATI's Y convention over Nvidia's. I generated the maps with Kaldera, which lets us choose either method.

    I found the DMC map had some clipping at the extreme ends of the scale, giving me flat little plateaus and plains. I wonder if this is adjustable?

    Thanks again for a great tool. This will be fun to play with some more.
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    Thanks for the suggestions!

    Next version will have y-toggling and backdrop detection. It will also have improvement in capturing detail, possibly involving a user-adjustable slider.
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    Version 2 is ready. Has y-toggling, backdrop detection, and significant refinement of the algorithm.

    Grab it from www.zarria.net, and please continue to give suggestions. DMC has plenty of room to grow yet.

    Here's what V2 does with your brain:
    brain_tangent-space_DISPLACEMENT2.jpg
  • Eric Chadwick
    Got a big deadline, won't be able to play with it until later this week, but thanks for the update. The tool is much appreciated, very cool.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Cool stuff, nice update! I might give it a try on my next normal-mapped model...
  • Faucet
    Offline / Send Message
    Faucet polycounter lvl 18
    Extremely cool, Ryan!

    I can't wait to try some of these techniques out!
  • Snowfly
    Offline / Send Message
    Snowfly polycounter lvl 18
    now the max payne artists have a technique to use for the next sequel! but that is seriously cool stuff
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    I just finished a new example: normal and displacement captured from decorative-wall-hanging-cross. These started with better source photographs than the peanuts, and used the improved version of DMC.

    screenshots:
    http://66.70.170.53/Ryan/nrmphoto/crossrenders3.jpg
    http://66.70.170.53/Ryan/nrmphoto/crossrenders%20copy.jpg
    http://66.70.170.53/Ryan/nrmphoto/crossrenders%20copy2.jpg

    captured maps:
    http://66.70.170.53/Ryan/nrmphoto/crossColor.jpg
    http://66.70.170.53/Ryan/nrmphoto/crossnrm.jpg
    http://66.70.170.53/Ryan/nrmphoto/cross_DISPLACEMENT.jpg
  • doc rob
    Offline / Send Message
    doc rob polycounter lvl 19
    thanks a bunch, Ryan. This is extremely cool!
  • kleinluka
    Offline / Send Message
    kleinluka polycounter lvl 18
    Very cool. thanks a lot! I wouldn't have thought the end results would turn out that well!
  • Psynex
    Offline / Send Message
    Psynex polycounter lvl 18
    Where in this process does tiling fall?
    I'd love to take advantage of a technique like this, but lighting from four different directions would make tiling a pain.
    Any steps on this?
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    Thanks for the suggestion! I should add an optional "tiling" step when I find the time.

    Best way I've found is to capture the normal map first, then modify it for tiling with the clone brush.

    You can record the tiling of your color map, and play back the same action on your normal map. A highpass filter before tiling is often helpful.

    I've posted actions that flip and rotate normal maps while transforming the normals to compensate. They make it easy to tile-by-mirroring, at least.

    DMC does take tiling to account. So if your normal map tiles, your displacement map will also tile.
  • Psynex
    Offline / Send Message
    Psynex polycounter lvl 18
    [ QUOTE ]

    I've posted actions that flip and rotate normal maps while transforming the normals to compensate. They make it easy to tile-by-mirroring, at least.


    [/ QUOTE ]

    What exactly do you mean? Is that like the high-pass filter method where you chop the image into like 4 sections and move them around, patch the seems and that sort of thing.
    I've seen it at cokane.com? Chris O'Kane? His website.
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    Conor O'Kane's tutorial looks excellent. Thanks for the link!

    The difference here is that you must apply the same steps to both color map and normal map.

    One difficulty is that "flip" and "rotate" commands can't be used on a Normal Map. Orientation is important with normals; if you change orientation, you need to change color accordingly. For example, a vertical flip must be followed by inversion of the green channel. The actions on my site let you flip or rotate the normal map in a single step.

    so, your workflow would look something like this:

    1. Open your color map, and begin recording a new action.
    2. Tile your color map, follwing the example of Conor O'Kane's tutorial.
    3. Stop recording your action.
    4. Look through the steps of your action, and replace each "flip" or "rotate" command with the corresponding action from my set.
    5. Open your normal map, and playback your modified tile-making steps.
  • KDR_11k
    Offline / Send Message
    KDR_11k polycounter lvl 18
    His name is Conor O'Kane, not Chris...
  • Ryan Clark
    Offline / Send Message
    Ryan Clark polycounter lvl 18
    Right you are. Thanks!
  • Eric Chadwick
    A few more tiling tutorials in the same vein, adding some more refinements...

    Matt Vainio's Texturing Tutorial
    http://www.twisted-strand.com/ut_tutorials/text_tut/index.html

    Peter Hajba's Power of the High Pass Filter
    http://www.3dgate.com/techniques/2001/010625/0625hajba.html

    Jeremy Birn's Fixing Lighting Irregulaties in Self-Tiling Maps
    http://www.3drender.com/light/EqTutorial/tiling.htm
  • Mark Dygert
    Why is this not a sticky yet? confused.gif I can't thank all of you enough for giving me great info to chew on and throughly confusing me. The more I start to wrap my head about these new ways of doing textures the more I find my brain hurting. I am starting to understand them more and more but the more I learn, the less it works when I try to apply it to a model.

    All of this normal map / displacement map stuff doesn't seem natural to me as an artist. It all makes sense, but is damn hard to apply correctly when the artist in me takes over. I am sure that is because I haven't played around with it much, and or it is so new they haven't made it WYSIWYG enough for me yet. Believe it or not, this does help quite a bit, and is a brilliant idea, I just can't really use it that well for the kind of textures I am working on right now. I will squirl this one away for a rainy day =)
  • valancedom
    Offline / Send Message
    valancedom polycounter lvl 18
    Does anyone have some tutorials for using Normal maps in maya. As in applying them to your models and rendering them on models? Thanks!
  • Eric Chadwick
Sign In or Register to comment.