thoughts on using normal mapping with hand painted textures?

snake85027 · Aug 2008

Do companies use normal maps when they do hand painted textures? Me and a friend are trying to recreate the castle from gargoyles.

I want to hand paint everything just because its based on a cartoon so it makes sense to me. Even though its a cartoon it does have some beautiful detailed scenes.

Should I hand paint it and use a normal map filter?
Any suggestions would be great!

Also we are using unreal ed 3. Now most of the buildings are huge. Like the towers and the curtain walls and the gate house and all of those will have rooms and archways. So my question is do I use bsp to make the buildings or would I use repeated meshes?

I just don't know the standard when it comes to BSP versus using models from max or maya.
Here is what we have so , a very basic layout.

rooster · Aug 2008

well, the first thing that comes to my mind is team fortress 2. they have hand painted looking stuff, and some of their surfaces defainately use normal maps. I think in tf its a case of 'what will benefit from using them' rather than a blanket everything must have or everything must not have.

Kevin Albers · Aug 2008

Usually hand painted textures don't have normal maps, but there is no reason they can't. If the look you want includes normal maps, then go for it. If you want to make normal maps, hand painting them (grayscale) and using Crazybump/Nvidia filter/whatever should work fine.

Neox · Aug 2008

well normalmaps don't have to look like doom3/u3 etc. you can of course use them with more flatshaded shaders and use them to influence your overall shading

Rob Galanakis · Aug 2008

Yes, you can (and often should) use normal maps with hand-painted textures. However, simply using a default blinn or phong material will hide lots of the beauty you can achieve with hand painting. You need to come up with your own shader for rendering with hand-painted diffuse with normal maps. Lighting is just as, if not more, important to achieving that 'painterly' quality as a painted diffuse is. We do exactly this on our game and it looks gorgeous.

Joseph Silverman · Aug 2008

It depends on your art style, what looks good, what you mean by 'hand painted', etc.

Make it look good. If it looks good without normal maps, cool. If it needs normal maps, add them, and look into more effective shaders for what you want.

AstroZombie · Aug 2008

snake85027 wrote: »

Also we are using unreal ed 3. Now most of the buildings are huge. Like the towers and the curtain walls and the gate house and all of those will have rooms and archways. So my question is do I use bsp to make the buildings or would I use repeated meshes?

Use BSP to block in the basic shell of the structure and then use mesh to add details such as archways, window ledges, stairs, etc.

Also, static mesh vertex lights by default so you will need to use a 2nd UV channel if you want to have lightmaps on your static mesh pieces.

greenj2 · Aug 2008

I recently finished working on a very cartoon-stylie game where our team used normal maps generated off hand-painted diffuse maps to get some pretty decent texture effects. Although it's not a magic bullet in all cases, you can get a nice 'hand-painted' look sometimes if that's what you're after.

Also, I'm all in favour of using high-res modeling to bake normal maps, but keep in mind that it does generally take a hell of a lot longer than just using a bitmap filter.

Why not try it and post some renders in Pimping & Previews for crits? Good luck!

snake85027 · Aug 2008

Well I guess my issue is the method of combining normal maps and the cartoony or hand painted effect of the diffuse. Before normal mapping came into play I was use to having to paint in the shadows and the different elevations of the texture to get it to bump out but with normal mapping you really cant paint it the same way because it will go against what the normal mapping does.
Well this is some rock pattern that I did a few weeks ago, this isn't necessarily for the walls in this level but its was i used to experiment with. I hand painted this and then used photoshops normal map filter and then converted that to gray scale and adjusted that to get the specular mapping but as you can see it doesnt really bump out.
So here is what I was messing around with.

Thanks for the reply's. Professor420 and whiteeEagle , what would be your guy's process? Most people would create the low and then the high poly and then bake an Ambient Occlusion map and use that mapping to start your diffuse texture. What method do you guys or anyone else reading this, when it comes to something cartoony look that I am trying to go for?

Thanks for your time.

EarthQuake · Aug 2008

The big problem here, is that you have all of the same detail in the diffuse, normals, etc. You have to change the way you're thinking. You used to paint in all the directional lighting into your diffuse textures, thats fine but you need to let the normals do that work now. Really the main element needs to be the detail in your normals, if you can create nice convincing shapes there, all you need is some simple color and ambocc in your diffuse map. A really good example here is this model by BoBo, the detail is great, a nice cartoony and stylized sculpt, with very minimal color work gives it a great "illustrated" or "painted" look.

http://boards.polycount.net/showthread.php?p=837088#post837088

I would experiment a bit with either starting to paint heightmaps(completely forget about how you paint your diffuse, its not going to give good results for normals) or try sculping some simple stuff in mud/z or whatever. Just because something has a highpoly source to bake normals from doesn't mean it automatically is/needs to be crazy ultra noise detailed photo-real crap. Keep the shapes simple in your highres/bump map and experiment with a painterly style.

sebas · Aug 2008

'Overlord' uses hand painted textures with normal maps...
http://pc.gamespy.com/dor/objects/825968/overlord/images/overlord-20070419111428090.html
Oh and don't miss this tutorial...
http://caponeart.blogspot.com/2007/09/flashback-no3-overlord-ancient-ruin.html

Hope it helps!

bugo · Aug 2008

here´s an example of normal map and handpainted diffuse I did for Freaky Creatures game
http://hugobeyer.carbonmade.com/projects/2080563#3

check the next image in a way you can see the diffuse one with other handpainted textures, I also used crazybump after making these to create my nmaps.

I hope it helps.

Xenobond · Aug 2008

You could also go into the red/green channels and add some variation to the separate chips you've got on that floor texture. You wouldn't want to muck with it too much, though. It;ll really get noticeable the more it tiles.

demoncage · Aug 2008

all depends on the lighting. normal maps on their own are best fit for minimal/dynamic lighting situations. the more diffuse the lighting is in the environment, the more you would have to augment the depth in your normal maps with additional depth information in your diffuse maps.

as far as the example you posted converting your painted textures from diffuse to normal, you have to think in terms of bump mapping. light vs dark = high vs low on the z axis. that's why the rim hilights around your cracks in the diffuse look wrong in the normal map, because the conversion will just raise that white area against the grey behind it. hilights have no place in a greyscale image intended for normal map conversion.

unless those edges are intended to be raised that high, in which case ignore me.

thoughts on using normal mapping with hand painted textures?

Replies