Skin Shader - Realtime Translucency
polycounter lvl 18
Haven't posted in ages, so I thought it was high time I did 
Recently I've been tinkering around with hlsl, with the goal of learning to hold my own when it comes to writing shaders.
To that end, first up I'm developing a skin shader to mimic highend rendering materials, and physical properties (eg, translucency, fresnal specular, oiliness, etc). Anyways, enough words, here's my WIP:
Shader:
1 pass, pixel shader 2.0
tested on a geforce fx 5200 and a quadro fx 2000
no idea how to benchmark these properly, but it runs smooooooooth full screen @ 1280x960
Asset:
I call him 'Squishy' and he goes *BOING*
~1100 polys,
no diffuse
no specular
512 normal map
256 full-white (eg, 100% on) temporary tex used in the translucency falloff and power slots
I'm also keeping a bit of a progression log which has a few earlier shots and a mini-diary detailing the progression of the shaders on my webpage - find it here.
Recently I've been tinkering around with hlsl, with the goal of learning to hold my own when it comes to writing shaders.
To that end, first up I'm developing a skin shader to mimic highend rendering materials, and physical properties (eg, translucency, fresnal specular, oiliness, etc). Anyways, enough words, here's my WIP:
Shader:
1 pass, pixel shader 2.0
tested on a geforce fx 5200 and a quadro fx 2000
no idea how to benchmark these properly, but it runs smooooooooth full screen @ 1280x960
Asset:
I call him 'Squishy' and he goes *BOING*
~1100 polys,
no diffuse
no specular
512 normal map
256 full-white (eg, 100% on) temporary tex used in the translucency falloff and power slots
I'm also keeping a bit of a progression log which has a few earlier shots and a mini-diary detailing the progression of the shaders on my webpage - find it here.
Replies
Vailias: yup, all hlsl - using the max7 directx 9 viewport material, hence why the shots are in the max viewport. Beauty of this is that the exact same file and code is used in both the viewport while you're creating your assets, as is used in your final engine... man I love that
As for getting a copy once it's done, short answer is yes, I do intend to release it after it's complete; long answer is that may be a while since my to-do list still contains a few more things like fresnal specular and a more robust lighting model. I'll also be "compressing" the translucency texture inputs in a similar way that normal maps represent RGB=XYZ, but something along the lines of R=trans power,G=trans ramp out, B=trans occlusion (but that wouldn't take into account the colour component which I want, so haven't decided about that yet).
...otherwise, looking freaking sweet...
In regards to faking internals, it does an occlusion kludge - it takes a texture for ramping out the translucency (eg, how far it can wrap around a certain area), and also the amount of occlusion which is multiplied by localised attenuation... which means you can fake internal depth(thickness/ramp out) and structure(occlusion). I just need to see if I can cheaply give it an ease in -and- ease out colour like mental rays fast sss
palm: cheers
Weiser_Cain: hehe, by profession I'm an artist, so I can understand frustration of having new shiney toys dangled in front of your face - I'm by no means a programmer, and I've only been at this for 3 weeks now, so it may be a little time before I release anything
Main reason for showing this is to get some feedback and ideas/crits (much the same as everyone shows off character art) - If there seems to be enough people wanting to try it out though, I might do an early release to get some more targeted feedback.
Illusions: yup, that's already in -- the trans. power texture is the intensity, or brightness of the translucency - the ramp out texture is how much a given area can wrap around into the shadow, so that effectively gives control of dermal depth. There will also be occlusion mapping to simulate the amount of "pass-through" light as well, just need to get round to it
cheers guys, hope to have another update soon
Noticed a comment in your blog: "normal maps have introduced a small seam error which I haven't been able to find any referance as to how to go about fixing it. It's kind of like the normals are inverse on Y/G from the uv stitch... kinda like you need to reset xform on one side of the uv's... "
Sounds to me like the tangents need to be computed. To fix it you need to massage the vertex normals to bring the lighting into tangent space. And that means not only duplicating normals along the seam, but also rotating their S and T vectors. (and that's about as far as my knowledge goes!)
Check out Nvidia's MeshMender for details. Probably won't help for a viewport shader, you would need to use a modifier that updates the normals' tangents every frame. But if you pulled that off, I'm sure a lot of people would come knocking on your door!
I'm eager to see where you go with this. Also the hair shader sounds promising.
unless i'm confused--??
one thing so, I assume you use the normals to find out if you are behind or outside, but wouldnt it be wrong compared to real sss when you are actually on the outside of the object at the backside ?
I think in the second shot "light inside belly" one can see that the rest of squishy is lit like regular front lighting, which makes the eyewrinkles and such "cold lit" as the "outside" case in the first shot, however since the light is inside they should be warmly lit as well. which is something that cant be trivially done, cause basically the shader would need more info on the mesh topology, some sort of volume information.
cause you cant really do raytracing to find out if you are inside the object and such ?
light | object | <- viewer
the backside would be backface culled, so you are "inside" the object
light object | <- viewer
which is somewhat different case then if the light actually hits the backfacce first, but well thats just my assumption
anyway its all about tricks, so yeah it looks cool as it is
this here is a less trivial approach to find out if you are really inside, how far inside...
http://www9.informatik.uni-erlangen.de/Persons/Stamminger/Research/Research/tsm
hey man, this is really good stuff! cutting edge, hardcore shit man.. i haven't seen this is next gen engines..
unless i'm confused--??
[/ QUOTE ]
While not exactly like this, UE3 has "transmission maps" which are it's version of SSS map.
This is looking AWESOME man. I didn't know you coded as well as made awesome art. Message me more on ICQ and I'll try to actually be on more. 8-)
Looking good, can't wait to see what else you come up with.
Thanks for the info on meshmender - I think it's not possible to re-solve the normal within the vertex shader, even if it was, it would be -really- expensive. after a bit more research it appears to be a rather bad bug in max's implementation of its viewport shader -- ben cloward had a good analysis of the problem here: http://bcloward.blogspot.com/2005/05/normal-map-seams-in-3ds-max.html
funny how the max7 beta didn't have the bug, it was introduced later on
MoP: cheers, sounds like it's got a bit of interest so I should release -something- in a week or two.
John_Warner: I know of a few next gen games that do have generalised sss shaders, but I'm not aware of any with this level of tweak - pretty much the equivelant of max's falloff and that's it (problem being it doesn't wrap into the dark areas, only gradients the light).
CrazyButcher: this is definitely an issue with the way it's being done cheaply - an easy kludge to resolve this is simply ramping through a self shadow occlusion checked against per normal attenuation. problem is the whole self shadowing part, I'm not too keen on getting that to happen in the max viewport as a dedicated unified lighting system would be a lot better to cope with that
cheers for the info and link though, lots to think about
poopinmymouth: cheers poopy
"I didn't know you coded as well as made awesome art" ... heh, neither did I
Keg: Yah, I had a brief chat with the project offset guys, they're doing some great stuff (still haven't cleaned up the stain I left from the object motion blur stuff they've done). cheers
About the UV seam being a bug between v7beta and v7, it seems Ben's investigation was only into what happens when the adjoining UV edges are in the same orientation in UV space... yeah that's a dumb bug to make.
But I was talking about a more general limitation that comes whenever adjoining UVs have different orientations in UV space... then you'll get a seam, v7beta or not. For this you need a MeshMender-like fix. From what I've seen it's pretty easy for coders to get this into a game engine, it just takes some pre-computation on the model before everything's peachfuzz-smooth.
Thanks again man, good link.
Wow, thanks for the link to Ben Cloward's blog. I didn't know he had that. Very informative reading.
About the UV seam being a bug between v7beta and v7, it seems Ben's investigation was only into what happens when the adjoining UV edges are in the same orientation in UV space... yeah that's a dumb bug to make.
[/ QUOTE ]
Thought you might like to know that the bug seems to be fixed in Max 8. I don't know the technical details like you guys, but I opened Ben's test file in both 7 and 8 and there is a definite difference.
(the 7800 demo is impressive as hell mind you)
It looks very good. Certianly not seem something like this working with a reasonable speed on an FX5200 before.
(I still use my FX5200 for every day use)
Though I'm not sure I like the Vertex Lighting, using another map or the alpha channel of one of the maps to fake bones would also provide a good effect.
As far as the FX Cards go, it's best to keep things as a more complex single pass. As multi-passes cause horrible slow-downs. However for Radeons you'll want to do it the other way around. Spread it out over a few passes.
Also, Nvidia just posted a PDF that gives a ton of details about their GeForce7 demos, including how they achieved the translucent skin effects in the Luna demo. It's pretty complex, but you might find something in there that you can use. Here's the URL:
http://download.nvidia.com/developer/presentations/2005/SIGGRAPH/Truth_About_NVIDIA_Demos_Med.pdf
Good luck!
Squishy scares me..
Eric: again, thanks for the info -- I've had a hell of a time getting my head around the different transforms to re-normalise the tangents, but yah, no such luck fixing that original problem - max's bug hasn't helped any either but at least I know where half of that problem is coming from
FatAssasin: doh, that'd be right... I'd been invited onto the max8 beta, but at the time I hadn't had time to even look at it. ah well, good to know however.
Ben: Cheers! The fps display is just what I was looking for, although it's showing two very similar, but different numbers, so not exactly sure which one to take note of
Your shaders are actually the things that kick-started me into writing my own - thankyou for being an inspiration, keep the awesome work flowing!
thanks for the links too! that nvidia one has given me a few grand ideas which I know I'll never be able to pull off at the moment, but... I also think I'm onto a pretty cool approximation kludge based off an idea from that, so fingers crossed
Raven: yah, I like to work off the 5200 as a lower spec middle-ground, but it's just so nice to play with the next gen specs too - that's kinda where this came from, having experianced a loss of control in the artists realm with next-gen specs, I wanted to see if I could bridge that gap and be able to collaborate more effectively with programmers on shaders, if not be able to just write my own
Darksun hehe cheers - Squishy shouldn't scare you... after all, he's only made up of rubber-like flesh and cartledge, not even able to bite anything... and that whole no limbs aspect frustrates him so much that he's taken to living in solace in an old pair of floral underpants under the bed.
Hearing that my shaders helped you get started really made my day. That's the whole reason I put them up. I'm really glad they helped you. I'm hoping that a lot of artists will take the plunge into shader writing because of the amazing power they give us in controling what our characters look like.
I'm very interested in helping you with this. Like I said, this is what I was going to work on for my next project. Feel free to email me if there's something you get stuck on or when you're ready for feedback.
here's two new pics showing off the new attenuation bloom ramping, and the first specular component being re-implemented. The specular needs a secondary fresnal modifier to give it that glancing sheen that skin has, but after that's done it's pretty much ready to release
The attenuation blooming and specular in action:
The shader inputs and settings for the previous image:
As you can see, I still haven't gotten around to putting the texture inputs for the specular component. How the textures are/will be working is as follows:
Diffuse map:
RGB=diffuse colour texture
A=Opacity
Translucency map:
RGB=translucency colour and intensity
A=ramp out amount (eg, light scattering radius)
Normal map
RGB=XYZ
A=Maybe height if Mr. Ben Cloward wanted to help me out with offset mapping
Specular map:
RGB=Specular colour and intensity
A=Glossiness
On a side note, I've had to upload these new pics to a free server, since that first pic chewed up ~420mb in under a week... my poor little hosting plan wouldn't cope with too much more
Ben: I totally agree about the advantages of adding shader authoring to an artists skillset - What I've experianced so far is that although working on next gen means higher levels of freedom in spec, the artists range of output has shrunk - meaning that we get more polys and pixels to play with, but the ability to create and collaborate on a broader range of elements has shrunk. We're relying on specialists to do their thing, instead of being able to collaborate togethor and get the best of both worlds. To this end, I've been thinking about writing up a quick shader overview for artists, explaining the basics of the shader structure language, and referancing where to go to learn everything else. Do you think that'd be worthwhile?
nice head as well.
there is quite some papers about introduction to shaders and such, and most of them also bring up some basics about the rendering pipeline. However you will mostly find these papers and articles if you dig it up on graphics coding sites, which are probably not that known to artists.
Some sort of "tech for artists" wiki or so, would be great and I'd love to contribute to it as well. Eric has some cool articles and his graphics dictionary, if we could set up some sort of central place, preferrably wiki, and put the material in a unified way so that people dont have to search thru forums and links or coding pages, it would be a great help I guess.
We could even expand it a bit and give descriptions on the tools, like normalmap generation,... "artists" introduction to fx composer or rendermonkey or so. well a wiki would make it fairly easy to expand from the shader basics.
we just have to be careful making it artist centric and not too much like regular graphics coding pages, ie we need lots of illustrations and sample pics
anyway such a "tech for artists" wiki would be a great idea imo. while surely one can hunt down the material himself now, it would be more beginner friendly and easier if there was a place dedicated to it.
CrazyButcher: A graphics technology wiki sounds like a very good idea to me too!
- Write a skin shader
- Write a tutorial on the basics of shader writing
Funny stuff. Thanks for helping me with my list, Joel. :0D I would love to contribute to a project helping artists learn shader writing. In fact, I have a ton of free server space that I use for my web site. It has unlimited bandwidth since it's hosted by a friend of mine. If you guys want to write some tutorials I'll be happy to host them. I don't know anything about setting up a wiki though. My knowledge of web programming only covers basic HTML. If we want a wiki (I think that would be best) we'll have to figure out something else.
Joel, offset mapping is very simple. Here's some sample code to do it:
In your vertex shader, calculate the eye vector and pass it to the pixel shader in the same space as your other vectors (tangent space, or world space). You're probably already doing this. Then do the following at the beginning of your pixel shader:
// normalize the eye vector
half3 V = normalize(In.eyeVec);
//because Max is Z up, you have to invert the Y
//component of the eye vector. I don't understand
//why completely. I just figured it out by trial
//and error.
half2 newV = V.xy;
newV.y = -newV.y;
//sample the alpha channel of the normal map
half height = tex2D(bumpTex, In.texCoord.xy).a;
//offset the texture coords based on the view vector
//and the height. offsetBias is a user definable value
//that usually works best when set to about 0.04
half2 offset = newV * (height * 2.0 - 1.0) * offsetBias;
half2 newTexCoord = In.texCoord.xy + offset;
Once you've caculated newTexCoord, use it as your texture coordinates to look up all the rest of the textures that you have to sample - your diffuse map, your normal map, your specular map, etc, instead of the standard texture coordinates. That's all offset mapping is. Just messing with the texture coordinates based on the viewing angle and the height map to give the illusion of depth. Give it a try.
Good luck!
sorry for the sidetracking of the original thread
btw the Y -/+ thing very likely is right handed vs left handed coordinate systems
this one shows off fresnel specular and the per pixel specular and gloss controls:
and for referance sake, here's a huge repository of photos of Asian girls, spread out over 277 thread pages titled "A tribute to the beauty of Asian girls"... great ref for studying skin properties
http://forum.japantoday.com/m_116851/mpage_1/key_/tm.htm
the more I see it, the more I want to try it
thanks a lot for showing this !
[edit] Or you could just bake it into your diffuse, i think that would be a lot more realistic =D
One question, is the spec/gloss map applied to the model? Becuase I don't see the lips getting any shinier than the rest of the face, even though the map itself is much brighter in that area.
vahl: shouldn't be too long now to release - I think a first version release will be happening sometime this week.
dom: I was thinking about doing another shader after this one in a kludged ambient occlude type of vein... playing around with generating a reflection cube map, and sampling points in depth to create a lighting point cloud -- we'll see how that goes, but then I'd try and plug it modularly into this one.
EarthQuake:hehe, well see dom's answer there, but I could do a seperate light map on top of this too, only trouble is that in a single pass and as a single effect, it's pushing the complexity boundaries... it's about at the limit of 2.0 instructions as it is, although it does need a bit more optomising
ghost_rider: cheers - like I said above, shouldn't be too long to a public release.
FatAssasin: thanks - yeah, the spec/gloss is on it, but it's toned down fairly subtle on that mesh. It works off the one map, RGB is for the specular intensity (luminance) and colour tinting (hue), and the alpha is for the gloss intensity. There's a global multiplier which controls the range that the 0 to 1 of the map affects, so even though the difference on that map is quite a range, it's stepping over only a little range when it's applied through the multiplier... my bad on that since I was playing around with the fresnel too much
<font color="orange">Public release, version R2 is ready to download
go here to grab it.
It's not perfect, there are inherant limitations, it's quite tweaky, it only supports one light+ambient, and the code isn't as pretty as it could be. With that said however, you can get some awesome results and I hope you enjoy playing around with it!
I'll have a play around with it as soon as I make something to test it on
diffuse: RGB, alpha not supported yet.
spec: RGB is the colour and intensity/power of the specular component. Alpha is the glossiness.
Trans: RGB, again it's colour and intensity/power. Alpha is the amount the light can bleed along the normals (which in turn is amplified by the attenuation bloom) - think of the alpha like the internal depth, and the rgb like the density of the internals... so alpha is the scatter radius and rgb is light penetration amount.
normal: rgb, xyz... pretty standard.
In regards to the settings, it defaults to some more stable base numbers, but it is incredibly flexible for a number of different surface types. I suggest moving the light source far away and balancing out the settings, then move it closer and play with the attenuation bloom/focus controls seperately. Fresnel settings are dependant on the base specular settings since it's all one pass, so I'd suggest doing the base spec settings then go onto the fresnel stuff after that's all good - just check the glancing angle flare that the fresnel settings give by rotating around your model. Pretty easy, but it all can be very tweaky. Anyways, have fun guys
go here to download: HEREHEREHERE
So what's new? Well apart from a few tweaks to the lighting model and some optomizations here and there, you've now got the choice of choosing between using 1, 2, or 3 light setups from the technique drop down.
Thanks to all the people that sent along pics of their wip models and comments - I hope you enjoy the next version. Enjoyerise