Subsurface Scattering Research and Experimentation
polycounter lvl 9
Hey guys I am just going to start this thread off with a small overview of the progress that I have made so far.
I am in the process of completing this project as part of a masters dissertation titled 'An investigation into creating physically accurate real-time skin shaders'
In terms of research I have been looking at technical skin rendering papers such as
Jorge Jimenez's - Screen-space perceptual rendering of human skin
and
Eugene d'Eon's - Effiecent Rendering of Human Skin
I have also been investigating both the texture space and screen space subsurface scattering solutions that are availible in both UDK and Crysis.(Crysis 1 & 2, UDK DX9 & DX11)
From my research I managed to identify strengths and weaknesses of each of the skin shading approaches. This has lead me to the stage where I am going to attempt to merge both the texture based solution (DT 3d - http://forums.epicgames.com/threads/732232-DT3D-UDK-Skin-Shader), with some of the newer intigrated screen space features from the newer DX11 UDK.
Current Problem: Miguel's shader works by the use of both a custom lighting and custom diffuse parameters, which is overwriting the extra screen-space attributes.
Once I have finished prepping the head then I plan on exporting it to UDK and see what elelments of Miguel's shader can be used without the use of the custom lighting and custom diffuse.
I am also planning on investigating the SSS withing Crysis and testing them in a physchophysical test with a series of check boxes so that I can see which technique is the most accurate.
If you got this far , Thanks for reading
I am in the process of completing this project as part of a masters dissertation titled 'An investigation into creating physically accurate real-time skin shaders'
In terms of research I have been looking at technical skin rendering papers such as
Jorge Jimenez's - Screen-space perceptual rendering of human skin
and
Eugene d'Eon's - Effiecent Rendering of Human Skin
I have also been investigating both the texture space and screen space subsurface scattering solutions that are availible in both UDK and Crysis.(Crysis 1 & 2, UDK DX9 & DX11)
From my research I managed to identify strengths and weaknesses of each of the skin shading approaches. This has lead me to the stage where I am going to attempt to merge both the texture based solution (DT 3d - http://forums.epicgames.com/threads/732232-DT3D-UDK-Skin-Shader), with some of the newer intigrated screen space features from the newer DX11 UDK.
Current Problem: Miguel's shader works by the use of both a custom lighting and custom diffuse parameters, which is overwriting the extra screen-space attributes.
Once I have finished prepping the head then I plan on exporting it to UDK and see what elelments of Miguel's shader can be used without the use of the custom lighting and custom diffuse.
I am also planning on investigating the SSS withing Crysis and testing them in a physchophysical test with a series of check boxes so that I can see which technique is the most accurate.
If you got this far , Thanks for reading
Replies
Helps to explore and calibrate shaders that way.
I think this project is HEADing in the right direction now as I have been able to get some realtime SSS working in UDK.
After a lot of tweaking I was able to alter Miguels shader, making it possible to remove the custom lighting and diffuse so that I could use my own lighting solution along with the new SSS features.
I still have much tweaking and testing to do to get the result right. I also need to test the FPS of each of the solutions and create a questionnaire to perceptually validate the shader.
Picture 1 is a shader using only the default UDK material slots along with the new SSS.
Picture 2 is Miguels full layer shader. It looks very noisy from the side , where as it is much better from the front.
Image 3 is the new hybrid shader, the specularity is quite poor so I have began tweaking it in pic 4.
It's not so much about the model looking like a photograph but physical accuracy of the skin., although, saying that I don't think throwing in a photo to compare the model to would hurt
Before I can test each of the shaders I need to create a questionnaire so that I can evaluate individuals perception of human skin then apply that to my own shader. I can then test the proposed shader against the others to see if the new shader is a better perception of humans skin.
Creating a questionnaire about skin is going to be fun to say the least \o/
I tried importing the head as a static mesh (ASE) but I couldn't get it to handle overlapping UV's so I ended up exporting the eyes as separate objects. Strangely enough it still thinks that either UV's are out of bounds or are overlapping for the head despite me triple checking in max. I deleted the eyes, applied a single material and a single material ID etc and it is still playing up, crazy UDK.
The static mesh heads look fine before the lighting is baked. Once it is baked they get strange shadows on them. It suggested applying physics to the model in the error log, so I applied Phys_Interpolate which has not changed anything
I have been doing some testing on how each shader reacts to different lighting conditions, as it was a issue that was raised in the literature review. (Default udk approach (right), DT3D's shader (middle), DX11 UDK (left)
For the screenshots and renders etc , I will place each of the heads in the same position so that the lighting is the same etc. At the moment they are all layed out so that I can set up the correct testing environment for a future psychophysical test. The test will be based off the results of the questionnaire.
I am in the process of creating a questionnaire to evaluate the perceptual characteristics of human skin at the moment.
The questions I am currently asking are broken into 4 themes at the moment;
The idea I have at the moment is that I want measure their perception of current characters skin accuracy against the newer SSS shaders. I also want to asses the order of importance in which they validate the perception of accurate skin so that I can apply it a new shader.
I have broken the questionnaire down into 3 sections, Existing Real-time solutions, Shader Comparison, General Perception and Extra Characteristics
Most of the questions are closed with the answer of 1 to 5. 1 being very bad and 5 being very good.
Existing real-time solutions
overall glow (sub surface scattering)
1-5
high frequency details (pores)
1-5
Skin Surface (Oil/Grease)
1-5
Colour Variation (Fleshy Colours)
1 - 5
Shader Comparison
overall glow (sub surface scattering)
1-5
high frequency details (pores)
1-5
Skin Surface (Oil/Grease)
1-5
Color Variation (Fleshy Colours)
1-5
General Perception
What 4 colours do you associate with human skin?
(make a choice out of 8 colours)
List the order in which you would navigate around a head to measure its accuracy?
eye brow 4
ear 3
hair 2
nose
lip 1
cheek 5
Extra Characteristics
Age
SSS, Spec, Normal Blur
Race
SSS , Spec ,Normal Blur
I am in the process of thinking up a way in which i can test perception that the shader has on the age and race of skin. I am planning on having 3 images with different amounts of blur, spec and SSS values, and seeing how the change in settings effects the appearance of aging. This will make it possible to measure the impact that each characteristic has on the shader. I just need to think of how I can measure the answer
Testing the FPS and lighting environments will happen in the psycho-physical tests. The where questions will be forced 2 choice eg Which looks better A or B?
As you can see in the image above of the interior lighting, there isn't much to go by to measure the perception in the dark haha
If you would like to see other questions asked please just say
http://www.google.co.uk/url?sa=t&rct=j&q=penner%20pre-integrated%20skin%20shading&source=web&cd=2&ved=0CCsQFjAB&url=http%3A%2F%2Fadvances.realtimerendering.com%2Fs2011%2FPenner%2520-%2520Pre-Integrated%2520Skin%2520Rendering%2520(Siggraph%25202011%2520Advances%2520in%2520Real-Time%2520Rendering%2520Course).pptx&ei=JqtDT_T6OtHZ8gPL_fC1CA&usg=AFQjCNG1FoQCDd7fNkQ3v-rZe2pqnbcS7A
I've had pretty decent results with this setup.
Update
I have managed to do all of my testing etc and I am coming to the end of the project now. I am stuck on the eyes though, I can't seem to get them looking right! Without fresnel they look too flat, with fresnel they only seem to look good in the middle and it seems that the fresnel highlight adds to the spec highlight which creates bloom and makes it seem metal-like at times. It is driving me insane lol
Here is the eye shader in UDK any suggestions will be greatly appreciated.
Also, the specular of your skin seems to be floating and fighting for Normal information, how did you set it up? Also, is it only 1 Specular layer or two?
Lastly, I don't see the issue with the eye you mentioned, and you lack a proper roughness mask, which could be also used alongside your Fresnel to control the amount of 'block' and edge rim you're getting.
This reminds me, if you're trying a Rim, avoid Adding, the correct math to use with it is it Multiply or Lerp on your Specular and working from there to your end result.
Here is the skin shader setup:
I have used DT3D's shader as a base and in order to get it to work with the DX11 features I had to mess around with it as it originally had a custom lighting overwrite, which might explain the angle issue.
I agree with the spec roughness I cant add a roughness map as it is already controlled by a param. As for the spec there are 2 layers. 1 s based on the spec map and the other is linked to the normals which would explain the floating spec map that you mentioned at certain angles. I will have a play around with it and see if I can get it to look more realistic.
Oh, also I have not used any masks for any of the shaders yet as I needed to test them at their basic levels to measure their performance.
I would definitively use one if it wasn't for the circumstances though, I am afraid that if I added one now it would throw all the test figures out of the window.
You mentioned a roughness mask for the fresnel
Thanks for mentioning the add and multiply, I changed it to multiply and it looks much better now, it was driving me insane
I guess I can see what the issue is now.
1-Try replacing the current DT3D setup with a Blended Normals option. Problem with DT3D is you lack any kind of 'Cyan' suppression, so you will always get that nasty 'blue' fade near the shadows, which, while cool, can look pretty bad.
It's cool for veins and such, but not much else.
Fun-Fact; DT3D is based upon a paper, which is nothing more then a glorified Blended Normals setup with extra options which make it heavier and gives very little in return. Also, the two Layer setup is pretty funky since it's create 'cuts' in the bleed, making it look pretty artificial, and 3 Layers don't work with this setup, which again, is a bummer.
If you want to be cheap, you could simply Lerp between a Blue color and the Normal Map to fake the 'blur' effect of the textures, which is MUCH better IMHO, since UDK doesn't allow for Normal Map MIP'ing, forcing you use two textures instead of one, which isn't very optimize friendly.
Not necessary, but it should be cool to see.
Also, if you use Max, you have the option of baking out a SS map, use that to control your setup.
---
2-That specular is really complicated, and is plugged in the wrong place. CustomLighting allows your create your own 'Specular' effect properly, UDK by default uses Phong.
What you currently have is a "Blinn with Fresnel" plugged into a Phong Specular, which is creating some funky results.
Also, the current Blinn with Fresnel seems to be wrong, I cannot figure out what the original intent of the effect was...it's too mixed up.
I have no idea how you could create a Physically correct Specular in UDK default material, sorry. Someone else might better serve you here.
---
3- Roughness map is another fancy work for Gloss, yes, which is VERY important in defining which parts of your skin and etc are oily or dry.
I suggest doing this, create a decent Gloss and Specular texture.
Multiply them by fresnels, constants, colors, etc, whatever stakes your fancy. Experiment and go crazy.
(Don't put any color values in your Gloss since that would be silly ;P )
Duplicate the setup for each and add them together (Gloss1+Gloss2) (Spec1+Spec2) the results and plug each finale into their respective slots. That's one way to simulate a layered material.
Again, problem is, a custom material would have much better control, such as each specular layer having it's own Normal Map control, Skin has 18 specular values on average, each one being a more 'blurred' one of the previous version and tones down version of the previous. So finding a way to fake that through two material is pretty hard but can pull off some great effects (gives great results for performance ratio comparisons).
Also, yes, you can use the Roughness Map for your Fresnel too, although I would suggest creating a special one for your Fresnel. Trust, Fresnel is a bitch to get to under control properly, having a proper map multiplied by a constant should aid the effort.
---
Go to your UserCP > Edit Options > and Tick "Enable Private Message" and Untick "Only from Member and Mod's"
Those two should allow me to send you the message.
for one thing use a lighting model that isn't phong! use a lighting model based around rough surfaces and you'll get a much better look from the start.
Next you want to get the lighting information and blur it across the surface probably using the depth as reference. You could fine tune a system for this to keep the jist of fine details or you could have multiple samples, much like the multi resolution SSAO technique.
Then you want to approximate subsurface scattering, I've not tickled this much yet so http://http.developer.nvidia.com/GPUGems/gpugems_ch16.html this should help.
I should point out specific shaders aren't my strong point, I'm more about global illumination and ambient occlusion techniques.
good luck :P
Instead, just make a standard Lambert, use a Lerp to 'push' forward the Lambert shadow values (but make them red instead with a mask) and use an inverted NdotL setup for fake the Backsurface transmittance.
Or instead, try a using a Power node to Focus your Half Lambert and darken it more to keep akin to the shadows.
I have unticked the private message thing, so hopefully it should all work now
@bombshell - Thanks for that link , I shall give it a read though
The current problem with using custom lighting to override the Phong lighting in UDK is that it can only handle light from one direction, I think Epic call them Zero-area Lights. And if other light types are used on it then it just reverts back to the phong lighting.
It is a pain in the bum lol
my advice is use XNA, a bunch of the model handling is done for you, everything is fairly basic in a user friendly language and you have COMPLETE control over HLSL shaders, render targets, light types in logic, shader and data stucture. etc. etc. Its perfect for experimenting with graphics programming, shader programming, lighting effects, etc. etc.
not trying to offend, but working in a material editor for creation of shaders is very inefficient, cheap and your not getting any where near enough control to do your proper research.
So you kinda have to 'fix' them yourself, which is a paradox itself, since from one side, it simplifies the process for many 'newbie' artists from making mistakes, but at the same time, stuff like skin (which need proper shadows and light inorder to show proper volume) gets taken away.
I also don't think you can also 'measure' volumetric length/distance in shader (at least, pixel form) to determine a dirty/cheap surface volume, this also causes issue in you trying to create a refraction shader for real-world distortion.
So yeah, UDK material editor needs alot of working around, and you can even easily get some awesome results, but the lack of some functions (such as MulWorldInverse) and finer control can get really stressful.
EDIT: Actually, UDK material is efficient. They made it more optimized not too long ago, and I notice that using a Custom node to create a Bodmas type of package (for those lazy like me) has the same instruction count as doing it manually, or 1 more at most.
So yeah, pretty efficient, although real efficiency comes from the artist/user and their abilities in creative thinking.
I also hope that I am not offending
I think you raise some good points, and XNA sounds like a good program to use if you were going to create a completely new type of shader model like Jorge Jimenez (http://giga.cps.unizar.es/~diegog/ficheros/pdf_papers/TAP_Jimenez_LR.pdf). By the sounds of things I think XNA would be more useful for a programmer, rather than an artist like myself.
I used UDK so that it was possible to test a combination of default and custom skin shading solutions(DT3D - http://forums.epicgames.com/threads/732232-DT3D-UDK-Skin-Shader); using the same lighting and shadowing environments, as well as the same limitations as other assets/games,for comparison purposes.
The idea was to identify current real-time skin rendering issues, such as that zero-lighting problem, and test the effect they they had on both the perceptual value and performance of the shader. I was then able to create my own shader (based off existing shaders/material setups) and test both the visual quality and performance against the other shaders.
I am in the process of creating a video that will show each of the shaders used;along with a tables showing the performance stuff
http://www.polycount.com/forum/showthread.php?t=96433
I must ask, what all of the effects such as the Chromatic Aberration have on the performance of your shader?
Have you managed to test it against in with different types of lighting etc?
Nope, I didn't try that.
As for lighting, yep, I did, and just like in movies, you need just the right setting with Bloom and stuff to make to look good, real finicky, I guess that's why over 100 years of movies always light the human form in the same way (fresnel/rim lighting is the most 'pop' solution out there).
Also, skin-fuzz. Since it's next to impossible for real-time apps/games to create micro-fiber based 'fur' effect, the next best solution is using Schlicks fresnel to fake the 'fuzz' scatter effect at glancing angles.
The entire shader is still pretty much WIP, back surface is hacky, so I need to replace with a Blended Normals to give that 'soft' look, and my Melanin is too 'flat', so I need to desaturate my Color texture and use that as a 'mask' with another 'real mask' for the ramp amount.
The entire thing is around 100+ instructions give or take with Specular included, so yeah, I would only suggest it for your main character when he's in the camera flirting with the gamer.
The real challenge in trying to make a nice-proper looking SS mask without painting everything manually, since UDK doesn't allow (last I checked) light to position transformation for simple depth measurement.
As for the release of the shader, yep, I want to release it public, but I'm not too sure on how many people will actually use it, or if it will be helpful
haha, that sounds like a similar situation to the skin shaders they made for Uncharted 2. One was used for cut scenes and the other for in-game. It would make life so much easier if consoles had more than 512mb of memory!
As the deadline is fast approaching I have just been making some final touches before render time.
I have had a play around with some of the values to give the spec more control over the surface. I have also played around with the SSS InScatteringColour values etc
It is pretty hard to see,(or at least it was on my screen)so I adjusted the levels in the pic on the right.
pale peach specular, lower the effect quite a bit so it almost looks like part of the diffuse, but is still noticeable as specular. (I'll say that the most noticeable thing about your material atm is the specular its far to high for a general skin material.)
On the light curve have white gradually turn to a very pale pink then on the edge of the shadow change into a more prominent pink then a generic skin shade into normal shadow. Think of it as picking colours for a cel shaded painting.
with sub surface scattering you can dim the effect at the edge of the shadow and replace it with SSS shades of pale peach into bright pink into normality.
As for the gradient...I think at this point, a 1D texture lookup for Gradient/Ramp map a la Toon shader style, but with more 'humanized' colors would be the quickest solution. Look up TF2 shader for UDK and you should fine an image with the setup I'm talking about.