Home 3D Art Showcase & Critiques

Stylised Witch for UDK

polycounter lvl 8
Offline / Send Message
Dr.HeatSync polycounter lvl 8
Hi, I’m here to talk about my Final Year Project I’ve been doing at Staffordshire University. The basic gist of things is that I am designing and creating a character and accompanying art style for use in UDK. There will be some HLSL on the side so that in theory the character could be made use of in other engines as well.

This post will concentrate on the character concept: the post following this one will concentrate on research and experimentation of HLSL/UDK.

The character being designed is a witch called Abby; a simple happy go lucky lady who has a job as a runner and is ideal for a light hearted platformer. As such the accompanying visual style should be warm and colourful.

The initial course of action was to design a sort of ‘default’; a sort of general version of the character which can be tinkered and tweaked as I saw fit. It meant that there was something to compare the potential styles to. Here she is below.


A wide variety of initial sort of potential shading styles were produced, covering what kind of colours could be used, methods of shading and other potential operations. The colours for the character in this image were not final.


Producing another sheet of concepts played with the proportions of the character. It was decided that the third concept on the bottom row would be advanced further.


Another sheet was produced to further finalise the shader concept. The drawing itself is not particularly good, but the shader details are what were important at this time.


Feeling that the character needed more energy in the hair and that the concept needed some neatening up I redrew the character with a variety of hairstyles.


My peers had suggested going with a braid, so I produced a sheet with some included. Initial braids were too large and were adjusted to become thinner and less cumbersome looking (well, less cumbersome.)


With the character shape finalised the shader could be visualised a little bit more concepted a little bit more neatly after deciding on a colour scheme.


As such the character was rendered out below with the visual style below.


However, the pose is rather problematic and the perspective implementation left little to be desired due to trying to work with a rough, so a new image was drawn and rendered. Below is the final character and shader concept.


More will come on how to achieve this shader, tests, and whatnot. I hope to update this at least once a week.


  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    I really like seeing all the preliminary work you are putting into this. Nothing to crit really, good luck!
  • PyrZern
    Offline / Send Message
    PyrZern polycounter lvl 9
    Great concept. Loving her already !!
  • Zeluron
    Ooh this looks familiar, good work! :poly124:
  • Dr.HeatSync
    Offline / Send Message
    Dr.HeatSync polycounter lvl 8
    So post two on this development; mostly on how to achieve different parts of the shader and the current progression of that and the character herself.

    There are currently 3 methods of getting outlines in UDK as far as I know: Fresnel effect, post processing and duplicated inverse mesh.

    As seen in a lot of animated films and series’, often times the background is painted in great detail, as the most that has to happen is the ‘camera’ panning across. Characters and moveable objects are rendered in a much simpler manner and have their forms defined by outlines.


    This cannot be currently achieved with post processing via the stock UDK, as postprocess materials have no way of distinguishing objects in the scene by a type or some kind of tag; as far as it’s concerned, it’s an image with a bunch of values per pixel.

    Fresnel effects as an ‘outline’ is not particularly accurate; rather the lines cut inwards because they’re based on the existing topology. Moreover, they depend on the mesh having a lot polygons due to the way that Fresnel is calculated (see below):


    So the way to achieve the look is via a duplicated mesh. This mesh is simply the original mesh with a push modifier and retextured to produce coloured outlines. To lessen the load of what is most likely a doubled polycount, the shader for this is made as simple as possible; emissive with a parameter for darkness (so if it’s a night level it isn’t fighting the lighting of the actual model) modifiable via script/kismet, a diffuse map that is much smaller and contains just blocks of colour. There may be experimentation with lighting outlines just for kicks.

    In the case of opacity planes, outlines will be drawn on the edges of the visible forms and no extra polygons will be used for them due to them being so flat.


    This decision did have an impact on the design of the character; due to the possible angles that the character and the outline could be viewed at, it’s not so possible to colour a ‘section’ multiple contrasting colours, as otherwise it runs the risk of showing the wrong outline colour. In short, the characters need well defined ‘sections’, each of them can’t have contrasting colours. It will be useful to paint the outlines in Zbrush/3D coat so that the viewing angles can be considered quickly.

    Diffusive Shading (with Ramp masking)

    The base shading model of this style is dependant on Valve’s Half Lambert term and their usage of ramps in games such as Team Fortress 2.


    The difference between mine and TF2’s is that the ramp sections are used as masks instead of just being multiplied directly. The ramp did not have to be coloured to achieve this (though it would probably cost less instructions if it was), but did require that the ‘hard section’ was absolutely 0 to be more easily distinguished from the soft section.


    When we have the sections defined, these are multiplied by colours (and later will be multiplied by a texture for more control). These are added to the diffuse after the ramp darkens it through a multiply node.

    The result below shows it in action with the different colours applied via material instance parameters.


    Exaggerated Rimlights and Phong specular (with Fresnel)

    Heavily based on TF2’s methods, the specular functions integrate with Fresnel’s input for normal maps. Currently I’m debating on Phong versus Blinn methods and will produce more tests on this shader in regards to the subject. The aim is stylisation, but also cost effectiveness.


    Below is the whole shader network as of now to achieve this. The normal map is one of the tiling brick maps present in the stock UDK (build July 2012). The real strength of this will be reliant on smooth, clean normal maps:



    (there was a version prior, but it was neither efficient nor did it distinguish the sectionings very well; the soft and hard sections bled into each other. This has been fixed.)

    Demo on a model I made previously for a different module:



    Modelling Progression:

    Some simple orthographics have been produced for the character to model the base mesh and sculpt to. These are below.



    As of now currently the base body has a simple base mesh to sculpt to. The rest will need to be modelled as base meshes or created via Dynamesh or similar.

  • Brian "Panda" Choi
    Offline / Send Message
    Brian "Panda" Choi high dynamic range
    the body mesh feels gesturally unconfident and a bit stiff. Bring the abdominals forward, and let the thigh and shins really curve in and out like an S curve.

    Gonna take me a while to read through your shader research, heh.
  • Dr.HeatSync
    Offline / Send Message
    Dr.HeatSync polycounter lvl 8
    My apologies for such a massive delay between updates; I’ve been working on another 3D project which will be posted on this forum soon.

    Almost all of the assets have been modelled and imported into Zbrush for sculpting. Current progression is below, with the face and the fringe currently being worked on:


    The fringe was aggressively Trim Dynamic’d to achieve the look of paintbrush strokes. Initally it looked something more like this:


    It was fine and used Dam Standard to draw in lines, but in comparison to an older image, it lacked the paintbrush-esque quality.


    So I reworked it, exaggerating and overlapping the trim dynamic brush to become this:


    However, I wanted to check if the sculpting style on the fringe was going to work with the shader that I created; If it worked it meant that I’d be able to apply the style along other parts of the clothing (skin is an exception; I want that to be quite soft in comparison to everything else). So I retopped half of it, and in the process worked out a workflow for producing my mesh based outlines.

    The outlines are producd by duplicating the mesh as a reference and applying Push and Normal (flip) modifiers. I imagine you may agree that’s quite simplistic; it’s an even line thickness all around the model with no tapering, so using a trick I’ve come up with I can paint the outline thickness with the pull/push brushes (I just mean I thought of it, someone out there probably documented it long before me, but I’ll talk about the way I did it regardless. However if you know of a better or similar technique I’d love to hear it!).


    (for those interested, here were a few notes I did whilst experimenting)

    This allows me to be able to see the outlines but still use the normals with the push/pull brush. For areas like the tips though manually moving the end vertex may be unavoidable, but this technique enables much easier conformed shaping of outlines.

    Having the outline as a reference will also unwrap the outline when the regular mesh is unwrapped, which is very useful for this coloured outline element in the visual style.

    I got the retopped mesh’s normal map via Xnormal and the insanely useful Handplane and got it into engine: The results seem to be quite good.


    What does this say: the workflow seems to work quite well (although low poly meshes will need to be imported into engine before making the outlines as there have been problems with missing faces). Other ideas include changing the thickness of the outlines via morph targets/blendshapes based on the camera's proximity (I think we can agree that these outlines are a bit thick at the moment) and multiplying the outline colour by a ‘global darkness value’ via Kismet to prevent the outline being brighter than the darkest relevant shade.

    This means that I can carry on sculpting without much worry with how the results will transfer.
  • Visceral
    This thread is the best thing ever!
  • nksilver5
    Offline / Send Message
    nksilver5 polycounter lvl 5
    I love all the breakdown of your workflow, really interesting to see how you achieve and get the look you want .

    Subbed ! :D
  • Lord McMutton
    Offline / Send Message
    Lord McMutton polycounter lvl 14
    That's pretty cool; the end result of the outlines looks like Okami.
  • Dr.HeatSync
    Offline / Send Message
    Dr.HeatSync polycounter lvl 8
    I completed the high poly sculpt and now the model is ready to be retopologised as a game asset. I’ll admit, it took longer than I expected. Some renders below:


    Some difficulties arose from trying to make the braid modular; As far as I know, Zbrush doesn’t have any equivalent of 3ds Max’s instances/references and becomes a cycle of trial and error editing and scaling, but so long as the piece is adjusted roughly enough to fit together the low poly should work fine. This modularity make more efficient use of UV space.


    The extra strands are created through max instead with an interesting workflow I’ve come up with. Oddly enough the game meshes are created first with the intent of making the unwrapping and texturing process as easy as possible: the base planes are created and divided using the connect tool to determine the smoothness of the end result. These are unwrapped and textured to preview the warping effect the mesh will have on the UV’s.


    With that done, the planes are cloned as references and placed around the model. However, rather than adding another Edit Poly modifier to the stack, I add FFD 4x4x4 or Cylinder modifiers instead as they allow for very smooth interpolation of the planes.


    High poly sculpt data can be projected onto the original planes, and because the planes are originally straight and rectangular it becomes very easy to make it compatible with anisotropic lighting direction. The only problem is that it’s not possible to get the unique curvature data since all the planes are different, but such is the price of modularity.

    Finally the shader has been updated once again to account for anisotropic lighting, using a little knowledge from ‘Real-Time Rendering of Human Hair using
    Programmable Graphics Hardware‘
    I was able to create it in UDK; mainly though this piece of code (converted to nodes in the material editor):

    sqrt(1- pow( < L.T >, 2)) * sqrt(1- pow( < V.T >, 2)) - (< L.T > * <V.T >)

    For the purposes of experimentation the Blinn version was looked at and created:

    sqrt(1- pow( < H.T >, 2))

    I took the fringe mesh I was experimenting on before and applied both methods to it. Results of such experimentation are below:


    For now, I’ve input the reflection based anisotropic method over the blinn one with the rest of the shader. The result is below, and a comparison between it and standard phong, and the updated shader node network as well.




    Next the low poly game asset needs to be created, along with the various textures and some more shader operations.
  • Bonkahe
    Offline / Send Message
    Bonkahe polycounter lvl 7
    Ok, time for a collective, DAMMMMNNNNNNN!

    Probably one of the best, most in depth looks at cartoon artwork in udk I've seen in the forums in a while.
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    hey dude, great work so far. I think the proportions of the face on that highpoly could still use some work though. The nostrils flare out too much and the corners of the mouth and jaw look unnatural to me. I think the eyes are a little too far apart, and that they are wrapping around the skull in a weird way.
    Again awesome work, but I would take the time to play with that face sculpt a bit more till I would call it done. Keep up the great work.
  • Dr.HeatSync
    Offline / Send Message
    Dr.HeatSync polycounter lvl 8
    hey dude, great work so far. I think the proportions of the face on that highpoly could still use some work though. The nostrils flare out too much and the corners of the mouth and jaw look unnatural to me. I think the eyes are a little too far apart, and that they are wrapping around the skull in a weird way.
    Again awesome work, but I would take the time to play with that face sculpt a bit more till I would call it done. Keep up the great work.

    Cheers for your feedback! I'll see if I can get those tweaks in before I start the low poly. Hopefully it shouldn't take long to get them sorted.
  • PHIllstead
    Offline / Send Message
    PHIllstead polycounter lvl 8
    I love all the breakdowns in this thread! Awesome stuff!+
  • mospheric
    Offline / Send Message
    mospheric polycounter lvl 11
    These breakdowns are awesome. Thanks for spending the time on these Dr.HeatSync!
  • Dr.HeatSync
    Offline / Send Message
    Dr.HeatSync polycounter lvl 8
    My apologies for the massive, massive delay between posts, but it should hopefully be worth it as I’ve worked on this project solid as crunch time approached for the end of uni.

    So a completed the low poly mesh was completed within 3DS Max. The mesh has 4 material IDs that correspond to head, body, hair and outline. The outline isn’t actually created until the original meshes are unwrapped. This is so that the outline’s UV layout has a nice starting point which can easily be tweaked.




    So the visual style with a character like this demands the use of three diffuse maps; not easy to go off and do 3 polypaints right? So the solution is down to colour coding; areas are coloured vastly different tones and hues, serving as masks to apply the normal diffuse, soft shade and hard shade colours in either Photoshop or 3D Coat. 3D coat is particularly useful for some last minute fixes or for more complicated diffuse maps.


    After projecting the high poly polypaint colours to the low poly a simple case of converting the colours to masks, made very easy when they’re projected as PNG’s (due to transparency data). With that any map like the set of 3 diffuses, specular, and so on can be created quite easily.


    The outline was created via the workflow I described earlier. When unwrapping, all of the parts are simply fitted within the UV 0-1 space; since they’re all flat colours anyway UV space distribution is not a priority.


    The shader below accomplishes the intended points that were highlighted in the original concept: we’ve got anisotropic lighting blended with phong lighting, the soft-solid shading via a ramp, 2 rimlights (one being the opposite direction to light vector and referred to as the ‘light driven’ rimlight, the other coming from the top), environment mapping, opacity mapping, silhouetting at a distance and some trickery to create the hard specular shapes found in cartoons and anime. Not only this, but it’s quite flexible to allow for new, different styles to be created by changing parameters.


    The rimlights are masked by Fresnel; the light driven rimlight runs with the clamped dot product of light vector and reflection vector, whilst the top rimlight simply dot’s the world up vector (0,0,1) with Fresnel. Both have their own adjustable power and strength parameters and can either be colourised by either a vector parameter or the environment map if it’s being used, as you can see above. In this case the character needs an additional texture to control the strength of the rimlight at various locations; it was necessary to prevent things such as below, where the hat receives too much rimlighting:


    The harder, larger specular highlights and rimlights are achieved by 2 power operations sandwiching a multiplicative boost; in other words the first power operation controls the initial size of the highlight, the multiply operation brings up the small values, forming harder edges, and the second power operation is the final tighten. This way, the highlights can be much larger and sharper than with normal specular power operations. All of these are also masked by diffusive lighting to prevent highlights appearing in darkened places.


    Environment maps are masked by ambient occlusion maps to serve as ambient lighting, typically being at quite low values on the mask so that reflective materials such as metals get the brightest whites. I feel it’s pretty much viewed as a percentage when making mask maps like this, so metal receives full reflectivity (1) and something duller would probably get something low like 0.2, and things like the strength value serve as an overall corrective. There is also a contrast parameter (just a power operation on the cubemap) that enhances the map.

    Silhouetting is a depth based comparison between itself and a value. It also fades gradually based on a parameter.

    Outlines have displacement and the option of tessellation on DX11 cards. In this instance I don’t really use the tessellation aspect as I think it tends to get slow very quickly, but it could be useful for close ups. The effect is exaggerated when the camera has a low field of view.


    There do appear to be render errors when the camera moves too close to the outline mesh, and is exaggerated with functions like TiledShot, like below.


    Finally map packing is in place to save on the number of texture samplers e.g. the shader expects the opacity map to be in the alpha channel of the diffuse map. In some cases it can be set to repurpose maps for semi relative aspects, such as the specular colour map can be desaturated for use as a specular power map if the filesize of the texture can or needs to be reduced further. The environment mask and the rimlighting mask are in the red and green channels of the so called 'mask map', with the blue and alpha channels being free slots for potential other things that may need to be masked.

    Renders are on the next post (don't want to run out of images on this post); these are taken straight from UDK using the shot command and forcing anti-aliasing via the Nvidia control panel.

    So, pros and cons of all this:

    Created a shader that can achieve the look of a variety of styles through changing parameters.

    Created a character that closely matches the look in the final concept and is quite expressive as well.

    Got something similar to the MVC3 style just by changing one parameter, or more professionally put: its easy to add things to in order to achieve almost any visual style.

    Although the model is fairly good, sitting at around 2x 15,000 tris with half of that being rendered quite cheaply and considering the complexity of the character, there are parts that could benefit from an increase in polygon density, such as the sleeves and other parts that trail along. It is also possibly related to parts of the outline showing up through the normal geometry.

    The shader currently relies on the 3 textures/vector parameters for diffusive shading; it needs an option to shade via multiplying the ramp against the diffusive textures as well.

    The cloth needs to be more clothlike, something that has to be rectified at the sculpt level.

    The character uses 3x 1024 for the main geometry and a 512 for the outline, really though for efficiency it may be better to use 1 2048 (which results in an extra 1024’s worth of uv space gain) and a 256 for the outline, since the shader is able to blend/mask anisotropic lighting, the only thing that may prevent such an operation is draw order which is down to both vertex creation order and the finicky UDK strip tool thing.

    So what next?

    I’ve currently started working on a sort of spammable enemy that uses this shader and fits within this character’s universe, in order to take advantage of the low end side of the shader: if Abby is sort of using everything the shader has to offer, then the enemy should be more like the opposite. Also, other things to prepare for the thesis I’m going to be writing next year, and hopefully with some luck Abby will be improved some more.
  • Dr.HeatSync
    Offline / Send Message
    Dr.HeatSync polycounter lvl 8
    Renders ahoy, both great and problematic. Middle is normal, left is a sort of 'night rendering' where the shadows are colourised more purple, and right is a sort of MVC3 style of rendering.

  • Quack!
    Offline / Send Message
    Quack! polycounter lvl 13
    Amazing work man. Love the technical detail too. I think your model works best with the white rim light. The character becomes much more voluminous and has a softer touch that compliments the model.
  • corrosion
    love your work man :thumbup:
Sign In or Register to comment.