Home Technical Talk

How to re-use facial expressions blendshapes on multiple face models?

polycounter lvl 10
Offline / Send Message
Pinned
XilenceX polycounter lvl 10
We're working on an RPG with 90+ characters, so we can't start from scratch with all the blendshape sculpting for each character. 

For now we're only concerned with male characters, which all share the same base mesh. 
Right now, all the different characters are set up as blendshapes of one rigged "neutral" base mesh.

The problem is, when I now add another blendshape to this base mesh with a facial expression blendshape, it doesn't work. 
Or rather, it works fine on the "neutral" base mesh, but when I have a different character blendshape set to 1 (100%) the facial expression does nothing.
So actually, I have three questions:
1. is there a way to deform a single base mesh via a blendshape into another character and the facial expression blendshapes of the base mesh work on top of that?
2. If the individual characters all have to be separate models. Is it then possible to create the facial expression blendshapes once for the neutral base mesh and transfer them to the other character models with a tool?
Something like this: https://docs.cryengine.com/display/SDKDOC2/Morph+Transfers+Between+Characters
But preferably a tool that's not marked as obsolete / not supported anymore? 

Finally, the third question only applies if both are possible: which solution would be best, 1. or 2.?
We're using Unreal Engine 4, so performance optimization is a concern here.
We'll be switching to Unreal Engine 5 as soon as it's released. The promise of "fully customizable mesh deformation" for characters in UE5 based on layers is tempting. However, I don't have enough information on this to rely on it solely for our character pipeline.

Replies

  • birb
    Offline / Send Message
    birb interpolator
    I don't have the faintest clue of how to do this with engine-specific tools but as long the base mesh is exactly the same—same number of vertices indexed in the same order—some 3D packages offer methods to transfer them between models. Eg.: Transfer Shape Key in Blender.

    I also know of an add-on to transfer blendshapes between similar meshes; I think this one is by vertex proximity, hence slower. Throw in a little Python and you'd be able to automate the entire process.
  • XilenceX
    Offline / Send Message
    XilenceX polycounter lvl 10
    birb said:
    I don't have the faintest clue of how to do this with engine-specific tools but as long the base mesh is exactly the same—same number of vertices indexed in the same order—some 3D packages offer methods to transfer them between models. Eg.: Transfer Shape Key in Blender.

    I also know of an add-on to transfer blendshapes between similar meshes; I think this one is by vertex proximity, hence slower. Throw in a little Python and you'd be able to automate the entire process.
    Don't worry, I'm not (specifically) looking for a engine specific solution.

    Thanks for the hints, with the help of your reply I was able to find this plugin for Maya:
    https://github.com/robertjoosten/maya-retarget-blendshape
    Purely based on the description, this seems exactly like what I need. 
    Plus I like that it's open source, so if there is an issue with the plugin mid-production I could potentially fix it myself, or hire someone to do that.

    Yes, automation is key for this task. 
    If someone knows a better plugin, or solution to this issue, I'm all ears! 
  • poopipe
    Online / Send Message
    poopipe grand marshal polycounter
    question 1 :  Yes. Use the same topology for all heads, treat each new face as a blendshape and simply apply that to the base model  - the expression / phoneme  blendshapes will ride on top of that.  The faces do need to be fairly similar in shape to prevent bad stuff happening but in sensible use cases it's very effective. 

    question 2 : when I've had to deal with this I've either used skinwrap (in Max) or constrained a rig to the deforming mesh and and transferred the animation to a rig that fits the other mesh.  

    It might be worth you looking at Wrap by russian3dscanner, I'm about to start evaluating it for use in our studio - haven't done anything yet so can't say how good it is but on paper it seems like the answer to a lot of problems
  • XilenceX
    Offline / Send Message
    XilenceX polycounter lvl 10
    With regard to question 1: All heads and expression blendshapes have the same topology, got that. 
    The base model has a new face type applied via a blendshape, done that. 
    Now, how is this third step of "- the expression / phoneme  blendshapes will ride on top of that" set up? 
    If I already have a new face type applied via a blendshape that's set to 1 (100%) the expression blendshape does nothing for me?

    I can highly recommend R3DS Wrap, it's an amazing tool! 
    (We use it to make the character variations with identical topology, based on our sculpts.)
    The way we used it in the past it's not really a "magic bullet" though, since there is still a lot of work in placing the point pairs it works with.

    With that said, we're currently experimenting with as usual having the identical lowpoly topology, but also identical topology for the high-res character sculpt.
    Now, in theory you could just load a new high res character into Wrap, use the previously set point pairs, run wrapping and done. Not sure yet, which types of characters this works for, without breaking, though. 
    If it works for all our characters, it's more or less a "magic bullet" solution to all the things. 
  • pior
    Online / Send Message
    pior grand marshal polycounter
    FWIW Blender does what you want by default. That is to say : if you have a head model with blendshapes and then sculpt it into another head/character, the blendshapes  deltas will carry over - and you will have only a little bit of cleanup to do, if any.
  • oglu
Sign In or Register to comment.