Home 3D Art Showcase & Critiques

Firbolg D&D Character - Unreal Mocap Demo

polycounter lvl 7
Offline / Send Message
AnaCarolinaPereira polycounter lvl 7
Hi there!

In an attempt to join my two favorite things - character art and technical art - I've decided to make my D&D character that never was, and to try to push the boundaries of what I can do with facial mocap in UE4, using the iPhone as my camera. 

I am going to try to make this as "bite sized" as possible, because I tend to make unrealistic goals for my portfolio and I am always to busy with work.

The overall goal is to focus on smart and quick iteration and to let my personal art style shine through :)

PHASE 1: Pre-production (a.k.a sketches XD)



^ a sketch done to understand her anatomy and proportions 

^ a rushed attempt to get some morphs working in maya


Stay tuned! I'm going to post as much as I can for this project. It may seem lofty, but I've done it all before - I'm hoping it will be a quick one. (relatively, haha)

Replies

  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Phase 2 - Quick Zbrush Sketch 

    This isn't the final thing, but I want to get the entire piece working with mocap before I start to lose myself in details and too much design.




    Needless to say, this is basically a proxy - and she looks emaciated :confounded: 

    BUT - This piece is the truest to my style in a long time, so I am super stoked to be working on it  B)
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Phase 3.1 - Fail Fast, Fail Often (In Maya) lol



    I am creating some really *basic* morph targets for facial positions/emotions. As you can see, the first attempt was a bit of a disaster, but after some work, it is starting to take shape! (no pun intended)



    The next step should be to get all of this into a scene in UE4 and materials set up.
  • HarlequinWerewolf
    Online / Send Message
    HarlequinWerewolf polycounter
    Super interested to see where this project goes. I'm trying to get to grips with some tech art stuff so would be interesting to see your process
  • Fatos
    Offline / Send Message
    Fatos polycounter lvl 2
    Looks pretty cool, I  look forward to see the next step.
  • Marinc
    I love the idea !! Amazing work ! :DD
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    LeahM said:
    Super interested to see where this project goes. I'm trying to get to grips with some tech art stuff so would be interesting to see your process
    Thanks! Maybe I'll make my updates more tech-y to be more helpful  :)  

    Fatos said:
    Looks pretty cool, I  look forward to see the next step.
    Thank you!
    Marinc said:
    I love the idea !! Amazing work ! :DD
    Yay! Thank you!
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Phase 3.2 - Super Quick Mockup in UE4

    I feel like I should talk a bit more about my iteration philosophy for this project. 

    I tend towards perfectionism, and sometimes get stuck overworking details of a personal project.
    To counteract this, I am taking a more rapid approach to this project. I want to get it all working, and then I can iterate on how it looks until I'm happy <3

    Back to the mockup! 

    I have some facial animations captured with an iPhone app called Face Cap working with my model in Unreal! 
    I still have clipping and all sorts of issues, but it's a start ¯\_(ツ)_/¯



  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Unproductive Update #1

    I haven't been working on this as I would like, but I'm posting just to keep the ball rolling.

    My biggest accomplishment since the last post was getting her face to match the ZBrush sculpt a bit better - don't ask how I had messed this up XD 


    Before I do anything else with this model, I MUST add eyelashes. I shouldn't have come this far without them, but at least we can fix it!
  • Alemja
    Offline / Send Message
    Alemja quad damage
    Hey I saw this on twitter! I'm really loving the face proportions you have going on
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Alemja said:
    Hey I saw this on twitter! I'm really loving the face proportions you have going on
    Thank you so much!! Playing around with the proportions is the most fun part of this project so far :)
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Phase 3.3 - First Makeup Explorations



    This is the initial makeup exploration, I'm trying to get the concept down. Looking at it now, I realize I need to do something bolder with the design. 

    Besides this, I've been painstakingly refining her F.A.C.s. I hope to finish the second pass on that soon!

    Feel free to let me know your opinion on her makeup. I like the color of the lipstick in the first of the second row, and the darker-upper-lip style of the last one. I also kinda like the red eyeshadow.
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Phase 3.3 - Facial Refinements part 1

    Ezra was looking a bit flat and sketchy, so during one of my streams I took the time to refine her facial features a bit more. I wanted to make her secondary forms a bit more original, and slightly less human. I also took a foray into wrinkles :)

    Slow before-and-after gif:


    Screenshots:




    Hope you have a productive week!
  • HarlequinWerewolf
    Online / Send Message
    HarlequinWerewolf polycounter
    Wow, that makes a huge difference. She feels more solid and realistic. Love it
  • snorrow
    Offline / Send Message
    snorrow greentooth
    The after is much better :) Those details add a lot the the her expression
  • Carabiner
    Offline / Send Message
    Carabiner greentooth
    I really love what you're doing here -- excited to see your progress!
  • Crazy_pixel
    Offline / Send Message
    Crazy_pixel interpolator
    This character looks beautiful, can't wait to see the progress :)
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Wow, that makes a huge difference. She feels more solid and realistic. Love it
    snorrow said:
    The after is much better :) Those details add a lot the the her expression
    Carabiner said:
    I really love what you're doing here -- excited to see your progress!
    This character looks beautiful, can't wait to see the progress :)
    Thank you so much! I hope to have more progress soon :)
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Quick Paintover

    I decided to do a little photoshop work to see what the final render could look like. Screengrab from UE4!
    I think I went too dark. The leaves and background are placeholders from pinterest.
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Second Iteration - Refining Morphs

    Hello friends! I am spending the weekend refining the facial animations, and I am in dire need of objective, constructive criticism. A lot of her expressions look a little off, especially the smile.

    As of right now, I need to refine the awkward smile, the squint and that one morph in which the lower lip loses all volume.



    Thank you!
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Second Iteration - Refining Morphs

    Yet more refinements! I got a bunch of good feedback and did lots of it. The teeth are resized and repositioned, the brows are exaggerated (maybe too much), the smile affects the eyes and the lips retain their volume a bit more than before.

    I feel like I am ready to move on temporarily to create her skin shader, whith pores, SSS and whatnot! I will revisit morphs later on when I have more time to refine.




  • Crazy_pixel
    Offline / Send Message
    Crazy_pixel interpolator
    Your anims looking pretty sweet. Her smile looks also better than the first draft. I book I can recommend is https://anatomy4sculptors.com/collections/books/products/anatomy-of-facial-expression-paperback?gclid=Cj0KCQiA_rfvBRCPARIsANlV66Nut6DMRnkXuprcXv-43zVFEVy6JwehEtRrupbzeGVTTijULHCNV7caAvzEEALw_wcB

    There alre also a lot of free samples in the internet :)
  • ScoobyDoofus
    Offline / Send Message
    ScoobyDoofus polycounter lvl 19
    I haven't been back to polycount much in ages, much less commented...
    This is a really cool project!

    I didn't see it mentioned anywhere and I'm curious, are you using a bone-based rig at all or is this entirely driven by morph targets? Could this setup drive a face rig in UE4 in real-time or are there processing steps involved with the perf capture?
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Your anims looking pretty sweet. Her smile looks also better than the first draft. I book I can recommend is https://anatomy4sculptors.com/collections/books/products/anatomy-of-facial-expression-paperback?gclid=Cj0KCQiA_rfvBRCPARIsANlV66Nut6DMRnkXuprcXv-43zVFEVy6JwehEtRrupbzeGVTTijULHCNV7caAvzEEALw_wcB

    There alre also a lot of free samples in the internet :)
    Thank you so much for the recommendation! I will definitely use it and other books too :)
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    I haven't been back to polycount much in ages, much less commented...
    This is a really cool project!

    I didn't see it mentioned anywhere and I'm curious, are you using a bone-based rig at all or is this entirely driven by morph targets? Could this setup drive a face rig in UE4 in real-time or are there processing steps involved with the perf capture?

    @ScoobyDoofus

    Thank you!

    This is entirely driven by morph targets, although in later iterations I plan on adding a bone-based rig for the neck and head movements (and maybe ears, I haven't decided how to approach those yet).
    I have seen people use this same process to get the real-time perf capture in unreal, but I am keeping it simple and just manually importing the animation data from the app into unreal.

  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Refining the Teeth/Smile

    After a lot of iteration, I think I've gotten the teeth to look more natural in the mouth! The smile was by far the thing that was bothering me the most, and now that it's a little better I can focus on other issues. Don't mind the fact that she is now a ghostly floating head  =) 


    It's far from perfect, but here's the before and after for comparison:

    Next, we're going to make her eyes look a little less intense XD
  • pcaben
    Hi, what software/app you use for mocap? I know some app can use for facial motion capture of iphone x or above. Unfortunately, I bought Iphone 8 plus when they both release, so any software can do the same thing in  "normal" phone? 
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    @pcaben

    I am using a software called "FaceCap", and I do think it only works with iPhones that have newer camera systems. I started working on a similar project to this when I had an old iPhone, and I just borrowed my coworker's phone that was compatible with the app.

  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    First Dynamic Wrinkles!


    Had the opportunity today to prototype the dynamic material maps! I only have the Brow down ones so far, but I am excited to get to sit down tomorrow and apply more.

    Creepy demo of the dynamic maps:


    Don't mind the red polys! They have the wrong material ID assigned to them. 

    My current struggle is not having the time to work on personal projects, but it's all about trying to make even 1% progress whenever I can!
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    More Dynamic Wrinkles!



    I added subtle smile dynamic maps, you can see them at the corners of the mouth and cheeks when she smiles! (hopefully).

    On to the next 1% of progress!
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Tiny Update on Accessories!



    Spent part of the past two days coming up with better ways to make plants and jewelry! 

    The pendant was ridiculously easy, when I make the final I hope to also make a tutorial.
  • CybranM
    Offline / Send Message
    CybranM interpolator
    Dont have much feedback but it looks like its coming along great :smile:

  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Accessories Part 2


    The accessories are starting to look more like how I envisioned, but I am concerned that they are a little too distracting. Still, they are an important part of the character, so I will have to find balance.

    Time for a trip down memory lane!


    I really like looking at past iterations of whatever project I am working on. This gives me a better appreciation for how far I have come.
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    First Pass on Hair



    Hair is always a test of patience. I just put on a good documentary, grab a coffee, and trust in the process. This is the first pass, I need more flyaways and eventually to complete the bun in the back.

    I wanted to use xgen in Maya for the hair, but ended up manually placing each card in Zbrush with the twist deformer and regular deformer. They gave me more control than I expected!

    Please let me know what you think!
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    Hair and Brows



    Hair and Eyebrows are done!

    Next, I will revisit the skin texture and shader, I want more detail, variation and SSS.

    There's definitely something weird with the eyes, and I haven't been able to figure it out yet. I am afraid that any fixes are going to cause a domino effect with the motion capture.
  • birb
    Offline / Send Message
    birb interpolator
    Lovely design and pretty cool mocap experiment. This is coming along great! 

    There's definitely something weird with the eyes, and I haven't been able to figure it out yet. I am afraid that any fixes are going to cause a domino effect with the motion capture.
    It's the irises/pupils proportions. She's not looking too far up yet there's a lot of exposed sclera under the irises. The irises themselves look small because a lot of the eyeball is also exposed to the sides and she's got strong, wide features. Meanwhile, the pupils look too dilated under the chosen lighting.

    These characteristics would be consistent with a surprised, afraid or angry expression, yet she's got a serene smile. Such crossed signals in a person would make us uneasy in real life.

    There are also other small details that can help to soften her gaze. Her upper eyelashes are extending too far to the sides, clashing with the shape of the eyes and makeup. Cut them in the outer parts to gradually reduce them again, like people do sometimes with fake eyelashes, re-add a couple of lower lashes, and the eyes will sit better on the face without touching them or face geometry at all.

    The third aspect that would soften the face would be adding a meniscus mesh. Meniscus is that water line that accumulates between the eyeball and the eyelids. It can potentially soften the transition from one material to other and make the eyes look less dry and more lively. Now, I don't have the faintest idea how feasible this is in Unreal—I've been meaning to start learning it for ages!—so feel free to call me an idiot if that's not really possible. :D

    Here's a quick overpaint, if you don't mind:



    I hope this helps. Keep up the good work!
  • AnaCarolinaPereira
    Offline / Send Message
    AnaCarolinaPereira polycounter lvl 7
    @birb I can't believe I forgot to respond to you sooner! I had to pause my progress on this project for a while.

    Thank you SO MUCH for the detailed feedback! Everything you mentioned is on point and totally correct - thank you for helping me narrow down what's wrong with the eyes. Adding the meniscus will be challenging due to the animations that already exist, but it's important so it will be worth the trouble.

    Thanks again! Hope to update this thread soon with improvements :) 

  • GMLima
    Offline / Send Message
    GMLima polycounter lvl 3
    First Dynamic Wrinkles!


    Had the opportunity today to prototype the dynamic material maps! I only have the Brow down ones so far, but I am excited to get to sit down tomorrow and apply more.

    Creepy demo of the dynamic maps:


    Don't mind the red polys! They have the wrong material ID assigned to them. 

    My current struggle is not having the time to work on personal projects, but it's all about trying to make even 1% progress whenever I can!

    Hi Ana, i'm a beginner in 3d modeling for games and would like to know how do you do the dynamic wrinkles, or if you have a tutorial of you have follow to learn  to do. Sorry if i committed some grammatical error, i do not speak english very well, and thanks for the attention
Sign In or Register to comment.