Firbolg D&D Character - Unreal Mocap Demo
polycounter lvl 7
Hi there!
In an attempt to join my two favorite things - character art and technical art - I've decided to make my D&D character that never was, and to try to push the boundaries of what I can do with facial mocap in UE4, using the iPhone as my camera.
I am going to try to make this as "bite sized" as possible, because I tend to make unrealistic goals for my portfolio and I am always to busy with work.
The overall goal is to focus on smart and quick iteration and to let my personal art style shine through
PHASE 1: Pre-production (a.k.a sketches XD)
^ a sketch done to understand her anatomy and proportions
^ a rushed attempt to get some morphs working in maya
Stay tuned! I'm going to post as much as I can for this project. It may seem lofty, but I've done it all before - I'm hoping it will be a quick one. (relatively, haha)
In an attempt to join my two favorite things - character art and technical art - I've decided to make my D&D character that never was, and to try to push the boundaries of what I can do with facial mocap in UE4, using the iPhone as my camera.
I am going to try to make this as "bite sized" as possible, because I tend to make unrealistic goals for my portfolio and I am always to busy with work.
The overall goal is to focus on smart and quick iteration and to let my personal art style shine through
PHASE 1: Pre-production (a.k.a sketches XD)
^ a sketch done to understand her anatomy and proportions
^ a rushed attempt to get some morphs working in maya
Stay tuned! I'm going to post as much as I can for this project. It may seem lofty, but I've done it all before - I'm hoping it will be a quick one. (relatively, haha)
Replies
This isn't the final thing, but I want to get the entire piece working with mocap before I start to lose myself in details and too much design.
Needless to say, this is basically a proxy - and she looks emaciated
BUT - This piece is the truest to my style in a long time, so I am super stoked to be working on it
I am creating some really *basic* morph targets for facial positions/emotions. As you can see, the first attempt was a bit of a disaster, but after some work, it is starting to take shape! (no pun intended)
The next step should be to get all of this into a scene in UE4 and materials set up.
Thank you!
Yay! Thank you!
I feel like I should talk a bit more about my iteration philosophy for this project.
I tend towards perfectionism, and sometimes get stuck overworking details of a personal project.
To counteract this, I am taking a more rapid approach to this project. I want to get it all working, and then I can iterate on how it looks until I'm happy
Back to the mockup!
I have some facial animations captured with an iPhone app called Face Cap working with my model in Unreal!
I still have clipping and all sorts of issues, but it's a start ¯\_(ツ)_/¯
I haven't been working on this as I would like, but I'm posting just to keep the ball rolling.
My biggest accomplishment since the last post was getting her face to match the ZBrush sculpt a bit better - don't ask how I had messed this up XD
Before I do anything else with this model, I MUST add eyelashes. I shouldn't have come this far without them, but at least we can fix it!
This is the initial makeup exploration, I'm trying to get the concept down. Looking at it now, I realize I need to do something bolder with the design.
Besides this, I've been painstakingly refining her F.A.C.s. I hope to finish the second pass on that soon!
Feel free to let me know your opinion on her makeup. I like the color of the lipstick in the first of the second row, and the darker-upper-lip style of the last one. I also kinda like the red eyeshadow.
Ezra was looking a bit flat and sketchy, so during one of my streams I took the time to refine her facial features a bit more. I wanted to make her secondary forms a bit more original, and slightly less human. I also took a foray into wrinkles
Slow before-and-after gif:
Screenshots:
Hope you have a productive week!
I decided to do a little photoshop work to see what the final render could look like. Screengrab from UE4!
I think I went too dark. The leaves and background are placeholders from pinterest.
Hello friends! I am spending the weekend refining the facial animations, and I am in dire need of objective, constructive criticism. A lot of her expressions look a little off, especially the smile.
As of right now, I need to refine the awkward smile, the squint and that one morph in which the lower lip loses all volume.
Thank you!
Yet more refinements! I got a bunch of good feedback and did lots of it. The teeth are resized and repositioned, the brows are exaggerated (maybe too much), the smile affects the eyes and the lips retain their volume a bit more than before.
I feel like I am ready to move on temporarily to create her skin shader, whith pores, SSS and whatnot! I will revisit morphs later on when I have more time to refine.
There alre also a lot of free samples in the internet
This is a really cool project!
I didn't see it mentioned anywhere and I'm curious, are you using a bone-based rig at all or is this entirely driven by morph targets? Could this setup drive a face rig in UE4 in real-time or are there processing steps involved with the perf capture?
Thank you!
This is entirely driven by morph targets, although in later iterations I plan on adding a bone-based rig for the neck and head movements (and maybe ears, I haven't decided how to approach those yet).
I have seen people use this same process to get the real-time perf capture in unreal, but I am keeping it simple and just manually importing the animation data from the app into unreal.
After a lot of iteration, I think I've gotten the teeth to look more natural in the mouth! The smile was by far the thing that was bothering me the most, and now that it's a little better I can focus on other issues. Don't mind the fact that she is now a ghostly floating head
It's far from perfect, but here's the before and after for comparison:
Next, we're going to make her eyes look a little less intense XD
I am using a software called "FaceCap", and I do think it only works with iPhones that have newer camera systems. I started working on a similar project to this when I had an old iPhone, and I just borrowed my coworker's phone that was compatible with the app.
Had the opportunity today to prototype the dynamic material maps! I only have the Brow down ones so far, but I am excited to get to sit down tomorrow and apply more.
Creepy demo of the dynamic maps:
Don't mind the red polys! They have the wrong material ID assigned to them.
My current struggle is not having the time to work on personal projects, but it's all about trying to make even 1% progress whenever I can!
I added subtle smile dynamic maps, you can see them at the corners of the mouth and cheeks when she smiles! (hopefully).
On to the next 1% of progress!
Spent part of the past two days coming up with better ways to make plants and jewelry!
The pendant was ridiculously easy, when I make the final I hope to also make a tutorial.
Thank you so much!
The accessories are starting to look more like how I envisioned, but I am concerned that they are a little too distracting. Still, they are an important part of the character, so I will have to find balance.
Time for a trip down memory lane!
I really like looking at past iterations of whatever project I am working on. This gives me a better appreciation for how far I have come.
Hair is always a test of patience. I just put on a good documentary, grab a coffee, and trust in the process. This is the first pass, I need more flyaways and eventually to complete the bun in the back.
I wanted to use xgen in Maya for the hair, but ended up manually placing each card in Zbrush with the twist deformer and regular deformer. They gave me more control than I expected!
Please let me know what you think!
Hair and Eyebrows are done!
Next, I will revisit the skin texture and shader, I want more detail, variation and SSS.
There's definitely something weird with the eyes, and I haven't been able to figure it out yet. I am afraid that any fixes are going to cause a domino effect with the motion capture.
It's the irises/pupils proportions. She's not looking too far up yet there's a lot of exposed sclera under the irises. The irises themselves look small because a lot of the eyeball is also exposed to the sides and she's got strong, wide features. Meanwhile, the pupils look too dilated under the chosen lighting.
These characteristics would be consistent with a surprised, afraid or angry expression, yet she's got a serene smile. Such crossed signals in a person would make us uneasy in real life.
There are also other small details that can help to soften her gaze. Her upper eyelashes are extending too far to the sides, clashing with the shape of the eyes and makeup. Cut them in the outer parts to gradually reduce them again, like people do sometimes with fake eyelashes, re-add a couple of lower lashes, and the eyes will sit better on the face without touching them or face geometry at all.
The third aspect that would soften the face would be adding a meniscus mesh. Meniscus is that water line that accumulates between the eyeball and the eyelids. It can potentially soften the transition from one material to other and make the eyes look less dry and more lively. Now, I don't have the faintest idea how feasible this is in Unreal—I've been meaning to start learning it for ages!—so feel free to call me an idiot if that's not really possible.
Here's a quick overpaint, if you don't mind:
I hope this helps. Keep up the good work!
Thank you SO MUCH for the detailed feedback! Everything you mentioned is on point and totally correct - thank you for helping me narrow down what's wrong with the eyes. Adding the meniscus will be challenging due to the animations that already exist, but it's important so it will be worth the trouble.
Thanks again! Hope to update this thread soon with improvements
Hi Ana, i'm a beginner in 3d modeling for games and would like to know how do you do the dynamic wrinkles, or if you have a tutorial of you have follow to learn to do. Sorry if i committed some grammatical error, i do not speak english very well, and thanks for the attention