Cinematic Teaser Using Photogrammetry [Bachelor Project]
polycounter lvl 11
LATEST PROGRESS:
Hello all
I have finally started my bachelor project here at darmstadt university 2 weeks ago - a kind of cinematic intro/teaser trailer for a non-existing game in a setting similar to the witcher or game of thrones.
On the technical side, the focus of my research thesis is photogrammetry, so I will do my best to capture as much as I can myself and render out the whole thing in Unreal Engine 4. My personal goal would be to have an environment at the end which showcases my current skill in environment art and figure out a good way to work with photogrammetry.
Oh, also this is a solo project (which probably means that the scope of this thing -might- be way too large) so I figured it would be a good idea to create a thread early in order to get some nice feedback.
Currently struggling with story related stuff as I suck at writing while doing research for the technical stuff.
Anyways, here's my current progress.
Character:
I started earlier with this one as I knew I would have a lot to do with the environment later on, so the model itself is almost finished except for hair, a dirt/wounds pass on the textures and the rigging.
Already posted most of my progress for this one in my learning diary.
Will pause this until I got some basic environment going.
Photogrammetry:
Did some capturing experiments with rocks, trees and ground textures so far, currently working on assembling a full tree for testing using a photogrammetry base and speedtree with a tileable bark extracted from the scan for the branches.
Most of these where captured using just a smartphone (samsung galaxy s5) as I do not have acces to a proper camera at the moment.
Foliage:
A special case is foliage as I quickly figured out that the results using regular photogrammetry look horrible for leafs and stuff.
So I spent the last week doing research and figuring out a way to capture these.
Currently I am using a technique called photometric stereo capture which, after some trial and error provides good results.
Still a bit more to figure out with this, also a proper camera instead of a smartphone should get me some more details.
Dead leaves are my first succesfull capture, the green ones are from yesterday.
Planning to also try my hands at capturing blades of grass using this technique the next days.
Environment:
Currently in blockout phase for the environment. Definetly will be outdoor. either a full forest or forest transitioning into grassland.
Final composition also depends on the story, which I unfortunately have not properly figured out yet so this might change.
Would really appreciate some feedback especially for the environment composition, even though it is still in a very early state.
That's the current progress, really excited to see what will come out of this.
Definetly got a lot to do in a short time frame
Will update again once I have some new stuff to show.
Hello all
I have finally started my bachelor project here at darmstadt university 2 weeks ago - a kind of cinematic intro/teaser trailer for a non-existing game in a setting similar to the witcher or game of thrones.
On the technical side, the focus of my research thesis is photogrammetry, so I will do my best to capture as much as I can myself and render out the whole thing in Unreal Engine 4. My personal goal would be to have an environment at the end which showcases my current skill in environment art and figure out a good way to work with photogrammetry.
Oh, also this is a solo project (which probably means that the scope of this thing -might- be way too large) so I figured it would be a good idea to create a thread early in order to get some nice feedback.
Currently struggling with story related stuff as I suck at writing while doing research for the technical stuff.
Anyways, here's my current progress.
Character:
I started earlier with this one as I knew I would have a lot to do with the environment later on, so the model itself is almost finished except for hair, a dirt/wounds pass on the textures and the rigging.
Already posted most of my progress for this one in my learning diary.
Will pause this until I got some basic environment going.
Photogrammetry:
Did some capturing experiments with rocks, trees and ground textures so far, currently working on assembling a full tree for testing using a photogrammetry base and speedtree with a tileable bark extracted from the scan for the branches.
Most of these where captured using just a smartphone (samsung galaxy s5) as I do not have acces to a proper camera at the moment.
Foliage:
A special case is foliage as I quickly figured out that the results using regular photogrammetry look horrible for leafs and stuff.
So I spent the last week doing research and figuring out a way to capture these.
Currently I am using a technique called photometric stereo capture which, after some trial and error provides good results.
Still a bit more to figure out with this, also a proper camera instead of a smartphone should get me some more details.
Dead leaves are my first succesfull capture, the green ones are from yesterday.
Planning to also try my hands at capturing blades of grass using this technique the next days.
Environment:
Currently in blockout phase for the environment. Definetly will be outdoor. either a full forest or forest transitioning into grassland.
Final composition also depends on the story, which I unfortunately have not properly figured out yet so this might change.
Would really appreciate some feedback especially for the environment composition, even though it is still in a very early state.
That's the current progress, really excited to see what will come out of this.
Definetly got a lot to do in a short time frame
Will update again once I have some new stuff to show.
Replies
@Doxturtle Thanks! Still working on improving the captures
Here's a small test I did with the last two leafs I scanned.
Rendered in Unreal, the plant is basically a cylinder with a few cards.
Also trying to figure out how to get a more cinematic look in Unreal..
Obviously all of them are super WIP.
Not too much progress the last days as I was still busy doing cinematography/story stuff and also got some sound people on board to help out with sound and music.
However I did start creating the snow, here is a small render test (still wip, think the subsurface scattering needs to be a bit more blueish and a little stronger)
Hopefully will have some kind of animatic soon so I can start working on the actual shots.
Also really need to get that hair done..
You have pretty good results with your smartphone. Can I know which application for photogrammetry do you use?
She definetly looks older now and the skin imperfections are a bit to strong. Also accidently changed the shape of the mouth a bit, will fix all these things tomorrow.
And I alsco scanned some more assets which I have not processed et except for this one:
Also started creating some ground textures.
Created (hopefully) all the blendshapes for the face as well as a stretched/compressed pose for the wrinkle maps.
Will probably have to adjust/redo quite a few of the shapes as I have never done a face rig before and have no idea what I am doing.
Combined expression for the wrinkle maps:
Kinda works but broke my lamp shortly after, so I will have to get a new light bulb before I can continue experimenting with scanned translucency textures.
Have a look at the most improvised and cheapest scanning setup ever:
scanned a bunch of foliage textures today, still processing a few of them. Here is the first plant I built from todays scans:
By the way, can you share any info on how to create your own custom scan setup? Or scanning objects, textures in general? Cheers!
@Elod.H Thanks, Glad you like it!
As mentioned above the setup for scanning flat textures/foliage is fairly simple and improvised, it is based on this blog post by allegorithmic:
https://www.allegorithmic.com/blog/your-smartphone-material-scanner
However I am not using Substance for processing as I found it hard to get stable results with that since the light angles have to match fairly well. Instead I am using a tool called Dabarti Capture:
http://dabarti.com/capture/
The advantage of this is that the tool is able to calculate the light direction based on a highlight which gives me the possibility to only shout roughly in 45° angles and not having to worry too much about being 100%accurate with the lighting direction.
However I am still planning to give Substance another try as well.
As this is also my theoretical bachelor topic I am planning to do a detailed write-up/tutorial of all my experiences and workflows sometime around february once I finished the whole thing
The full assets are just regular photogrammetry and processed in RealityCapture (which I find to be giving slightly more details than agisoft photoscan and has faster processing times. Disadvantage is that Agisoft is much better in aligning images, especially with less overlap).
Aside from this, just trying to look for cloudy/overcast weather and shooting enough images and have them overlap enough.
Delighting is currently done using the Unity Delighting tool:
https://github.com/Unity-Technologies/DeLightingTool
However I still have to look more into that, quite a few assets are not delight at all right now.
Since I currently have no proper access to a good camera, everything is captured with my phone (Galaxy S5) which gives fairly good results, but a proper camera would increase the overall resolution and quality a bit more.
They are meant to be blended together in engine, one providing the base and the other one to go on top in order to scatter leaves.
Processed in RealityCapture, adjusted in Substance Designer.
And finally got some lighting going which I kinda like even though it still requires some work:
(Finding it super hard to do outdoor overcast lighting which is not super boring)
Also imported my blendshapes and rig into unreal and made the wrinkle map blending work:
Have to go back and adjust most of the blendshapes. Also the eye shadow shader kinda broke in the video, no idea why.
The character is also looking solid, though I think the texturing could use a bit more love (particularly on the metal and leather bits).
Just a suggestion, but I think you should use a slightly cooler color grade. As of right now I'm not getting a cold winter type vibe that it looks like you're shooting for.
original
color corrected
Thanks for your feedback!
You are right about the scale, some of the plants feel way to big - have to adjust that.
Already thought about doing another pass of texturing on the character but I still have a lot to do and not too much time, so I have to finish some other stuff first before going into polishing mode.
Your color grading suggestion looks good - I am not too experienced with color grading and have to take another look at that.
Kinda happy with the current direction of the environment so I will get back to some character related stuff soon.
Also did some lighting for the eye-shot, should kinda be the final quality I am going for.
Now I just have to finish the rest
However here is a little preview - would love to get some feedback as I might still have some time to integrate it into the final shots
@MattN Thanks! I actually really like reality capture. It is easy to use, picks up more small details than agisoft photoscan and is really fast, even on my laptop with a gtx 860. However, the price is a major downside. I am currently using a 3 month subscription for my bachelor, but for further (personal) projects I am fairly sure that I will go back to using Agisoft, simply because it is way cheaper
Will be much simpler than the other one as I do not have too much time left.
Also another pose test of the character (still have to chreate final renders!)
Even though I was struggling a lot with this I learned a ton and am actually quite happy with the outcome!
More images on my artstation:
https://www.artstation.com/artwork/4r942
Finished the overall composition, now I -just- have to add all the missing assets and polish it up.
Starting to enter crunchtime, 1.5 weeks left until the final deadline and I still have to write a 60-pages thesis >.<
Updated the dude hanging on the tree with some more detail and finished all shots except for the last one (which is the most complex one since it shows the full environment). Here are a couple screenshots from the recent progress:
A chainmail-substance I made for the armor:
Here are a couple of screenshots from some of the shots with the hanging dude.
Thanks, glad you like it
Probably won't sleep tomorrow in order to polish it up as much as I can in the remaining 24 hours.
Here's the current version of the environment, UI will hopefully get a prettier version as well.
Here's an image of the finished final scene:
Will do some polishing now and then release the finished trailer in a couple of days.
Also, I wrote a large breakdown article for 80.lv in case you are interested in some breakdown material:
https://80.lv/articles/how-to-build-a-look-for-the-game/
Let me know what you think!
Overall this has probably been the hardest project I have done so far, but I am fairly happy with the outcome
Eyelashes are probably bothering me the most right now, but unfortunately I can't go back without redoing parts of the rig/animations..