Cinematic Teaser Using Photogrammetry [Bachelor Project]

polycounter lvl 7
Offline / Send Message
Shyralon polycounter lvl 7
LATEST PROGRESS:


 Hello all :)

I have finally started my bachelor project here at darmstadt university 2 weeks ago - a kind of cinematic intro/teaser trailer for a non-existing game in a setting similar to the witcher or game of thrones.
On the technical side, the focus of my research thesis is photogrammetry, so I will do my best to capture as much as I can myself and render out the whole thing in Unreal Engine 4. My personal goal would be to have an environment at the end which showcases my current skill in environment art and figure out a good way to work with photogrammetry.

Oh, also this is a solo project (which probably means that the scope of this thing -might- be way too large) so I figured it would be a good idea to create a thread early in order to get some nice feedback.
Currently struggling with story related stuff as I suck at writing while doing research for the technical stuff.

Anyways, here's my current progress.
Character:


I started earlier with this one as I knew I would have a lot to do with the environment later on, so the model itself is almost finished except for hair, a dirt/wounds pass on the textures and the rigging.
Already posted most of my progress for this one in my learning diary.
Will pause this until I got some basic environment going.

Photogrammetry:
Did some capturing experiments with rocks, trees and ground textures so far, currently working on assembling a full tree for testing using a photogrammetry base and speedtree with a tileable bark extracted from the scan for the branches.




Most of these where captured using just a smartphone (samsung galaxy s5) as I do not have acces to a proper camera at the moment.

Foliage:
A special case is foliage as I quickly figured out that the results using regular photogrammetry look horrible for leafs and stuff.
So I spent the last week doing research and figuring out a way to capture these.
Currently  I am using a technique called photometric stereo capture which, after some trial and error provides good results.
Still a bit more to figure out with this, also a proper camera instead of a smartphone should get me some more details.
Dead leaves are my first succesfull capture, the green ones are from yesterday.
Planning to also try my hands at capturing blades of grass using this technique the next days.



Environment:
Currently in blockout phase for the environment. Definetly will be outdoor. either a full forest or forest transitioning into grassland.
Final composition also depends on the story, which I unfortunately have not properly figured out yet so this might change.
Would really appreciate some feedback especially for the environment composition, even though it is still in a very early state.


That's the current progress, really excited to see what will come out of this.
Definetly got a lot to do in a short time frame D:
Will update again once I have some new stuff to show.

Replies

  • stororokw
    Offline / Send Message
    stororokw polycounter lvl 2
    Looking forward to what you will be able to achieve. Unfortunately, I cant say anything about the composition. How much time do you have?

  • Doxturtle
    Offline / Send Message
    Doxturtle polycounter lvl 4
    great amount of detail you managed to capture
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @stororokw Thanks for your comment! I have until end of january to finish the whole thing, but I will also need some time out of this to write the actual thesis.. so all in all something around 3-4 months?
    @Doxturtle Thanks! Still working on improving the captures :)

    Here's  a small test I did with the last two leafs I scanned.
    Rendered in Unreal, the plant is basically a cylinder with a few cards.

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Did some test shots and compositions to get some ideas, here are a couple ones I kinda like.
    Also trying to figure out how to get a more cinematic look in Unreal..
    Obviously all of them are super WIP.


  • FreneticPonies
    Offline / Send Message
    FreneticPonies polycounter lvl 3
    Well that looks fantastic! Hope you abuse the hell out of Unreal's automated LOD creation so you can keep as much of that detail as possible. Looking forward to more.
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @FreneticPonies Thanks! As I am planning to render it out at the end I only have to make it run at a performance which is workable for me, so hopefully I can get as much detail as possible :)

    Not too much progress the last days as I was still busy doing cinematography/story stuff and also got some sound people on board to help out with sound and music.
    However I did start creating the snow, here is a small render test (still wip, think the subsurface scattering needs to be a bit more blueish and a little stronger)

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    And even more test renders.
    Hopefully will have some kind of animatic soon so I can start working on the actual shots.
    Also really need to get that hair done..


  • Go2frag
    Offline / Send Message
    Go2frag polycounter lvl 4
    Those stuff looks amazing! Topic added to watched!
    You have pretty good results with your smartphone. Can I know which application for photogrammetry do you use? 
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @Go2frag Thanks! I am using RealityCapture to process the data and an app called Camera FV-5 on my phone for shooting the photos (which basically just gives more option to controll the camera itself compared to the standard android camera app).
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Retextured parts of the face again. Have been staring at this thing for 8 hours straight now, can't tell anymore if it is getting better or worse.
    She definetly looks older now and the skin imperfections are a bit to strong. Also accidently changed the shape of the mouth a bit, will fix all these things tomorrow.

    And I alsco scanned some more assets which I have not processed et except for this one:


  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Started working on the dirt and blood of the face and also created a shader which allows me to dynamically change and adjust the amount of dirt/blud in-engine. Also, some smaller texture tweaks on the face.


  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Integrated the new version of the face into my current animatic, here is a little testrender:

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Progress is rather slow, but here is another lighting/color grading test.
    Also started creating some ground textures.

  • FreneticPonies
    Offline / Send Message
    FreneticPonies polycounter lvl 3
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @FreneticPonies Thanks!

    Created (hopefully) all the blendshapes for the face as well as a stretched/compressed pose for the wrinkle maps.
    Will probably have to adjust/redo quite a few of the shapes as I have never done a face rig before and have no idea what I am doing.

    Combined expression for the wrinkle maps:


  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Updated my scan setup to also be able to capture translucency for foliage.
    Kinda works but broke my lamp shortly after, so I will have to get a new light bulb before I can continue experimenting with scanned translucency textures.
    Have a look at the most improvised and cheapest scanning setup ever:


    scanned a bunch of foliage textures today, still processing a few of them. Here is the first plant I built from todays scans:




  • Doxturtle
    Offline / Send Message
    Doxturtle polycounter lvl 4
    Did you use photogrammetry for the character? If so, would you mind explaining how you got it working so well :)
  • Elod.H
    Offline / Send Message
    Elod.H polycounter lvl 6
    Wow, just noticed this thread. This is shaping up to be awesome. Keep it up, can't wait to see this progress!
    By the way, can you share any info on how to create your own custom scan setup? Or scanning objects, textures in general? Cheers!
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @Doxturtle I did not use photogrammetry for the character, everything was sculpted by hand in zBrush and textured using Quixel Suite and Photoshop. I did some smaller experiments with photogrammetry for characters but the time needed to clean up the scan in combination with not having the armor and gear available to scan was the reason I decided pretty early on to do it all by hand :smile:

    @Elod.H Thanks, Glad you like it!
    As mentioned above the setup for scanning flat textures/foliage is fairly simple and improvised, it is based on this blog post by allegorithmic:
    https://www.allegorithmic.com/blog/your-smartphone-material-scanner
    However I am not using Substance for processing as I found it hard to get stable results with that since the light angles have to match fairly well. Instead I am using a tool called Dabarti Capture:
    http://dabarti.com/capture/
    The advantage of this is that the tool is able to calculate the light direction based on a highlight which gives me the possibility to only shout roughly in 45° angles and not having to worry too much about being 100%accurate with the lighting direction.
    However I am still planning to give Substance another try as well.
    As this is also my theoretical bachelor topic I am planning to do a detailed write-up/tutorial of all my experiences and workflows sometime around february once I finished the whole thing :)

    The full assets are just regular photogrammetry and processed in RealityCapture (which I find to be giving slightly more details than agisoft photoscan and has faster processing times. Disadvantage is that Agisoft is much better in aligning images, especially with less overlap).
    Aside from this, just trying to look for cloudy/overcast weather and shooting enough images and have them overlap enough.
    Delighting is currently done using the Unity Delighting tool:
    https://github.com/Unity-Technologies/DeLightingTool
    However I still have to look more into that, quite a few assets are not delight at all right now.

    Since I currently have no proper access to a good camera, everything is captured with my phone (Galaxy S5) which gives fairly good results, but a proper camera would increase the overall resolution and quality a bit more.



  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Created a little test scene using some of the newer assets:

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Created tileable photogrammetry ground textures.
    They are meant to be blended together in engine, one providing the base and the other one to go on top in order to scatter leaves.
    Processed in RealityCapture, adjusted in Substance Designer.

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Created a master material for the ground which allows me to blend the above textures along with some snow through vertex painting:
    And finally got some lighting going which I kinda like even though it still requires some work:
    (Finding it super hard to do outdoor overcast lighting which is not super boring)

    Also imported my blendshapes and rig into unreal and made the wrinkle map blending work:

    Have to go back and adjust most of the blendshapes. Also the eye shadow shader kinda broke in the video, no idea why.


  • Jack M.
    Offline / Send Message
    Jack M. interpolator
    Really digging your stuff so far. I think some of the scale is a bit odd though. For example the leaves on the ground look comparatively very small compared to the surrounding foliage. Besides that the snow looks phenomenal. 

    The character is also looking solid, though I think the texturing could use a bit more love (particularly on the metal and leather bits).


    Just a suggestion, but I think you should use a slightly cooler color grade. As of right now I'm not getting a cold winter type vibe that it looks like you're shooting for.

    original

    color corrected

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @Jack M.
    Thanks for your feedback!
    You are right about the scale, some of the plants feel way to big - have to adjust that.
    Already thought about doing another pass of texturing on the character but I still have a lot to do and not too much time, so I have to finish some other stuff first before going into polishing mode.
    Your color grading suggestion looks good - I am not too experienced with color grading and have to take another look at that.
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Reworked the lighting and atmosphere even more, built a couple new assets.
    Kinda happy with the current direction of the environment so I will get back to some character related stuff soon.


  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Finished the hair, was a huge challenge for me as I have not done any realtime hair before and placing haircards really sucks >.>
    Also did some lighting for the eye-shot, should kinda be the final quality I am going for.
    Now I just have to finish the rest :)



  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    It has been a while since I last posted - I finished the character and the rig now, and am currently in the process of posing and doing final presentation shots for the character.
    However here is a little preview - would love to get some feedback as I might still have some time to integrate it into the final shots :)


  • honglong99
    Offline / Send Message
    honglong99 vertex
    Nice artwork
  • MattN
    Offline / Send Message
    MattN triangle
    Dam some really nice results, the leaves and ground materials in particular came out really well. Really impressed you are getting these results from a phone too! How do you find reality capture? From personal experance I have found it to be the best solution, just sucks it has a really expensive subscription model.

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @honglong99 Thank you!
    @MattN Thanks! I actually really like reality capture. It is easy to use, picks up more small details than agisoft photoscan and is really fast, even on my laptop with a gtx 860. However, the price is a major downside. I am currently using a 3 month subscription for my bachelor, but for further (personal) projects I am fairly sure that I will go back to using Agisoft, simply because it is way cheaper :/
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Started a blockout for the second "character", the body hanging on the tree.
    Will be much simpler than the other one as I do not have too much time left.

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Scanned some rocks yesterday and put together a small scene with them. The top of the rock still needs cleanup and I need to create a proper tree with some roots growing down but other than that it is slowly coming along.

    Also another pose test of the character (still have to chreate final renders!)

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Finally finished the character!
    Even though I was struggling a lot with this I learned a ton and am actually quite happy with the outcome!
    More images on my artstation:
    https://www.artstation.com/artwork/4r942


  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Now that the character is finished I went back to working on the environment.
    Finished the overall composition, now I -just- have to add all the missing assets and polish it up.

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    It's been a while since I last posted progress.
    Starting to enter crunchtime, 1.5 weeks left until the final deadline and I still have to write a 60-pages thesis >.<
    Updated the dude hanging on the tree with some more detail and finished all shots except for the last one (which is the most complex one since it shows the full environment). Here are a couple screenshots from the recent progress:

    A chainmail-substance I made for the armor:




  • MattN
    Offline / Send Message
    MattN triangle
    mail is looking pretty nice, environment came together really well too :)  The bush looks a little too yellow to me though like it has a hard time standing out from the grass maybe add just a little more colour to it?
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Thanks for your comment! You might be right about the bush. I don't want them to stand out too much, but a little difference could be nice, will try it later!
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Deadline is next week and I am almost done, only one shot is missing.
    Here are a couple of screenshots from some of the shots with the hanging dude.




  • ibrahim05
    wow! amazing Job!
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @ibrahim05
    Thanks, glad you like it :)

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Finished writing my thesis, one more day until the deadline!
    Probably won't sleep tomorrow in order to polish it up as much as I can in the remaining 24 hours.
    Here's the current version of the environment, UI will hopefully get a prettier version as well.

  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Finished the project and handed it in today!
    Here's an image of the finished final scene:

    Will do some polishing now and then release the finished trailer in a couple of days.
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    Finally finished the project, take a look at the final result:


    Also, I wrote a large breakdown article for 80.lv in case you are interested in some breakdown material:
    https://80.lv/articles/how-to-build-a-look-for-the-game/

    Let me know what you think!
    Overall this has probably been the hardest project I have done so far, but I am fairly happy with the outcome :)

  • FreneticPonies
    Offline / Send Message
    FreneticPonies polycounter lvl 3
    The end result is fantastic, with some points sticking out. The face is great, except for the eyelashes which stick out a lot. Pulling out the camera makes the assets look great, but there's definitely too much close camera. At times your video res is exceeding the material's resolution by a wide amount, the close ins can just look blurry even if it looks great from farther away. The grass also stick out as highly repetitive and odd looking in the final pullout. I don't know exactly what's going on there, but the rest of the scene looks good while the grass stick out as odd.
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @FreneticPonies Thanks for your feedback! I agree with you on the repeating grass, would have probably been necessary to create a couple variations - simply had no time left for that at the end :(
    Eyelashes are probably bothering me the most right now, but unfortunately I can't go back without redoing parts of the rig/animations..
  • tarek56
    Offline / Send Message
    tarek56 triangle
    Hi great work i really like it he inspire me lot thanks again 
  • Shyralon
    Offline / Send Message
    Shyralon polycounter lvl 7
    @tarek56 Thanks a lot, glad you like it :)
Sign In or Register to comment.